2024-03-26T18:33:45.001 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-26T18:33:45.002 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_lflores@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/postgres}} --name lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi --archive /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817 -- /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817/orig.config.yaml 2024-03-26T18:33:45.058 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T18:33:45.161 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817 branch: wip-batrick-testing-20240320.145742 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/postgres}} email: null first_in_suite: false job_id: '7623817' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240320.145742 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - \(PG_DEGRADED - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 360516069d9393362c4cc6eb9371680fe16d66ab subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 360516069d9393362c4cc6eb9371680fe16d66ab kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-tracker-65018 sha1: 72e2437271dcafb9826f2f49bb1001426fa6de81 owner: scheduled_lflores@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4067 sha1: 360516069d9393362c4cc6eb9371680fe16d66ab sleep_before_teardown: 0 subset: 502/512 suite: fs suite_branch: wip-tracker-65018 suite_path: /home/teuthworker/src/github.com_ljflores_ceph_72e2437271dcafb9826f2f49bb1001426fa6de81/qa suite_relpath: qa suite_repo: https://github.com/ljflores/ceph.git suite_sha1: 72e2437271dcafb9826f2f49bb1001426fa6de81 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPXlOA3/hBjX3/UQCMdIc5Kxd+dwYhKV8kmzAPyFhNV1/16javNIxvKzL/6xmEIH+E05aA2BJsJRk+ixSyyKRAI= smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGNqjiTX4EDAoMqp7oUa40hM1jCxmPcujTGj3Lozueao4FyU22SfY4slt1lzC5riz62Gi6SLNeifivbsHUWzD1o= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJiep7pffesW1SUf7ISWXkdHXqUioyvL9H9E9ZF5bhCQeyN7YaNCXoEDefv+0J7Wu99Vv6+gxSoISwXRBVaMDyI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-26_17:10:07 tube: smithi user: lflores verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-26T18:33:45.161 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ljflores_ceph_72e2437271dcafb9826f2f49bb1001426fa6de81/qa; will attempt to use it 2024-03-26T18:33:45.162 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ljflores_ceph_72e2437271dcafb9826f2f49bb1001426fa6de81/qa/tasks 2024-03-26T18:33:45.162 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-26T18:33:45.164 INFO:teuthology.task.internal:Checking packages... 2024-03-26T18:33:45.187 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '360516069d9393362c4cc6eb9371680fe16d66ab' 2024-03-26T18:33:45.187 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-26T18:33:45.187 INFO:teuthology.packaging:ref: None 2024-03-26T18:33:45.187 INFO:teuthology.packaging:tag: None 2024-03-26T18:33:45.187 INFO:teuthology.packaging:branch: wip-batrick-testing-20240320.145742 2024-03-26T18:33:45.187 INFO:teuthology.packaging:sha1: 360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:33:45.187 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240320.145742 2024-03-26T18:33:45.325 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2311.g36051606 2024-03-26T18:33:45.327 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-26T18:33:45.335 INFO:teuthology.task.internal:no buildpackages task found 2024-03-26T18:33:45.335 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-26T18:33:45.341 INFO:teuthology.task.internal:Saving configuration 2024-03-26T18:33:45.362 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-26T18:33:45.367 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T18:33:45.390 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 18:28:56.892677', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPXlOA3/hBjX3/UQCMdIc5Kxd+dwYhKV8kmzAPyFhNV1/16javNIxvKzL/6xmEIH+E05aA2BJsJRk+ixSyyKRAI='} 2024-03-26T18:33:45.414 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 18:28:56.890783', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGNqjiTX4EDAoMqp7oUa40hM1jCxmPcujTGj3Lozueao4FyU22SfY4slt1lzC5riz62Gi6SLNeifivbsHUWzD1o='} 2024-03-26T18:33:45.436 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 18:28:56.894297', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJiep7pffesW1SUf7ISWXkdHXqUioyvL9H9E9ZF5bhCQeyN7YaNCXoEDefv+0J7Wu99Vv6+gxSoISwXRBVaMDyI='} 2024-03-26T18:33:45.436 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-26T18:33:45.442 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-26T18:33:45.443 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-26T18:33:45.443 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-26T18:33:45.443 INFO:teuthology.run_tasks:Running task console_log... 2024-03-26T18:33:45.596 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f56f892a160>, signals=[15]) 2024-03-26T18:33:45.596 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-26T18:33:45.602 INFO:teuthology.task.internal:Opening connections... 2024-03-26T18:33:45.602 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-03-26T18:33:45.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T18:33:45.694 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-03-26T18:33:45.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T18:33:45.781 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-03-26T18:33:45.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T18:33:45.869 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-26T18:33:45.876 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-03-26T18:33:45.894 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-03-26T18:33:45.894 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-03-26T18:33:45.951 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T18:33:45.952 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T18:33:45.953 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-03-26T18:33:45.979 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-03-26T18:33:45.996 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-03-26T18:33:45.996 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-03-26T18:33:46.052 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T18:33:46.053 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T18:33:46.053 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T18:33:46.053 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-03-26T18:33:46.079 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-03-26T18:33:46.097 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-03-26T18:33:46.097 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-03-26T18:33:46.154 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-03-26T18:33:46.154 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T18:33:46.155 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T18:33:46.156 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-03-26T18:33:46.179 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-26T18:33:46.189 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-26T18:33:46.195 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-26T18:33:46.195 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-03-26T18:33:46.198 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-03-26T18:33:46.200 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-03-26T18:33:46.216 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-26T18:33:46.222 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-26T18:33:46.222 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-03-26T18:33:46.255 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-03-26T18:33:46.257 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-03-26T18:33:46.287 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-26T18:33:46.359 INFO:teuthology.run_tasks:Running task kernel... 2024-03-26T18:33:46.373 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-26T18:33:46.374 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-26T18:33:46.374 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-26T18:33:46.374 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T18:33:46.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:33:46.391 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-26T18:33:46.706 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T18:33:46.707 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-26T18:33:46.707 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-03-26T18:33:46.724 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-361.el9.x86_64 2024-03-26T18:33:46.724 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T18:33:46.724 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-26T18:33:46.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum upgrade -y linux-firmware 2024-03-26T18:33:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-26T18:33:50.416 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-03-26T18:33:51.556 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-03-26T18:33:54.441 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-03-26T18:34:07.202 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 678 kB/s | 6.1 MB 00:09 2024-03-26T18:34:08.600 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 94 kB/s | 16 kB 00:00 2024-03-26T18:34:10.001 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-26T18:34:14.876 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-03-26T18:34:17.151 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-26T18:34:17.161 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 1 Package 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 375 M 2024-03-26T18:34:17.162 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-26T18:34:24.277 INFO:teuthology.orchestra.run.smithi097.stdout:linux-firmware-20240219-143.el9.noarch.rpm 53 MB/s | 375 MB 00:07 2024-03-26T18:34:24.278 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-26T18:34:24.279 INFO:teuthology.orchestra.run.smithi097.stdout:Total 53 MB/s | 375 MB 00:07 2024-03-26T18:34:24.279 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-26T18:34:24.371 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-26T18:34:24.371 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-26T18:34:26.737 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-26T18:34:26.738 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-26T18:34:33.088 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-26T18:34:33.320 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T18:34:34.727 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T18:34:34.727 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T18:34:35.117 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T18:34:35.117 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:34:35.117 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-03-26T18:34:35.117 INFO:teuthology.orchestra.run.smithi097.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-26T18:34:35.117 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:34:35.117 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-26T18:34:35.416 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-03-26T18:34:35.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T18:34:35.640 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-26T18:34:35.640 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-26T18:34:35.643 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi097.front.sepia.ceph.com to complete... 2024-03-26T18:34:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26 18:34:47 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-26T18:34:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:FINISHED --2024-03-26 18:34:47-- 2024-03-26T18:34:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:Total wall clock time: 11s 2024-03-26T18:34:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:Downloaded: 1 files, 321M in 11s (28.9 MB/s) 2024-03-26T18:34:47.062 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-03-26T18:34:47.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-26T18:34:48.994 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying... ######################################## 2024-03-26T18:34:49.120 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing... ######################################## 2024-03-26T18:34:49.199 INFO:teuthology.orchestra.run.smithi097.stdout:Updating / installing... 2024-03-26T18:34:56.645 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-26T18:35:12.788 INFO:teuthology.orchestra.run.smithi097.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T18:35:35.157 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-26T18:35:35.158 DEBUG:teuthology.orchestra.run.smithi097:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-26T18:35:35.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-26T18:35:35.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-26T18:35:35.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-26T18:35:35.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-26T18:35:35.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-26T18:35:35.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-26T18:35:35.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-26T18:35:35.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-26T18:35:35.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-26T18:35:35.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-26T18:35:35.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-26T18:35:35.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-26T18:35:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-26T18:35:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-26T18:35:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-26T18:35:35.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-26T18:35:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-26T18:35:35.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-26T18:35:35.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-26T18:35:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-26T18:35:35.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-26T18:35:35.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-26T18:35:35.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-26T18:35:35.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-26T18:35:35.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-26T18:35:35.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-26T18:35:35.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-26T18:35:35.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-26T18:35:35.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-26T18:35:35.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-26T18:35:35.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-26T18:35:35.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-26T18:35:35.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-26T18:35:35.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-26T18:35:35.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-26T18:35:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-26T18:35:35.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-26T18:35:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-26T18:35:35.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-26T18:35:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-26T18:35:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-26T18:35:35.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-26T18:35:35.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-26T18:35:35.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-26T18:35:35.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-26T18:35:35.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-26T18:35:35.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-26T18:35:35.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-26T18:35:35.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-26T18:35:35.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-26T18:35:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-26T18:35:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-26T18:35:35.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-26T18:35:35.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-26T18:35:35.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-26T18:35:35.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-26T18:35:35.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-26T18:35:35.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-26T18:35:35.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-26T18:35:35.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-26T18:35:35.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-26T18:35:35.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-26T18:35:35.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-26T18:35:35.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-26T18:35:35.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-26T18:35:35.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-26T18:35:35.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-26T18:35:35.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-26T18:35:35.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-26T18:35:35.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-26T18:35:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-26T18:35:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-26T18:35:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-26T18:35:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-26T18:35:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-26T18:35:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-26T18:35:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-26T18:35:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-26T18:35:35.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-26T18:35:35.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-26T18:35:35.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-26T18:35:35.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-26T18:35:35.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-26T18:35:35.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-26T18:35:35.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-26T18:35:35.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-26T18:35:35.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-26T18:35:35.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-26T18:35:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-26T18:35:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-26T18:35:35.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-26T18:35:35.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-26T18:35:35.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-26T18:35:35.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-26T18:35:35.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-26T18:35:35.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-26T18:35:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-26T18:35:35.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-26T18:35:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-26T18:35:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-26T18:35:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-26T18:35:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-26T18:35:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-26T18:35:35.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-26T18:35:35.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-26T18:35:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-26T18:35:35.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-26T18:35:35.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-26T18:35:35.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-26T18:35:35.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-26T18:35:35.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-26T18:35:35.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-26T18:35:35.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-26T18:35:35.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-26T18:35:35.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-26T18:35:35.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-26T18:35:35.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-26T18:35:35.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-26T18:35:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-26T18:35:35.351 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-26T18:35:35.352 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-26T18:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-26T18:35:35.354 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-26T18:35:35.355 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-26T18:35:35.356 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-26T18:35:35.357 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-26T18:35:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-26T18:35:35.359 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-26T18:35:35.360 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-26T18:35:35.361 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-26T18:35:35.362 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-26T18:35:35.363 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-26T18:35:35.364 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-26T18:35:35.365 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-26T18:35:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-26T18:35:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-26T18:35:35.368 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-26T18:35:35.369 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-26T18:35:35.370 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-26T18:35:35.371 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-26T18:35:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-26T18:35:35.373 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-26T18:35:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-26T18:35:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-26T18:35:35.376 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-26T18:35:35.377 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-26T18:35:35.378 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-26T18:35:35.379 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-26T18:35:35.380 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-26T18:35:35.381 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-26T18:35:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-26T18:35:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-26T18:35:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-26T18:35:35.385 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-26T18:35:35.386 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-26T18:35:35.387 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-26T18:35:35.388 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-26T18:35:35.389 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-26T18:35:35.390 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-26T18:35:35.390 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-26T18:35:35.390 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-26T18:35:35.390 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-26T18:35:35.390 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-26T18:35:35.390 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-26T18:35:35.390 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-26T18:35:35.390 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-26T18:35:35.390 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T18:35:35.394 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-26T18:35:35.394 DEBUG:teuthology.orchestra.run.smithi097:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout:postinstall scriptlet (using /bin/sh): 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout:for file in vmlinuz System.map config; do 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout: fi 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout:done 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-26T18:35:35.417 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-26T18:35:35.418 INFO:teuthology.orchestra.run.smithi097.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-26T18:35:35.418 INFO:teuthology.orchestra.run.smithi097.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-26T18:35:35.418 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-26T18:35:35.418 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-03-26T18:35:35.418 INFO:teuthology.orchestra.run.smithi097.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-26T18:35:35.418 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-26T18:35:35.418 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-26T18:35:35.418 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-03-26T18:35:35.419 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -qi grub2-tools 2024-03-26T18:35:35.495 INFO:teuthology.orchestra.run.smithi097.stdout:Name : grub2-tools 2024-03-26T18:35:35.495 INFO:teuthology.orchestra.run.smithi097.stdout:Epoch : 1 2024-03-26T18:35:35.495 INFO:teuthology.orchestra.run.smithi097.stdout:Version : 2.06 2024-03-26T18:35:35.495 INFO:teuthology.orchestra.run.smithi097.stdout:Release : 68.el9 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: x86_64 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Group : Unspecified 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Size : 8264254 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:License : GPLv3+ 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Packager : builder@centos.org 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Vendor : CentOS 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T18:35:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:Summary : Support tools for GRUB. 2024-03-26T18:35:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:Description : 2024-03-26T18:35:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:35:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T18:35:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T18:35:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T18:35:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:hardware devices. 2024-03-26T18:35:35.497 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:35:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T18:35:35.498 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T18:35:35.498 INFO:teuthology.task.kernel:Updating grub on smithi097 to boot 6.8.0-g1df0c7e8a235 2024-03-26T18:35:35.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T18:35:36.117 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-03-26T18:35:37.472 INFO:teuthology.orchestra.run.smithi097.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T18:35:37.511 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-03-26T18:35:37.513 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-03-26T18:35:37.528 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.wHrS93DrNe 2024-03-26T18:35:37.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wHrS93DrNe 2024-03-26T18:35:37.613 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.wHrS93DrNe 2024-03-26T18:35:37.752 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.wHrS93DrNe is 6KB 2024-03-26T18:35:37.801 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.wHrS93DrNe 2024-03-26T18:35:37.816 DEBUG:teuthology.orchestra.run.smithi097:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T18:35:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T18:35:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T18:35:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T18:35:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-26T18:35:37.889 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-26T18:35:38.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-03-26T18:36:08.049 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-26T18:36:08.051 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-26T18:36:08.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T18:36:26.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-03-26T18:36:35.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-26T18:36:35.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T18:36:38.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-03-26T18:36:50.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-26T18:36:50.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T18:37:22.671 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-26T18:37:23.346 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-26T18:37:23.346 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g1df0c7e8a235"... 2024-03-26T18:37:23.347 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-03-26T18:37:23.389 INFO:teuthology.orchestra.run.smithi097.stdout:6.8.0-g1df0c7e8a235 2024-03-26T18:37:23.389 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-26T18:37:23.390 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T18:37:23.390 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T18:37:24.390 DEBUG:teuthology.parallel:result is None 2024-03-26T18:37:24.391 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-26T18:37:24.398 INFO:teuthology.task.internal:Creating test directory... 2024-03-26T18:37:24.399 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T18:37:24.404 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T18:37:24.408 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T18:37:24.445 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-26T18:37:24.452 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-26T18:37:24.462 INFO:teuthology.task.internal:Creating archive directory... 2024-03-26T18:37:24.462 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T18:37:24.491 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T18:37:24.493 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T18:37:24.597 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-26T18:37:24.605 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-26T18:37:24.605 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T18:37:24.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:37:24.622 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T18:37:24.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:37:24.639 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T18:37:24.643 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T18:37:24.665 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T18:37:24.705 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T18:37:24.720 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T18:37:24.733 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T18:37:24.756 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T18:37:24.774 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T18:37:24.865 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T18:37:24.869 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-26T18:37:24.875 INFO:teuthology.task.internal:Configuring sudo... 2024-03-26T18:37:24.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T18:37:24.879 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T18:37:24.882 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T18:37:24.946 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-26T18:37:24.956 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-26T18:37:24.956 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T18:37:24.961 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T18:37:24.963 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T18:37:24.996 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T18:37:25.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T18:37:25.197 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T18:37:25.266 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T18:37:25.355 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:37:25.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T18:37:25.444 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T18:37:25.481 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T18:37:25.545 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T18:37:25.623 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T18:37:25.674 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:37:25.675 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T18:37:25.743 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T18:37:25.788 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T18:37:25.847 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T18:37:25.925 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T18:37:25.976 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:37:25.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T18:37:26.045 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-03-26T18:37:26.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-03-26T18:37:26.054 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-03-26T18:37:26.093 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T18:37:26.127 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T18:37:26.155 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T18:37:26.659 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-26T18:37:26.667 INFO:teuthology.task.internal:Starting timer... 2024-03-26T18:37:26.667 INFO:teuthology.run_tasks:Running task pcp... 2024-03-26T18:37:26.679 INFO:teuthology.run_tasks:Running task selinux... 2024-03-26T18:37:26.688 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-26T18:37:26.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-03-26T18:37:26.786 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-03-26T18:37:26.790 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-03-26T18:37:26.840 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-03-26T18:37:26.842 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-03-26T18:37:26.921 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-03-26T18:37:26.923 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-26T18:37:26.923 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-03-26T18:37:26.964 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-03-26T18:37:26.965 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-03-26T18:37:26.990 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-03-26T18:37:26.990 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-03-26T18:37:27.014 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-03-26T18:37:27.015 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive'} 2024-03-26T18:37:27.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T18:37:27.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:37:27.088 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T18:37:27.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:37:27.117 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T18:37:27.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:37:27.145 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-26T18:37:27.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-03-26T18:37:27.222 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-03-26T18:37:27.258 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-03-26T18:37:27.291 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-26T18:37:27.301 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-26T18:37:27.317 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-26T18:37:27.318 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi105.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-26T18:43:42.565 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-03-26T18:43:42.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-26T18:43:42.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T18:43:42.711 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-26T18:43:42.863 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-26T18:43:42.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-26T18:43:42.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T18:43:42.952 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-03-26T18:43:43.021 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-26T18:43:43.022 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-26T18:43:43.022 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T18:43:43.111 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-03-26T18:43:43.182 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-26T18:43:43.183 INFO:teuthology.run_tasks:Running task clock... 2024-03-26T18:43:43.196 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-26T18:43:43.196 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T18:43:43.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T18:43:43.204 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T18:43:43.204 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T18:43:43.206 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T18:43:43.206 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T18:43:43.240 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T18:43:43.257 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T18:43:43.258 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T18:43:43.274 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T18:43:43.288 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-03-26T18:43:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T18:43:43.302 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-03-26T18:43:43.304 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-03-26T18:43:43.319 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-03-26T18:43:43.320 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T18:43:43.335 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T18:43:43.336 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T18:43:43.351 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T18:43:43.354 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T18:43:43.381 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-26T18:43:43.384 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T18:43:43.384 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-26T18:43:43.384 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.384 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.384 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.384 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.395 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-26T18:43:43.398 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T18:43:43.398 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-26T18:43:43.398 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.398 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.398 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.398 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.445 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-03-26T18:43:43.484 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-03-26T18:43:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T18:43:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T18:43:43.716 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-26T18:43:43.724 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T18:43:43.725 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-26T18:43:43.725 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.725 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.725 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.725 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:43:43.726 INFO:teuthology.run_tasks:Running task install... 2024-03-26T18:43:43.738 DEBUG:teuthology.task.install:project ceph 2024-03-26T18:43:43.738 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab'}} 2024-03-26T18:43:43.738 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab'} 2024-03-26T18:43:43.738 INFO:teuthology.task.install:Using flavor: default 2024-03-26T18:43:43.744 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-26T18:43:43.744 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-26T18:43:43.744 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'tag': None, 'wait_for_package': False} 2024-03-26T18:43:43.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:43:43.747 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'tag': None, 'wait_for_package': False} 2024-03-26T18:43:43.747 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:43:43.748 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'tag': None, 'wait_for_package': False} 2024-03-26T18:43:43.748 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:43:43.893 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/ 2024-03-26T18:43:43.893 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2311.g36051606 2024-03-26T18:43:43.901 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/ 2024-03-26T18:43:43.901 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2311.g36051606 2024-03-26T18:43:43.904 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/ 2024-03-26T18:43:43.905 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2311.g36051606 2024-03-26T18:43:44.191 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T18:43:44.192 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:43:44.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T18:43:44.224 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T18:43:44.224 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/360516069d9393362c4cc6eb9371680fe16d66ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T18:43:44.249 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T18:43:44.249 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:43:44.249 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T18:43:44.251 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T18:43:44.251 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:43:44.251 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T18:43:44.280 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T18:43:44.280 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/360516069d9393362c4cc6eb9371680fe16d66ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T18:43:44.296 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T18:43:44.349 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T18:43:44.350 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T18:43:44.351 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/360516069d9393362c4cc6eb9371680fe16d66ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T18:43:44.379 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T18:43:44.431 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T18:43:44.447 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-03-26T18:43:44.448 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-03-26T18:43:44.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T18:43:44.497 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-03-26T18:43:44.499 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-03-26T18:43:44.623 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T18:43:44.685 INFO:teuthology.orchestra.run.smithi105.stdout:74 files removed 2024-03-26T18:43:44.712 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T18:43:44.713 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-03-26T18:43:44.717 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-03-26T18:43:44.733 INFO:teuthology.orchestra.run.smithi144.stdout:74 files removed 2024-03-26T18:43:44.760 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T18:43:45.078 INFO:teuthology.orchestra.run.smithi097.stdout:74 files removed 2024-03-26T18:43:45.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T18:43:45.769 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 108 kB/s | 83 kB 00:00 2024-03-26T18:43:46.020 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 86 kB/s | 83 kB 00:00 2024-03-26T18:43:46.403 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 17 kB/s | 10 kB 00:00 2024-03-26T18:43:46.598 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 84 kB/s | 83 kB 00:00 2024-03-26T18:43:46.754 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-03-26T18:43:47.087 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-03-26T18:43:47.340 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 16 kB/s | 10 kB 00:00 2024-03-26T18:43:47.789 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 9.9 MB/s | 6.7 MB 00:00 2024-03-26T18:43:47.821 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 1.8 kB/s | 1.9 kB 00:01 2024-03-26T18:43:48.091 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-03-26T18:43:48.640 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 8.5 MB/s | 6.7 MB 00:00 2024-03-26T18:43:48.893 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 9.2 MB/s | 6.7 MB 00:00 2024-03-26T18:43:49.448 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-26T18:43:50.123 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-26T18:43:50.586 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2024-03-26T18:43:51.624 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-26T18:43:52.472 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.0 MB 00:02 2024-03-26T18:43:53.181 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.0 MB 00:02 2024-03-26T18:43:54.773 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 34 MB/s | 19 MB 00:00 2024-03-26T18:43:54.940 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-03-26T18:43:58.924 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-26T18:43:59.699 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-03-26T18:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-03-26T18:44:02.233 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-03-26T18:44:03.260 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-26T18:44:08.209 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-26T18:44:10.568 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T18:44:10.569 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T18:44:10.587 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T18:44:10.587 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T18:44:10.587 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T18:44:10.587 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T18:44:10.588 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T18:44:10.591 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T18:44:10.591 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T18:44:10.672 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-26T18:44:10.684 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-26T18:44:10.684 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-26T18:44:10.684 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-26T18:44:10.684 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-26T18:44:10.684 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T18:44:10.684 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.0.0-2311.g36051606.el9 ceph 6.6 k 2024-03-26T18:44:10.684 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.4 M 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 890 k 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2311.g36051606.el9 ceph 145 k 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.0.0-2311.g36051606.el9 ceph 1.5 M 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 140 k 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 1.8 M 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 7.4 M 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 49 k 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 9.8 M 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.0.0-2311.g36051606.el9 ceph 48 M 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 264 k 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 750 k 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 221 k 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell x86_64 2:19.0.0-2311.g36051606.el9 ceph 27 k 2024-03-26T18:44:10.685 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 18 k 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 32 k 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 746 k 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 127 k 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T18:44:10.686 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.0.0-2311.g36051606.el9 ceph 161 k 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.0.0-2311.g36051606.el9 ceph 321 k 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 298 k 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 99 k 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 85 k 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 170 k 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T18:44:10.687 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.4 M 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T18:44:10.688 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T18:44:10.689 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 21 M 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 34 k 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.0.0-2311.g36051606.el9 ceph 2.4 M 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 248 k 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.0.0-2311.g36051606.el9 ceph 4.7 M 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.0.0-2311.g36051606.el9 ceph 17 M 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 16 k 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.0.0-2311.g36051606.el9 ceph 25 k 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T18:44:10.690 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.0.0-2311.g36051606.el9 ceph 163 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 500 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.1 M 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T18:44:10.691 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T18:44:10.692 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:19.0.0-2311.g36051606.el9 ceph 45 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 131 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T18:44:10.693 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T18:44:10.694 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T18:44:10.695 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T18:44:10.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T18:44:10.697 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:44:10.698 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-26T18:44:10.698 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-26T18:44:10.698 INFO:teuthology.orchestra.run.smithi144.stdout:Install 150 Packages 2024-03-26T18:44:10.698 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 24 Packages 2024-03-26T18:44:10.698 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:44:10.698 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 225 M 2024-03-26T18:44:10.698 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-26T18:44:11.610 INFO:teuthology.orchestra.run.smithi144.stdout:(1/174): ceph-19.0.0-2311.g36051606.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-03-26T18:44:11.843 INFO:teuthology.orchestra.run.smithi144.stdout:(2/174): ceph-fuse-19.0.0-2311.g36051606.el9.x8 3.7 MB/s | 890 kB 00:00 2024-03-26T18:44:11.919 INFO:teuthology.orchestra.run.smithi144.stdout:(3/174): ceph-immutable-object-cache-19.0.0-231 1.9 MB/s | 145 kB 00:00 2024-03-26T18:44:12.452 INFO:teuthology.orchestra.run.smithi144.stdout:(4/174): ceph-mds-19.0.0-2311.g36051606.el9.x86 4.4 MB/s | 2.4 MB 00:00 2024-03-26T18:44:12.653 INFO:teuthology.orchestra.run.smithi144.stdout:(5/174): ceph-common-19.0.0-2311.g36051606.el9. 14 MB/s | 21 MB 00:01 2024-03-26T18:44:12.736 INFO:teuthology.orchestra.run.smithi144.stdout:(6/174): ceph-mgr-19.0.0-2311.g36051606.el9.x86 5.3 MB/s | 1.5 MB 00:00 2024-03-26T18:44:13.003 INFO:teuthology.orchestra.run.smithi144.stdout:(7/174): ceph-mon-19.0.0-2311.g36051606.el9.x86 13 MB/s | 4.7 MB 00:00 2024-03-26T18:44:13.253 INFO:teuthology.orchestra.run.smithi144.stdout:(8/174): ceph-base-19.0.0-2311.g36051606.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-03-26T18:44:13.345 INFO:teuthology.orchestra.run.smithi144.stdout:(9/174): ceph-selinux-19.0.0-2311.g36051606.el9 275 kB/s | 25 kB 00:00 2024-03-26T18:44:13.762 INFO:teuthology.orchestra.run.smithi144.stdout:(10/174): ceph-radosgw-19.0.0-2311.g36051606.el 13 MB/s | 9.8 MB 00:00 2024-03-26T18:44:13.829 INFO:teuthology.orchestra.run.smithi144.stdout:(11/174): cephfs-mirror-19.0.0-2311.g36051606.e 3.3 MB/s | 221 kB 00:00 2024-03-26T18:44:13.879 INFO:teuthology.orchestra.run.smithi144.stdout:(12/174): cephfs-shell-19.0.0-2311.g36051606.el 543 kB/s | 27 kB 00:00 2024-03-26T18:44:13.929 INFO:teuthology.orchestra.run.smithi144.stdout:(13/174): libcephfs-devel-19.0.0-2311.g36051606 635 kB/s | 32 kB 00:00 2024-03-26T18:44:14.022 INFO:teuthology.orchestra.run.smithi144.stdout:(14/174): libcephfs2-19.0.0-2311.g36051606.el9. 7.9 MB/s | 746 kB 00:00 2024-03-26T18:44:14.080 INFO:teuthology.orchestra.run.smithi144.stdout:(15/174): libcephsqlite-19.0.0-2311.g36051606.e 2.7 MB/s | 163 kB 00:00 2024-03-26T18:44:14.147 INFO:teuthology.orchestra.run.smithi144.stdout:(16/174): librados-devel-19.0.0-2311.g36051606. 1.9 MB/s | 127 kB 00:00 2024-03-26T18:44:14.222 INFO:teuthology.orchestra.run.smithi144.stdout:(17/174): libradosstriper1-19.0.0-2311.g3605160 6.5 MB/s | 500 kB 00:00 2024-03-26T18:44:14.681 INFO:teuthology.orchestra.run.smithi144.stdout:(18/174): librgw2-19.0.0-2311.g36051606.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-26T18:44:14.765 INFO:teuthology.orchestra.run.smithi144.stdout:(19/174): python3-ceph-argparse-19.0.0-2311.g36 543 kB/s | 45 kB 00:00 2024-03-26T18:44:14.832 INFO:teuthology.orchestra.run.smithi144.stdout:(20/174): python3-ceph-common-19.0.0-2311.g3605 1.9 MB/s | 131 kB 00:00 2024-03-26T18:44:14.906 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 877 kB/s | 19 MB 00:22 2024-03-26T18:44:14.907 INFO:teuthology.orchestra.run.smithi144.stdout:(21/174): python3-cephfs-19.0.0-2311.g36051606. 2.1 MB/s | 161 kB 00:00 2024-03-26T18:44:14.991 INFO:teuthology.orchestra.run.smithi144.stdout:(22/174): python3-rados-19.0.0-2311.g36051606.e 3.8 MB/s | 321 kB 00:00 2024-03-26T18:44:15.158 INFO:teuthology.orchestra.run.smithi144.stdout:(23/174): ceph-osd-19.0.0-2311.g36051606.el9.x8 7.0 MB/s | 17 MB 00:02 2024-03-26T18:44:15.183 INFO:teuthology.orchestra.run.smithi144.stdout:(24/174): python3-rbd-19.0.0-2311.g36051606.el9 1.5 MB/s | 298 kB 00:00 2024-03-26T18:44:15.200 INFO:teuthology.orchestra.run.smithi144.stdout:(25/174): python3-rgw-19.0.0-2311.g36051606.el9 2.3 MB/s | 99 kB 00:00 2024-03-26T18:44:15.233 INFO:teuthology.orchestra.run.smithi144.stdout:(26/174): rbd-fuse-19.0.0-2311.g36051606.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-26T18:44:15.300 INFO:teuthology.orchestra.run.smithi144.stdout:(27/174): rbd-nbd-19.0.0-2311.g36051606.el9.x86 2.5 MB/s | 170 kB 00:00 2024-03-26T18:44:15.367 INFO:teuthology.orchestra.run.smithi144.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2311.g 509 kB/s | 34 kB 00:00 2024-03-26T18:44:15.443 INFO:teuthology.orchestra.run.smithi144.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2311.g3605160 1.8 MB/s | 140 kB 00:00 2024-03-26T18:44:15.643 INFO:teuthology.orchestra.run.smithi144.stdout:(30/174): rbd-mirror-19.0.0-2311.g36051606.el9. 7.0 MB/s | 3.1 MB 00:00 2024-03-26T18:44:15.685 INFO:teuthology.orchestra.run.smithi144.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2311.g36051 7.4 MB/s | 1.8 MB 00:00 2024-03-26T18:44:15.769 INFO:teuthology.orchestra.run.smithi144.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2311.g36 2.9 MB/s | 248 kB 00:00 2024-03-26T18:44:15.835 INFO:teuthology.orchestra.run.smithi144.stdout:(33/174): ceph-mgr-rook-19.0.0-2311.g36051606.e 743 kB/s | 49 kB 00:00 2024-03-26T18:44:15.894 INFO:teuthology.orchestra.run.smithi144.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2311.g3 282 kB/s | 16 kB 00:00 2024-03-26T18:44:15.952 INFO:teuthology.orchestra.run.smithi144.stdout:(35/174): ceph-volume-19.0.0-2311.g36051606.el9 4.4 MB/s | 264 kB 00:00 2024-03-26T18:44:16.086 INFO:teuthology.orchestra.run.smithi144.stdout:(36/174): cephadm-19.0.0-2311.g36051606.el9.noa 5.5 MB/s | 750 kB 00:00 2024-03-26T18:44:16.120 INFO:teuthology.orchestra.run.smithi144.stdout:(37/174): cephfs-top-19.0.0-2311.g36051606.el9. 545 kB/s | 18 kB 00:00 2024-03-26T18:44:16.378 INFO:teuthology.orchestra.run.smithi144.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-03-26T18:44:16.579 INFO:teuthology.orchestra.run.smithi144.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.4 MB/s | 662 kB 00:00 2024-03-26T18:44:16.662 INFO:teuthology.orchestra.run.smithi144.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 920 kB 00:00 2024-03-26T18:44:16.687 INFO:teuthology.orchestra.run.smithi144.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 983 kB/s | 106 kB 00:00 2024-03-26T18:44:16.742 INFO:teuthology.orchestra.run.smithi144.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.8 MB/s | 307 kB 00:00 2024-03-26T18:44:16.768 INFO:teuthology.orchestra.run.smithi144.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 394 kB/s | 32 kB 00:00 2024-03-26T18:44:16.877 INFO:teuthology.orchestra.run.smithi144.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 158 kB/s | 17 kB 00:00 2024-03-26T18:44:16.935 INFO:teuthology.orchestra.run.smithi144.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-03-26T18:44:16.961 INFO:teuthology.orchestra.run.smithi144.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 122 kB/s | 10 kB 00:00 2024-03-26T18:44:16.994 INFO:teuthology.orchestra.run.smithi144.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.6 MB/s | 159 kB 00:00 2024-03-26T18:44:17.011 INFO:teuthology.orchestra.run.smithi144.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2024-03-26T18:44:17.045 INFO:teuthology.orchestra.run.smithi144.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-26T18:44:17.070 INFO:teuthology.orchestra.run.smithi144.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-03-26T18:44:17.087 INFO:teuthology.orchestra.run.smithi144.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-26T18:44:17.122 INFO:teuthology.orchestra.run.smithi144.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-26T18:44:17.137 INFO:teuthology.orchestra.run.smithi144.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T18:44:17.555 INFO:teuthology.orchestra.run.smithi144.stdout:(54/174): ceph-test-19.0.0-2311.g36051606.el9.x 11 MB/s | 48 MB 00:04 2024-03-26T18:44:17.580 INFO:teuthology.orchestra.run.smithi144.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 473 kB/s | 217 kB 00:00 2024-03-26T18:44:17.605 INFO:teuthology.orchestra.run.smithi144.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 73 kB/s | 34 kB 00:00 2024-03-26T18:44:17.714 INFO:teuthology.orchestra.run.smithi144.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-03-26T18:44:17.739 INFO:teuthology.orchestra.run.smithi144.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 597 kB/s | 78 kB 00:00 2024-03-26T18:44:17.773 INFO:teuthology.orchestra.run.smithi144.stdout:(59/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.2 MB/s | 1.0 MB 00:00 2024-03-26T18:44:17.890 INFO:teuthology.orchestra.run.smithi144.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-26T18:44:17.915 INFO:teuthology.orchestra.run.smithi144.stdout:(61/174): python3-jsonpatch-1.21-16.el9.noarch. 141 kB/s | 24 kB 00:00 2024-03-26T18:44:17.942 INFO:teuthology.orchestra.run.smithi144.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 110 kB/s | 18 kB 00:00 2024-03-26T18:44:17.965 INFO:teuthology.orchestra.run.smithi144.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-26T18:44:17.991 INFO:teuthology.orchestra.run.smithi144.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-26T18:44:18.082 INFO:teuthology.orchestra.run.smithi144.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-26T18:44:18.108 INFO:teuthology.orchestra.run.smithi144.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-26T18:44:18.133 INFO:teuthology.orchestra.run.smithi144.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-26T18:44:18.150 INFO:teuthology.orchestra.run.smithi144.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-26T18:44:18.183 INFO:teuthology.orchestra.run.smithi144.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-03-26T18:44:18.209 INFO:teuthology.orchestra.run.smithi144.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-26T18:44:18.234 INFO:teuthology.orchestra.run.smithi144.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-26T18:44:18.259 INFO:teuthology.orchestra.run.smithi144.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-26T18:44:18.284 INFO:teuthology.orchestra.run.smithi144.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 631 kB/s | 47 kB 00:00 2024-03-26T18:44:18.309 INFO:teuthology.orchestra.run.smithi144.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 572 kB/s | 43 kB 00:00 2024-03-26T18:44:18.368 INFO:teuthology.orchestra.run.smithi144.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 465 kB/s | 39 kB 00:00 2024-03-26T18:44:18.518 INFO:teuthology.orchestra.run.smithi144.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 198 kB/s | 41 kB 00:00 2024-03-26T18:44:18.694 INFO:teuthology.orchestra.run.smithi144.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-26T18:44:18.719 INFO:teuthology.orchestra.run.smithi144.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 861 kB/s | 300 kB 00:00 2024-03-26T18:44:18.744 INFO:teuthology.orchestra.run.smithi144.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 252 kB/s | 56 kB 00:00 2024-03-26T18:44:18.769 INFO:teuthology.orchestra.run.smithi144.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 839 kB/s | 63 kB 00:00 2024-03-26T18:44:18.794 INFO:teuthology.orchestra.run.smithi144.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-03-26T18:44:18.820 INFO:teuthology.orchestra.run.smithi144.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-26T18:44:18.853 INFO:teuthology.orchestra.run.smithi144.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2024-03-26T18:44:18.887 INFO:teuthology.orchestra.run.smithi144.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-26T18:44:18.912 INFO:teuthology.orchestra.run.smithi144.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-26T18:44:18.937 INFO:teuthology.orchestra.run.smithi144.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-26T18:44:18.963 INFO:teuthology.orchestra.run.smithi144.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-26T18:44:19.274 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-26T18:44:19.463 INFO:teuthology.orchestra.run.smithi144.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 340 kB/s | 187 kB 00:00 2024-03-26T18:44:19.488 INFO:teuthology.orchestra.run.smithi144.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 372 kB/s | 195 kB 00:00 2024-03-26T18:44:19.572 INFO:teuthology.orchestra.run.smithi144.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-26T18:44:19.597 INFO:teuthology.orchestra.run.smithi144.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-03-26T18:44:19.631 INFO:teuthology.orchestra.run.smithi144.stdout:(92/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-03-26T18:44:19.864 INFO:teuthology.orchestra.run.smithi144.stdout:(93/174): postgresql-13.14-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-03-26T18:44:20.031 INFO:teuthology.orchestra.run.smithi144.stdout:(94/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 9.5 MB/s | 4.1 MB 00:00 2024-03-26T18:44:20.081 INFO:teuthology.orchestra.run.smithi144.stdout:(95/174): postgresql-contrib-13.14-1.el9.x86_64 4.1 MB/s | 911 kB 00:00 2024-03-26T18:44:20.148 INFO:teuthology.orchestra.run.smithi144.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-03-26T18:44:20.198 INFO:teuthology.orchestra.run.smithi144.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-03-26T18:44:20.341 INFO:teuthology.orchestra.run.smithi144.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-03-26T18:44:20.425 INFO:teuthology.orchestra.run.smithi144.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-03-26T18:44:20.476 INFO:teuthology.orchestra.run.smithi144.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 717 kB/s | 36 kB 00:00 2024-03-26T18:44:20.528 INFO:teuthology.orchestra.run.smithi144.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 908 kB/s | 47 kB 00:00 2024-03-26T18:44:20.578 INFO:teuthology.orchestra.run.smithi144.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-26T18:44:20.720 INFO:teuthology.orchestra.run.smithi144.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-03-26T18:44:20.987 INFO:teuthology.orchestra.run.smithi144.stdout:(104/174): postgresql-server-13.14-1.el9.x86_64 6.4 MB/s | 5.8 MB 00:00 2024-03-26T18:44:21.192 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-03-26T18:44:21.221 INFO:teuthology.orchestra.run.smithi144.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-03-26T18:44:21.288 INFO:teuthology.orchestra.run.smithi144.stdout:(106/174): libuuid-devel-2.37.4-18.el9.x86_64.r 15 kB/s | 25 kB 00:01 2024-03-26T18:44:21.338 INFO:teuthology.orchestra.run.smithi144.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-26T18:44:21.363 INFO:teuthology.orchestra.run.smithi144.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-03-26T18:44:21.389 INFO:teuthology.orchestra.run.smithi144.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-03-26T18:44:21.489 INFO:teuthology.orchestra.run.smithi144.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-03-26T18:44:21.514 INFO:teuthology.orchestra.run.smithi144.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 389 kB/s | 49 kB 00:00 2024-03-26T18:44:21.572 INFO:teuthology.orchestra.run.smithi144.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2024-03-26T18:44:21.656 INFO:teuthology.orchestra.run.smithi144.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-26T18:44:21.731 INFO:teuthology.orchestra.run.smithi144.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-03-26T18:44:21.773 INFO:teuthology.orchestra.run.smithi144.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-03-26T18:44:21.807 INFO:teuthology.orchestra.run.smithi144.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-03-26T18:44:21.832 INFO:teuthology.orchestra.run.smithi144.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 444 kB/s | 43 kB 00:00 2024-03-26T18:44:21.857 INFO:teuthology.orchestra.run.smithi144.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 647 kB/s | 32 kB 00:00 2024-03-26T18:44:21.874 INFO:teuthology.orchestra.run.smithi144.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-03-26T18:44:21.899 INFO:teuthology.orchestra.run.smithi144.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-26T18:44:21.925 INFO:teuthology.orchestra.run.smithi144.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-03-26T18:44:21.950 INFO:teuthology.orchestra.run.smithi144.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-03-26T18:44:22.009 INFO:teuthology.orchestra.run.smithi144.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 327 kB/s | 35 kB 00:00 2024-03-26T18:44:22.034 INFO:teuthology.orchestra.run.smithi144.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-03-26T18:44:22.301 INFO:teuthology.orchestra.run.smithi144.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-03-26T18:44:22.326 INFO:teuthology.orchestra.run.smithi144.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 456 kB/s | 144 kB 00:00 2024-03-26T18:44:22.351 INFO:teuthology.orchestra.run.smithi144.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 34 kB/s | 11 kB 00:00 2024-03-26T18:44:22.376 INFO:teuthology.orchestra.run.smithi144.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-03-26T18:44:22.402 INFO:teuthology.orchestra.run.smithi144.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-03-26T18:44:22.427 INFO:teuthology.orchestra.run.smithi144.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-03-26T18:44:22.452 INFO:teuthology.orchestra.run.smithi144.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-03-26T18:44:22.477 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 975 kB/s | 21 MB 00:21 2024-03-26T18:44:22.669 INFO:teuthology.orchestra.run.smithi144.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-03-26T18:44:22.694 INFO:teuthology.orchestra.run.smithi144.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 173 kB/s | 46 kB 00:00 2024-03-26T18:44:22.720 INFO:teuthology.orchestra.run.smithi144.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 295 kB/s | 79 kB 00:00 2024-03-26T18:44:22.745 INFO:teuthology.orchestra.run.smithi144.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-03-26T18:44:22.770 INFO:teuthology.orchestra.run.smithi144.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-26T18:44:22.796 INFO:teuthology.orchestra.run.smithi144.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-03-26T18:44:22.821 INFO:teuthology.orchestra.run.smithi144.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-26T18:44:22.846 INFO:teuthology.orchestra.run.smithi144.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-03-26T18:44:22.871 INFO:teuthology.orchestra.run.smithi144.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-03-26T18:44:22.896 INFO:teuthology.orchestra.run.smithi144.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-26T18:44:22.921 INFO:teuthology.orchestra.run.smithi144.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-03-26T18:44:22.947 INFO:teuthology.orchestra.run.smithi144.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 479 kB/s | 36 kB 00:00 2024-03-26T18:44:22.981 INFO:teuthology.orchestra.run.smithi144.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 873 kB/s | 51 kB 00:00 2024-03-26T18:44:23.006 INFO:teuthology.orchestra.run.smithi144.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-26T18:44:23.031 INFO:teuthology.orchestra.run.smithi144.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-26T18:44:23.181 INFO:teuthology.orchestra.run.smithi144.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-03-26T18:44:23.206 INFO:teuthology.orchestra.run.smithi144.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 114 kB/s | 20 kB 00:00 2024-03-26T18:44:23.248 INFO:teuthology.orchestra.run.smithi144.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 883 kB/s | 192 kB 00:00 2024-03-26T18:44:23.290 INFO:teuthology.orchestra.run.smithi144.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-03-26T18:44:23.366 INFO:teuthology.orchestra.run.smithi144.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-03-26T18:44:23.583 INFO:teuthology.orchestra.run.smithi144.stdout:(152/174): librados2-19.0.0-2311.g36051606.el9. 9.0 MB/s | 3.4 MB 00:00 2024-03-26T18:44:23.816 INFO:teuthology.orchestra.run.smithi144.stdout:(153/174): librbd1-19.0.0-2311.g36051606.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-03-26T18:44:24.133 INFO:teuthology.orchestra.run.smithi144.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 133 kB/s | 42 kB 00:00 2024-03-26T18:44:24.317 INFO:teuthology.orchestra.run.smithi144.stdout:(155/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.4 MB/s | 2.3 MB 00:00 2024-03-26T18:44:24.400 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 6.8 MB/s | 21 MB 00:03 2024-03-26T18:44:24.442 INFO:teuthology.orchestra.run.smithi144.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 572 kB/s | 71 kB 00:00 2024-03-26T18:44:24.467 INFO:teuthology.orchestra.run.smithi144.stdout:(157/174): NetworkManager-tui-1.47.2-1.el9.x86_ 744 kB/s | 248 kB 00:00 2024-03-26T18:44:24.517 INFO:teuthology.orchestra.run.smithi144.stdout:(158/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.0 MB/s | 1.8 MB 00:00 2024-03-26T18:44:24.567 INFO:teuthology.orchestra.run.smithi144.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 290 kB/s | 36 kB 00:00 2024-03-26T18:44:24.593 INFO:teuthology.orchestra.run.smithi144.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.5 MB/s | 192 kB 00:00 2024-03-26T18:44:24.643 INFO:teuthology.orchestra.run.smithi144.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-03-26T18:44:24.685 INFO:teuthology.orchestra.run.smithi144.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 196 kB/s | 23 kB 00:00 2024-03-26T18:44:24.710 INFO:teuthology.orchestra.run.smithi144.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 924 kB/s | 108 kB 00:00 2024-03-26T18:44:24.769 INFO:teuthology.orchestra.run.smithi144.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 155 kB 00:00 2024-03-26T18:44:24.811 INFO:teuthology.orchestra.run.smithi144.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-26T18:44:24.828 INFO:teuthology.orchestra.run.smithi144.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 544 kB/s | 64 kB 00:00 2024-03-26T18:44:24.886 INFO:teuthology.orchestra.run.smithi144.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 244 kB/s | 29 kB 00:00 2024-03-26T18:44:24.962 INFO:teuthology.orchestra.run.smithi144.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 8.0 MB/s | 1.2 MB 00:00 2024-03-26T18:44:25.104 INFO:teuthology.orchestra.run.smithi144.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.2 MB/s | 465 kB 00:00 2024-03-26T18:44:25.171 INFO:teuthology.orchestra.run.smithi144.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.0 MB/s | 2.3 MB 00:00 2024-03-26T18:44:25.238 INFO:teuthology.orchestra.run.smithi144.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 233 kB/s | 31 kB 00:00 2024-03-26T18:44:25.321 INFO:teuthology.orchestra.run.smithi144.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.2 MB/s | 1.1 MB 00:00 2024-03-26T18:44:25.347 INFO:teuthology.orchestra.run.smithi144.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 684 kB/s | 74 kB 00:00 2024-03-26T18:44:25.464 INFO:teuthology.orchestra.run.smithi144.stdout:(174/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.4 MB/s | 2.2 MB 00:00 2024-03-26T18:44:25.469 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-26T18:44:25.469 INFO:teuthology.orchestra.run.smithi144.stdout:Total 15 MB/s | 225 MB 00:14 2024-03-26T18:44:26.046 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-26T18:44:26.318 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-26T18:44:26.318 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-26T18:44:27.457 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-26T18:44:28.047 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-26T18:44:28.047 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-26T18:44:29.820 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T18:44:29.821 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T18:44:29.833 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T18:44:29.834 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T18:44:29.834 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T18:44:29.835 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T18:44:29.837 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T18:44:29.839 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T18:44:29.840 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T18:44:29.920 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-03-26T18:44:29.939 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.0.0-2311.g36051606.el9 ceph 6.6 k 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.4 M 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 890 k 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2311.g36051606.el9 ceph 145 k 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.0.0-2311.g36051606.el9 ceph 1.5 M 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 140 k 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 1.8 M 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 7.4 M 2024-03-26T18:44:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 49 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 9.8 M 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.0.0-2311.g36051606.el9 ceph 48 M 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 264 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 750 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 221 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.0.0-2311.g36051606.el9 ceph 27 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 18 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 32 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 746 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 127 k 2024-03-26T18:44:29.941 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.0.0-2311.g36051606.el9 ceph 161 k 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.0.0-2311.g36051606.el9 ceph 321 k 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 298 k 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 99 k 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 85 k 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 170 k 2024-03-26T18:44:29.942 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T18:44:29.943 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-03-26T18:44:29.943 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T18:44:29.943 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T18:44:29.943 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T18:44:29.943 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T18:44:29.943 INFO:teuthology.orchestra.run.smithi105.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T18:44:29.943 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T18:44:29.943 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T18:44:29.943 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.4 M 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T18:44:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 21 M 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 34 k 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.0.0-2311.g36051606.el9 ceph 2.4 M 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 248 k 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.0.0-2311.g36051606.el9 ceph 4.7 M 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.0.0-2311.g36051606.el9 ceph 17 M 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 16 k 2024-03-26T18:44:29.945 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.0.0-2311.g36051606.el9 ceph 25 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.0.0-2311.g36051606.el9 ceph 163 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T18:44:29.946 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 500 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.1 M 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T18:44:29.947 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.0.0-2311.g36051606.el9 ceph 45 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 131 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T18:44:29.948 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T18:44:29.949 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T18:44:29.950 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T18:44:29.951 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T18:44:29.952 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T18:44:29.953 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T18:44:29.953 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-03-26T18:44:29.953 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T18:44:29.953 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:44:29.953 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-03-26T18:44:29.953 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-26T18:44:29.953 INFO:teuthology.orchestra.run.smithi105.stdout:Install 150 Packages 2024-03-26T18:44:29.953 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 24 Packages 2024-03-26T18:44:29.953 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:44:29.954 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 225 M 2024-03-26T18:44:29.954 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-03-26T18:44:30.151 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-26T18:44:30.759 INFO:teuthology.orchestra.run.smithi105.stdout:(1/174): ceph-19.0.0-2311.g36051606.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-03-26T18:44:31.051 INFO:teuthology.orchestra.run.smithi105.stdout:(2/174): ceph-base-19.0.0-2311.g36051606.el9.x8 9.2 MB/s | 5.4 MB 00:00 2024-03-26T18:44:31.101 INFO:teuthology.orchestra.run.smithi105.stdout:(3/174): ceph-immutable-object-cache-19.0.0-231 2.9 MB/s | 145 kB 00:00 2024-03-26T18:44:31.226 INFO:teuthology.orchestra.run.smithi105.stdout:(4/174): ceph-fuse-19.0.0-2311.g36051606.el9.x8 1.9 MB/s | 890 kB 00:00 2024-03-26T18:44:31.268 INFO:teuthology.orchestra.run.smithi105.stdout:(5/174): ceph-mds-19.0.0-2311.g36051606.el9.x86 14 MB/s | 2.4 MB 00:00 2024-03-26T18:44:31.569 INFO:teuthology.orchestra.run.smithi105.stdout:(6/174): ceph-mon-19.0.0-2311.g36051606.el9.x86 16 MB/s | 4.7 MB 00:00 2024-03-26T18:44:31.710 INFO:teuthology.orchestra.run.smithi105.stdout:(7/174): ceph-mgr-19.0.0-2311.g36051606.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-03-26T18:44:32.569 INFO:teuthology.orchestra.run.smithi105.stdout:(8/174): ceph-osd-19.0.0-2311.g36051606.el9.x86 17 MB/s | 17 MB 00:01 2024-03-26T18:44:32.628 INFO:teuthology.orchestra.run.smithi105.stdout:(9/174): ceph-selinux-19.0.0-2311.g36051606.el9 434 kB/s | 25 kB 00:00 2024-03-26T18:44:32.722 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T18:44:32.723 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T18:44:32.742 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T18:44:32.743 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T18:44:32.745 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T18:44:32.746 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T18:44:32.748 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T18:44:32.752 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T18:44:32.753 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T18:44:32.839 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-2311.g36051606.el9 ceph 6.6 k 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.4 M 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 890 k 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2311.g36051606.el9 ceph 145 k 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-2311.g36051606.el9 ceph 1.5 M 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 140 k 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 1.8 M 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 7.4 M 2024-03-26T18:44:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 49 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 9.8 M 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-2311.g36051606.el9 ceph 48 M 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 264 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 750 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 221 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-2311.g36051606.el9 ceph 27 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 18 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 32 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 746 k 2024-03-26T18:44:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 127 k 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-2311.g36051606.el9 ceph 161 k 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-2311.g36051606.el9 ceph 321 k 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 298 k 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 99 k 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 85 k 2024-03-26T18:44:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 170 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T18:44:32.870 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.4 M 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T18:44:32.871 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 21 M 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 34 k 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-2311.g36051606.el9 ceph 2.4 M 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 248 k 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-2311.g36051606.el9 ceph 4.7 M 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-2311.g36051606.el9 ceph 17 M 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 16 k 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-2311.g36051606.el9 ceph 25 k 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T18:44:32.872 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-2311.g36051606.el9 ceph 163 k 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 500 k 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T18:44:32.873 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.1 M 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T18:44:32.874 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-2311.g36051606.el9 ceph 45 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 131 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T18:44:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T18:44:32.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T18:44:32.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T18:44:32.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-26T18:44:32.879 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-26T18:44:32.880 INFO:teuthology.orchestra.run.smithi097.stdout:Install 150 Packages 2024-03-26T18:44:32.880 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 24 Packages 2024-03-26T18:44:32.880 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:44:32.880 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 225 M 2024-03-26T18:44:32.880 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-26T18:44:33.678 INFO:teuthology.orchestra.run.smithi105.stdout:(10/174): ceph-radosgw-19.0.0-2311.g36051606.el 5.0 MB/s | 9.8 MB 00:01 2024-03-26T18:44:33.770 INFO:teuthology.orchestra.run.smithi105.stdout:(11/174): cephfs-mirror-19.0.0-2311.g36051606.e 2.4 MB/s | 221 kB 00:00 2024-03-26T18:44:33.870 INFO:teuthology.orchestra.run.smithi105.stdout:(12/174): cephfs-shell-19.0.0-2311.g36051606.el 271 kB/s | 27 kB 00:00 2024-03-26T18:44:33.962 INFO:teuthology.orchestra.run.smithi105.stdout:(13/174): libcephfs-devel-19.0.0-2311.g36051606 346 kB/s | 32 kB 00:00 2024-03-26T18:44:34.236 INFO:teuthology.orchestra.run.smithi097.stdout:(1/174): ceph-19.0.0-2311.g36051606.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-03-26T18:44:34.262 INFO:teuthology.orchestra.run.smithi105.stdout:(14/174): libcephfs2-19.0.0-2311.g36051606.el9. 2.4 MB/s | 746 kB 00:00 2024-03-26T18:44:34.354 INFO:teuthology.orchestra.run.smithi105.stdout:(15/174): libcephsqlite-19.0.0-2311.g36051606.e 1.7 MB/s | 163 kB 00:00 2024-03-26T18:44:34.454 INFO:teuthology.orchestra.run.smithi105.stdout:(16/174): librados-devel-19.0.0-2311.g36051606. 1.2 MB/s | 127 kB 00:00 2024-03-26T18:44:34.580 INFO:teuthology.orchestra.run.smithi105.stdout:(17/174): libradosstriper1-19.0.0-2311.g3605160 3.9 MB/s | 500 kB 00:00 2024-03-26T18:44:35.153 INFO:teuthology.orchestra.run.smithi097.stdout:(2/174): ceph-fuse-19.0.0-2311.g36051606.el9.x8 972 kB/s | 890 kB 00:00 2024-03-26T18:44:35.162 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-26T18:44:35.267 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T18:44:35.304 INFO:teuthology.orchestra.run.smithi097.stdout:(3/174): ceph-immutable-object-cache-19.0.0-231 974 kB/s | 145 kB 00:00 2024-03-26T18:44:35.310 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T18:44:35.349 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T18:44:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T18:44:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T18:44:35.452 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T18:44:35.485 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T18:44:35.511 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T18:44:35.522 INFO:teuthology.orchestra.run.smithi105.stdout:(18/174): ceph-common-19.0.0-2311.g36051606.el9 4.1 MB/s | 21 MB 00:05 2024-03-26T18:44:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T18:44:35.605 INFO:teuthology.orchestra.run.smithi105.stdout:(19/174): librgw2-19.0.0-2311.g36051606.el9.x86 5.0 MB/s | 5.1 MB 00:01 2024-03-26T18:44:35.622 INFO:teuthology.orchestra.run.smithi105.stdout:(20/174): python3-ceph-argparse-19.0.0-2311.g36 452 kB/s | 45 kB 00:00 2024-03-26T18:44:35.778 INFO:teuthology.orchestra.run.smithi097.stdout:(4/174): ceph-mds-19.0.0-2311.g36051606.el9.x86 5.0 MB/s | 2.4 MB 00:00 2024-03-26T18:44:35.879 INFO:teuthology.orchestra.run.smithi097.stdout:(5/174): ceph-base-19.0.0-2311.g36051606.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-03-26T18:44:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T18:44:35.937 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T18:44:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:(6/174): ceph-mgr-19.0.0-2311.g36051606.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-03-26T18:44:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T18:44:36.056 INFO:teuthology.orchestra.run.smithi105.stdout:(21/174): ceph-test-19.0.0-2311.g36051606.el9.x 14 MB/s | 48 MB 00:03 2024-03-26T18:44:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T18:44:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:(22/174): python3-ceph-common-19.0.0-2311.g3605 277 kB/s | 131 kB 00:00 2024-03-26T18:44:36.106 INFO:teuthology.orchestra.run.smithi105.stdout:(23/174): python3-cephfs-19.0.0-2311.g36051606. 334 kB/s | 161 kB 00:00 2024-03-26T18:44:36.154 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T18:44:36.156 INFO:teuthology.orchestra.run.smithi105.stdout:(24/174): python3-rados-19.0.0-2311.g36051606.e 3.1 MB/s | 321 kB 00:00 2024-03-26T18:44:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T18:44:36.224 INFO:teuthology.orchestra.run.smithi105.stdout:(25/174): rbd-fuse-19.0.0-2311.g36051606.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-26T18:44:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T18:44:36.249 INFO:teuthology.orchestra.run.smithi105.stdout:(26/174): python3-rgw-19.0.0-2311.g36051606.el9 698 kB/s | 99 kB 00:00 2024-03-26T18:44:36.274 INFO:teuthology.orchestra.run.smithi105.stdout:(27/174): python3-rbd-19.0.0-2311.g36051606.el9 1.5 MB/s | 298 kB 00:00 2024-03-26T18:44:36.366 INFO:teuthology.orchestra.run.smithi105.stdout:(28/174): rbd-nbd-19.0.0-2311.g36051606.el9.x86 1.4 MB/s | 170 kB 00:00 2024-03-26T18:44:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T18:44:36.413 INFO:teuthology.orchestra.run.smithi105.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2311.g 246 kB/s | 34 kB 00:00 2024-03-26T18:44:36.440 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T18:44:36.469 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T18:44:36.505 INFO:teuthology.orchestra.run.smithi105.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2311.g3605160 1.0 MB/s | 140 kB 00:00 2024-03-26T18:44:36.555 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T18:44:36.572 INFO:teuthology.orchestra.run.smithi105.stdout:(31/174): rbd-mirror-19.0.0-2311.g36051606.el9. 8.9 MB/s | 3.1 MB 00:00 2024-03-26T18:44:36.578 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T18:44:36.625 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T18:44:36.647 INFO:teuthology.orchestra.run.smithi105.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2311.g36 3.2 MB/s | 248 kB 00:00 2024-03-26T18:44:36.665 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T18:44:36.688 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T18:44:36.704 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T18:44:36.722 INFO:teuthology.orchestra.run.smithi105.stdout:(33/174): ceph-mgr-rook-19.0.0-2311.g36051606.e 657 kB/s | 49 kB 00:00 2024-03-26T18:44:36.790 INFO:teuthology.orchestra.run.smithi105.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2311.g3 246 kB/s | 16 kB 00:00 2024-03-26T18:44:36.823 INFO:teuthology.orchestra.run.smithi105.stdout:(35/174): ceph-mgr-dashboard-19.0.0-2311.g36051 4.4 MB/s | 1.8 MB 00:00 2024-03-26T18:44:36.873 INFO:teuthology.orchestra.run.smithi105.stdout:(36/174): ceph-volume-19.0.0-2311.g36051606.el9 3.1 MB/s | 264 kB 00:00 2024-03-26T18:44:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T18:44:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T18:44:36.965 INFO:teuthology.orchestra.run.smithi105.stdout:(37/174): cephfs-top-19.0.0-2311.g36051606.el9. 199 kB/s | 18 kB 00:00 2024-03-26T18:44:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T18:44:36.999 INFO:teuthology.orchestra.run.smithi105.stdout:(38/174): cephadm-19.0.0-2311.g36051606.el9.noa 4.2 MB/s | 750 kB 00:00 2024-03-26T18:44:37.062 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T18:44:37.157 INFO:teuthology.orchestra.run.smithi105.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-03-26T18:44:37.167 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T18:44:37.186 INFO:teuthology.orchestra.run.smithi097.stdout:(7/174): ceph-mon-19.0.0-2311.g36051606.el9.x86 3.6 MB/s | 4.7 MB 00:01 2024-03-26T18:44:37.191 INFO:teuthology.orchestra.run.smithi105.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 5.4 MB/s | 920 kB 00:00 2024-03-26T18:44:37.216 INFO:teuthology.orchestra.run.smithi105.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-26T18:44:37.230 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T18:44:37.250 INFO:teuthology.orchestra.run.smithi105.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-03-26T18:44:37.275 INFO:teuthology.orchestra.run.smithi105.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 546 kB/s | 32 kB 00:00 2024-03-26T18:44:37.318 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T18:44:37.359 INFO:teuthology.orchestra.run.smithi105.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-26T18:44:37.364 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T18:44:37.384 INFO:teuthology.orchestra.run.smithi105.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 160 kB/s | 17 kB 00:00 2024-03-26T18:44:37.418 INFO:teuthology.orchestra.run.smithi105.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 303 kB/s | 10 kB 00:00 2024-03-26T18:44:37.443 INFO:teuthology.orchestra.run.smithi105.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-26T18:44:37.468 INFO:teuthology.orchestra.run.smithi105.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-26T18:44:37.493 INFO:teuthology.orchestra.run.smithi105.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-26T18:44:37.527 INFO:teuthology.orchestra.run.smithi105.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-03-26T18:44:37.552 INFO:teuthology.orchestra.run.smithi105.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 804 kB/s | 46 kB 00:00 2024-03-26T18:44:37.578 INFO:teuthology.orchestra.run.smithi105.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-26T18:44:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T18:44:37.603 INFO:teuthology.orchestra.run.smithi105.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T18:44:37.628 INFO:teuthology.orchestra.run.smithi105.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-26T18:44:37.645 INFO:teuthology.orchestra.run.smithi105.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 826 kB/s | 34 kB 00:00 2024-03-26T18:44:37.654 INFO:teuthology.orchestra.run.smithi097.stdout:(8/174): ceph-common-19.0.0-2311.g36051606.el9. 5.4 MB/s | 21 MB 00:03 2024-03-26T18:44:37.707 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T18:44:37.762 INFO:teuthology.orchestra.run.smithi105.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-03-26T18:44:37.796 INFO:teuthology.orchestra.run.smithi105.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-03-26T18:44:37.807 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T18:44:37.821 INFO:teuthology.orchestra.run.smithi105.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-26T18:44:37.836 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T18:44:37.863 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T18:44:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:(9/174): ceph-osd-19.0.0-2311.g36051606.el9.x86 9.1 MB/s | 17 MB 00:01 2024-03-26T18:44:37.898 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T18:44:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:(10/174): ceph-selinux-19.0.0-2311.g36051606.el 95 kB/s | 25 kB 00:00 2024-03-26T18:44:37.955 INFO:teuthology.orchestra.run.smithi105.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 37 MB/s | 5.8 MB 00:00 2024-03-26T18:44:37.971 INFO:teuthology.orchestra.run.smithi105.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 166 kB/s | 24 kB 00:00 2024-03-26T18:44:37.972 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T18:44:37.988 INFO:teuthology.orchestra.run.smithi105.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 544 kB/s | 18 kB 00:00 2024-03-26T18:44:38.014 INFO:teuthology.orchestra.run.smithi105.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-26T18:44:38.027 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T18:44:38.030 INFO:teuthology.orchestra.run.smithi105.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 752 kB/s | 32 kB 00:00 2024-03-26T18:44:38.038 INFO:teuthology.orchestra.run.smithi097.stdout:(11/174): cephfs-mirror-19.0.0-2311.g36051606.e 1.9 MB/s | 221 kB 00:00 2024-03-26T18:44:38.065 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T18:44:38.105 INFO:teuthology.orchestra.run.smithi097.stdout:(12/174): cephfs-shell-19.0.0-2311.g36051606.el 412 kB/s | 27 kB 00:00 2024-03-26T18:44:38.130 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T18:44:38.164 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T18:44:38.165 INFO:teuthology.orchestra.run.smithi105.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-26T18:44:38.181 INFO:teuthology.orchestra.run.smithi097.stdout:(13/174): libcephfs-devel-19.0.0-2311.g36051606 428 kB/s | 32 kB 00:00 2024-03-26T18:44:38.189 INFO:teuthology.orchestra.run.smithi105.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-26T18:44:38.208 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T18:44:38.215 INFO:teuthology.orchestra.run.smithi105.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-26T18:44:38.240 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T18:44:38.240 INFO:teuthology.orchestra.run.smithi105.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-26T18:44:38.257 INFO:teuthology.orchestra.run.smithi105.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 986 kB/s | 41 kB 00:00 2024-03-26T18:44:38.268 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T18:44:38.282 INFO:teuthology.orchestra.run.smithi105.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 5.5 MB/s | 238 kB 00:00 2024-03-26T18:44:38.308 INFO:teuthology.orchestra.run.smithi105.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-26T18:44:38.333 INFO:teuthology.orchestra.run.smithi105.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-26T18:44:38.337 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T18:44:38.359 INFO:teuthology.orchestra.run.smithi105.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-03-26T18:44:38.381 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 39/198 2024-03-26T18:44:38.383 INFO:teuthology.orchestra.run.smithi105.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 852 kB/s | 43 kB 00:00 2024-03-26T18:44:38.397 INFO:teuthology.orchestra.run.smithi097.stdout:(14/174): libcephfs2-19.0.0-2311.g36051606.el9. 3.4 MB/s | 746 kB 00:00 2024-03-26T18:44:38.416 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 40/198 2024-03-26T18:44:38.445 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 41/198 2024-03-26T18:44:38.498 INFO:teuthology.orchestra.run.smithi097.stdout:(15/174): libcephsqlite-19.0.0-2311.g36051606.e 1.6 MB/s | 163 kB 00:00 2024-03-26T18:44:38.514 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T18:44:38.560 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T18:44:38.567 INFO:teuthology.orchestra.run.smithi105.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 212 kB/s | 39 kB 00:00 2024-03-26T18:44:38.590 INFO:teuthology.orchestra.run.smithi097.stdout:(16/174): librados-devel-19.0.0-2311.g36051606. 1.4 MB/s | 127 kB 00:00 2024-03-26T18:44:38.634 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T18:44:38.768 INFO:teuthology.orchestra.run.smithi105.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-26T18:44:38.768 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T18:44:38.773 INFO:teuthology.orchestra.run.smithi097.stdout:(17/174): libradosstriper1-19.0.0-2311.g3605160 2.7 MB/s | 500 kB 00:00 2024-03-26T18:44:38.793 INFO:teuthology.orchestra.run.smithi105.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 184 kB/s | 41 kB 00:00 2024-03-26T18:44:38.818 INFO:teuthology.orchestra.run.smithi105.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-26T18:44:38.820 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T18:44:38.843 INFO:teuthology.orchestra.run.smithi105.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-26T18:44:38.868 INFO:teuthology.orchestra.run.smithi105.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-26T18:44:38.894 INFO:teuthology.orchestra.run.smithi105.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 875 kB/s | 44 kB 00:00 2024-03-26T18:44:38.919 INFO:teuthology.orchestra.run.smithi105.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-26T18:44:38.946 INFO:teuthology.orchestra.run.smithi105.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.6 MB/s | 241 kB 00:00 2024-03-26T18:44:38.980 INFO:teuthology.orchestra.run.smithi105.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 19 MB/s | 1.1 MB 00:00 2024-03-26T18:44:39.005 INFO:teuthology.orchestra.run.smithi105.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-26T18:44:39.030 INFO:teuthology.orchestra.run.smithi105.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-26T18:44:39.055 INFO:teuthology.orchestra.run.smithi105.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-26T18:44:39.340 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T18:44:39.364 INFO:teuthology.orchestra.run.smithi105.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 562 kB/s | 187 kB 00:00 2024-03-26T18:44:39.415 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T18:44:39.473 INFO:teuthology.orchestra.run.smithi105.stdout:(88/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-03-26T18:44:39.496 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T18:44:39.536 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T18:44:39.546 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T18:44:39.576 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T18:44:39.617 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T18:44:39.639 INFO:teuthology.orchestra.run.smithi105.stdout:(89/174): ceph-mgr-diskprediction-local-19.0.0- 2.4 MB/s | 7.4 MB 00:03 2024-03-26T18:44:39.663 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 53/198 2024-03-26T18:44:39.692 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T18:44:39.723 INFO:teuthology.orchestra.run.smithi105.stdout:(90/174): elfutils-libelf-devel-0.190-2.el9.x86 261 kB/s | 21 kB 00:00 2024-03-26T18:44:39.748 INFO:teuthology.orchestra.run.smithi105.stdout:(91/174): libquadmath-11.4.1-3.el9.x86_64.rpm 509 kB/s | 195 kB 00:00 2024-03-26T18:44:39.774 INFO:teuthology.orchestra.run.smithi105.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-03-26T18:44:39.857 INFO:teuthology.orchestra.run.smithi105.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 326 kB/s | 125 kB 00:00 2024-03-26T18:44:39.924 INFO:teuthology.orchestra.run.smithi105.stdout:(94/174): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-26T18:44:39.946 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T18:44:39.966 INFO:teuthology.orchestra.run.smithi105.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 133 kB 00:00 2024-03-26T18:44:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:(96/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 14 MB/s | 4.1 MB 00:00 2024-03-26T18:44:40.083 INFO:teuthology.orchestra.run.smithi105.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 5.8 MB/s | 245 kB 00:00 2024-03-26T18:44:40.108 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T18:44:40.125 INFO:teuthology.orchestra.run.smithi105.stdout:(98/174): postgresql-contrib-13.14-1.el9.x86_64 3.3 MB/s | 911 kB 00:00 2024-03-26T18:44:40.150 INFO:teuthology.orchestra.run.smithi105.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-03-26T18:44:40.175 INFO:teuthology.orchestra.run.smithi105.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-03-26T18:44:40.178 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T18:44:40.192 INFO:teuthology.orchestra.run.smithi105.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 855 kB/s | 36 kB 00:00 2024-03-26T18:44:40.198 INFO:teuthology.orchestra.run.smithi097.stdout:(18/174): ceph-radosgw-19.0.0-2311.g36051606.el 3.3 MB/s | 9.8 MB 00:03 2024-03-26T18:44:40.209 INFO:teuthology.orchestra.run.smithi105.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-03-26T18:44:40.222 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T18:44:40.235 INFO:teuthology.orchestra.run.smithi105.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-03-26T18:44:40.315 INFO:teuthology.orchestra.run.smithi097.stdout:(19/174): librgw2-19.0.0-2311.g36051606.el9.x86 3.3 MB/s | 5.1 MB 00:01 2024-03-26T18:44:40.340 INFO:teuthology.orchestra.run.smithi097.stdout:(20/174): python3-ceph-argparse-19.0.0-2311.g36 320 kB/s | 45 kB 00:00 2024-03-26T18:44:40.343 INFO:teuthology.orchestra.run.smithi105.stdout:(104/174): postgresql-server-13.14-1.el9.x86_64 15 MB/s | 5.8 MB 00:00 2024-03-26T18:44:40.355 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T18:44:40.366 INFO:teuthology.orchestra.run.smithi097.stdout:(21/174): python3-ceph-common-19.0.0-2311.g3605 2.6 MB/s | 131 kB 00:00 2024-03-26T18:44:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T18:44:40.399 INFO:teuthology.orchestra.run.smithi097.stdout:(22/174): python3-cephfs-19.0.0-2311.g36051606. 2.7 MB/s | 161 kB 00:00 2024-03-26T18:44:40.446 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T18:44:40.449 INFO:teuthology.orchestra.run.smithi097.stdout:(23/174): python3-rados-19.0.0-2311.g36051606.e 3.8 MB/s | 321 kB 00:00 2024-03-26T18:44:40.483 INFO:teuthology.orchestra.run.smithi097.stdout:(24/174): python3-rbd-19.0.0-2311.g36051606.el9 3.5 MB/s | 298 kB 00:00 2024-03-26T18:44:40.511 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T18:44:40.527 INFO:teuthology.orchestra.run.smithi105.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 70 kB/s | 22 kB 00:00 2024-03-26T18:44:40.553 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T18:44:40.590 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T18:44:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T18:44:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T18:44:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T18:44:40.727 INFO:teuthology.orchestra.run.smithi105.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-03-26T18:44:40.786 INFO:teuthology.orchestra.run.smithi105.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-03-26T18:44:40.819 INFO:teuthology.orchestra.run.smithi105.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-03-26T18:44:40.878 INFO:teuthology.orchestra.run.smithi105.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-03-26T18:44:40.937 INFO:teuthology.orchestra.run.smithi105.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-03-26T18:44:40.995 INFO:teuthology.orchestra.run.smithi105.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-26T18:44:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:(25/174): ceph-test-19.0.0-2311.g36051606.el9.x 15 MB/s | 48 MB 00:03 2024-03-26T18:44:41.021 INFO:teuthology.orchestra.run.smithi105.stdout:(112/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-03-26T18:44:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:(26/174): python3-rgw-19.0.0-2311.g36051606.el9 173 kB/s | 99 kB 00:00 2024-03-26T18:44:41.028 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T18:44:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:(27/174): rbd-fuse-19.0.0-2311.g36051606.el9.x8 150 kB/s | 85 kB 00:00 2024-03-26T18:44:41.054 INFO:teuthology.orchestra.run.smithi105.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-03-26T18:44:41.101 INFO:teuthology.orchestra.run.smithi097.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2311.g 686 kB/s | 34 kB 00:00 2024-03-26T18:44:41.134 INFO:teuthology.orchestra.run.smithi097.stdout:(29/174): rbd-nbd-19.0.0-2311.g36051606.el9.x86 1.5 MB/s | 170 kB 00:00 2024-03-26T18:44:41.171 INFO:teuthology.orchestra.run.smithi105.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-03-26T18:44:41.174 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T18:44:41.222 INFO:teuthology.orchestra.run.smithi105.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-03-26T18:44:41.230 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T18:44:41.246 INFO:teuthology.orchestra.run.smithi105.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 590 kB/s | 43 kB 00:00 2024-03-26T18:44:41.260 INFO:teuthology.orchestra.run.smithi097.stdout:(30/174): rbd-mirror-19.0.0-2311.g36051606.el9. 13 MB/s | 3.1 MB 00:00 2024-03-26T18:44:41.268 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T18:44:41.280 INFO:teuthology.orchestra.run.smithi105.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-03-26T18:44:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:(31/174): ceph-mgr-cephadm-19.0.0-2311.g3605160 765 kB/s | 140 kB 00:00 2024-03-26T18:44:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T18:44:41.314 INFO:teuthology.orchestra.run.smithi105.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 223 kB/s | 15 kB 00:00 2024-03-26T18:44:41.348 INFO:teuthology.orchestra.run.smithi105.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-26T18:44:41.381 INFO:teuthology.orchestra.run.smithi105.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-03-26T18:44:41.405 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T18:44:41.440 INFO:teuthology.orchestra.run.smithi105.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 604 kB/s | 35 kB 00:00 2024-03-26T18:44:41.444 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T18:44:41.444 INFO:teuthology.orchestra.run.smithi097.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2311.g36 1.5 MB/s | 248 kB 00:00 2024-03-26T18:44:41.465 INFO:teuthology.orchestra.run.smithi105.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-03-26T18:44:41.483 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T18:44:41.499 INFO:teuthology.orchestra.run.smithi105.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-26T18:44:41.509 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T18:44:41.534 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T18:44:41.536 INFO:teuthology.orchestra.run.smithi097.stdout:(33/174): ceph-mgr-rook-19.0.0-2311.g36051606.e 543 kB/s | 49 kB 00:00 2024-03-26T18:44:41.557 INFO:teuthology.orchestra.run.smithi105.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-26T18:44:41.586 INFO:teuthology.orchestra.run.smithi097.stdout:(34/174): ceph-mgr-dashboard-19.0.0-2311.g36051 4.0 MB/s | 1.8 MB 00:00 2024-03-26T18:44:41.606 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T18:44:41.611 INFO:teuthology.orchestra.run.smithi097.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2311.g3 221 kB/s | 16 kB 00:00 2024-03-26T18:44:41.616 INFO:teuthology.orchestra.run.smithi105.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-03-26T18:44:41.641 INFO:teuthology.orchestra.run.smithi105.stdout:(126/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.3 MB/s | 838 kB 00:00 2024-03-26T18:44:41.645 INFO:teuthology.orchestra.run.smithi097.stdout:(36/174): ceph-volume-19.0.0-2311.g36051606.el9 4.5 MB/s | 264 kB 00:00 2024-03-26T18:44:41.675 INFO:teuthology.orchestra.run.smithi105.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-03-26T18:44:41.695 INFO:teuthology.orchestra.run.smithi097.stdout:(37/174): cephfs-top-19.0.0-2311.g36051606.el9. 372 kB/s | 18 kB 00:00 2024-03-26T18:44:41.700 INFO:teuthology.orchestra.run.smithi105.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-03-26T18:44:41.734 INFO:teuthology.orchestra.run.smithi105.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-03-26T18:44:41.735 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T18:44:41.759 INFO:teuthology.orchestra.run.smithi105.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-03-26T18:44:41.801 INFO:teuthology.orchestra.run.smithi105.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-03-26T18:44:41.820 INFO:teuthology.orchestra.run.smithi097.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-03-26T18:44:41.826 INFO:teuthology.orchestra.run.smithi105.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 694 kB/s | 46 kB 00:00 2024-03-26T18:44:41.833 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T18:44:41.860 INFO:teuthology.orchestra.run.smithi105.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-26T18:44:41.876 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T18:44:41.893 INFO:teuthology.orchestra.run.smithi105.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-03-26T18:44:41.918 INFO:teuthology.orchestra.run.smithi105.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-03-26T18:44:41.924 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T18:44:41.929 INFO:teuthology.orchestra.run.smithi097.stdout:(39/174): cephadm-19.0.0-2311.g36051606.el9.noa 2.3 MB/s | 750 kB 00:00 2024-03-26T18:44:41.944 INFO:teuthology.orchestra.run.smithi105.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-26T18:44:41.969 INFO:teuthology.orchestra.run.smithi105.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-03-26T18:44:41.979 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T18:44:41.994 INFO:teuthology.orchestra.run.smithi105.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-26T18:44:42.011 INFO:teuthology.orchestra.run.smithi105.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 420 kB/s | 28 kB 00:00 2024-03-26T18:44:42.028 INFO:teuthology.orchestra.run.smithi105.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 524 kB/s | 31 kB 00:00 2024-03-26T18:44:42.070 INFO:teuthology.orchestra.run.smithi105.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-03-26T18:44:42.097 INFO:teuthology.orchestra.run.smithi105.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 517 kB/s | 36 kB 00:00 2024-03-26T18:44:42.131 INFO:teuthology.orchestra.run.smithi105.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-03-26T18:44:42.157 INFO:teuthology.orchestra.run.smithi105.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 592 kB/s | 51 kB 00:00 2024-03-26T18:44:42.171 INFO:teuthology.orchestra.run.smithi097.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 920 kB 00:00 2024-03-26T18:44:42.182 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T18:44:42.182 INFO:teuthology.orchestra.run.smithi105.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-26T18:44:42.205 INFO:teuthology.orchestra.run.smithi097.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 1.3 MB/s | 662 kB 00:00 2024-03-26T18:44:42.207 INFO:teuthology.orchestra.run.smithi105.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-26T18:44:42.230 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T18:44:42.230 INFO:teuthology.orchestra.run.smithi097.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 436 kB/s | 106 kB 00:00 2024-03-26T18:44:42.241 INFO:teuthology.orchestra.run.smithi105.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-03-26T18:44:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-26T18:44:42.288 INFO:teuthology.orchestra.run.smithi097.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 307 kB 00:00 2024-03-26T18:44:42.313 INFO:teuthology.orchestra.run.smithi097.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 392 kB/s | 32 kB 00:00 2024-03-26T18:44:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-03-26T18:44:42.375 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T18:44:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-26T18:44:42.411 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T18:44:42.425 INFO:teuthology.orchestra.run.smithi105.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-03-26T18:44:42.444 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T18:44:42.464 INFO:teuthology.orchestra.run.smithi097.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 99 kB/s | 17 kB 00:00 2024-03-26T18:44:42.475 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T18:44:42.494 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T18:44:42.506 INFO:teuthology.orchestra.run.smithi097.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 53 kB/s | 10 kB 00:00 2024-03-26T18:44:42.528 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T18:44:42.567 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T18:44:42.599 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T18:44:42.629 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T18:44:42.631 INFO:teuthology.orchestra.run.smithi097.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.5 MB/s | 3.0 MB 00:00 2024-03-26T18:44:42.656 INFO:teuthology.orchestra.run.smithi097.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 848 kB/s | 159 kB 00:00 2024-03-26T18:44:42.681 INFO:teuthology.orchestra.run.smithi097.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 259 kB/s | 44 kB 00:00 2024-03-26T18:44:42.705 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T18:44:42.732 INFO:teuthology.orchestra.run.smithi097.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.6 MB/s | 661 kB 00:00 2024-03-26T18:44:42.739 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T18:44:42.774 INFO:teuthology.orchestra.run.smithi097.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.6 MB/s | 578 kB 00:00 2024-03-26T18:44:42.775 INFO:teuthology.orchestra.run.smithi105.stdout:(152/174): librados2-19.0.0-2311.g36051606.el9. 6.8 MB/s | 3.4 MB 00:00 2024-03-26T18:44:42.781 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T18:44:42.799 INFO:teuthology.orchestra.run.smithi097.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 419 kB/s | 46 kB 00:00 2024-03-26T18:44:42.822 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T18:44:42.832 INFO:teuthology.orchestra.run.smithi097.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 282 kB 00:00 2024-03-26T18:44:42.857 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T18:44:42.866 INFO:teuthology.orchestra.run.smithi097.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-26T18:44:42.916 INFO:teuthology.orchestra.run.smithi097.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.9 MB/s | 217 kB 00:00 2024-03-26T18:44:42.942 INFO:teuthology.orchestra.run.smithi097.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 326 kB/s | 34 kB 00:00 2024-03-26T18:44:42.946 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T18:44:42.967 INFO:teuthology.orchestra.run.smithi105.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 4.3 MB/s | 2.3 MB 00:00 2024-03-26T18:44:43.045 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T18:44:43.092 INFO:teuthology.orchestra.run.smithi097.stdout:(57/174): python3-attrs-20.3.0-7.el9.noarch.rpm 531 kB/s | 78 kB 00:00 2024-03-26T18:44:43.096 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T18:44:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T18:44:43.134 INFO:teuthology.orchestra.run.smithi097.stdout:(58/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-03-26T18:44:43.175 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T18:44:43.234 INFO:teuthology.orchestra.run.smithi097.stdout:(59/174): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-03-26T18:44:43.243 INFO:teuthology.orchestra.run.smithi105.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 153 kB/s | 42 kB 00:00 2024-03-26T18:44:43.272 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T18:44:43.293 INFO:teuthology.orchestra.run.smithi105.stdout:(155/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.5 MB/s | 1.8 MB 00:00 2024-03-26T18:44:43.321 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T18:44:43.359 INFO:teuthology.orchestra.run.smithi097.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-03-26T18:44:43.368 INFO:teuthology.orchestra.run.smithi105.stdout:(156/174): librbd1-19.0.0-2311.g36051606.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-03-26T18:44:43.408 INFO:teuthology.orchestra.run.smithi097.stdout:(61/174): python3-jsonpatch-1.21-16.el9.noarch. 98 kB/s | 24 kB 00:00 2024-03-26T18:44:43.435 INFO:teuthology.orchestra.run.smithi097.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 96 kB/s | 18 kB 00:00 2024-03-26T18:44:43.477 INFO:teuthology.orchestra.run.smithi097.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.5 MB/s | 144 kB 00:00 2024-03-26T18:44:43.502 INFO:teuthology.orchestra.run.smithi097.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 483 kB/s | 32 kB 00:00 2024-03-26T18:44:43.627 INFO:teuthology.orchestra.run.smithi105.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 231 kB/s | 71 kB 00:00 2024-03-26T18:44:43.635 INFO:teuthology.orchestra.run.smithi097.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2024-03-26T18:44:43.652 INFO:teuthology.orchestra.run.smithi105.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 141 kB/s | 36 kB 00:00 2024-03-26T18:44:43.660 INFO:teuthology.orchestra.run.smithi097.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-26T18:44:43.677 INFO:teuthology.orchestra.run.smithi105.stdout:(159/174): NetworkManager-tui-1.47.2-1.el9.x86_ 572 kB/s | 248 kB 00:00 2024-03-26T18:44:43.686 INFO:teuthology.orchestra.run.smithi097.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-26T18:44:43.719 INFO:teuthology.orchestra.run.smithi097.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 2.1 MB/s | 69 kB 00:00 2024-03-26T18:44:43.745 INFO:teuthology.orchestra.run.smithi097.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 41 kB 00:00 2024-03-26T18:44:43.778 INFO:teuthology.orchestra.run.smithi097.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 238 kB 00:00 2024-03-26T18:44:43.812 INFO:teuthology.orchestra.run.smithi097.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-03-26T18:44:43.837 INFO:teuthology.orchestra.run.smithi097.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2024-03-26T18:44:43.862 INFO:teuthology.orchestra.run.smithi097.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 588 kB/s | 47 kB 00:00 2024-03-26T18:44:43.927 INFO:teuthology.orchestra.run.smithi097.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 487 kB/s | 43 kB 00:00 2024-03-26T18:44:43.936 INFO:teuthology.orchestra.run.smithi105.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 623 kB/s | 192 kB 00:00 2024-03-26T18:44:43.961 INFO:teuthology.orchestra.run.smithi105.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 97 kB/s | 23 kB 00:00 2024-03-26T18:44:44.050 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T18:44:44.053 INFO:teuthology.orchestra.run.smithi105.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 638 kB/s | 256 kB 00:00 2024-03-26T18:44:44.088 INFO:teuthology.orchestra.run.smithi097.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 184 kB/s | 39 kB 00:00 2024-03-26T18:44:44.093 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T18:44:44.127 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T18:44:44.175 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T18:44:44.262 INFO:teuthology.orchestra.run.smithi105.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 333 kB/s | 108 kB 00:00 2024-03-26T18:44:44.263 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T18:44:44.287 INFO:teuthology.orchestra.run.smithi105.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 475 kB/s | 155 kB 00:00 2024-03-26T18:44:44.288 INFO:teuthology.orchestra.run.smithi097.stdout:(76/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-26T18:44:44.294 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T18:44:44.305 INFO:teuthology.orchestra.run.smithi097.stdout:(77/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 112 kB/s | 41 kB 00:00 2024-03-26T18:44:44.338 INFO:teuthology.orchestra.run.smithi097.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2024-03-26T18:44:44.355 INFO:teuthology.orchestra.run.smithi097.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 1.2 MB/s | 56 kB 00:00 2024-03-26T18:44:44.372 INFO:teuthology.orchestra.run.smithi097.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 995 kB/s | 63 kB 00:00 2024-03-26T18:44:44.389 INFO:teuthology.orchestra.run.smithi097.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 909 kB/s | 44 kB 00:00 2024-03-26T18:44:44.404 INFO:teuthology.orchestra.run.smithi105.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 387 kB/s | 136 kB 00:00 2024-03-26T18:44:44.445 INFO:teuthology.orchestra.run.smithi097.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2024-03-26T18:44:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T18:44:44.473 INFO:teuthology.orchestra.run.smithi097.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.4 MB/s | 241 kB 00:00 2024-03-26T18:44:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T18:44:44.504 INFO:teuthology.orchestra.run.smithi105.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 132 kB/s | 29 kB 00:00 2024-03-26T18:44:44.529 INFO:teuthology.orchestra.run.smithi105.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 239 kB/s | 64 kB 00:00 2024-03-26T18:44:44.531 INFO:teuthology.orchestra.run.smithi097.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 8.3 MB/s | 1.1 MB 00:00 2024-03-26T18:44:44.557 INFO:teuthology.orchestra.run.smithi097.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-26T18:44:44.578 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T18:44:44.582 INFO:teuthology.orchestra.run.smithi097.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-26T18:44:44.615 INFO:teuthology.orchestra.run.smithi097.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 187 kB 00:00 2024-03-26T18:44:44.741 INFO:teuthology.orchestra.run.smithi097.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-26T18:44:44.774 INFO:teuthology.orchestra.run.smithi097.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-26T18:44:44.808 INFO:teuthology.orchestra.run.smithi097.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-26T18:44:44.858 INFO:teuthology.orchestra.run.smithi097.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-03-26T18:44:44.888 INFO:teuthology.orchestra.run.smithi105.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 2.5 MB/s | 1.2 MB 00:00 2024-03-26T18:44:45.088 INFO:teuthology.orchestra.run.smithi105.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.7 MB/s | 2.2 MB 00:00 2024-03-26T18:44:45.155 INFO:teuthology.orchestra.run.smithi105.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.7 MB/s | 2.3 MB 00:00 2024-03-26T18:44:45.218 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T18:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 1.1 MB/s | 465 kB 00:00 2024-03-26T18:44:45.331 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T18:44:45.364 INFO:teuthology.orchestra.run.smithi105.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 113 kB/s | 31 kB 00:00 2024-03-26T18:44:45.389 INFO:teuthology.orchestra.run.smithi105.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 904 kB/s | 74 kB 00:00 2024-03-26T18:44:45.475 INFO:teuthology.orchestra.run.smithi097.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 31 kB/s | 21 kB 00:00 2024-03-26T18:44:45.500 INFO:teuthology.orchestra.run.smithi097.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 37 kB/s | 25 kB 00:00 2024-03-26T18:44:45.765 INFO:teuthology.orchestra.run.smithi105.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 1.1 MB 00:00 2024-03-26T18:44:45.771 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-03-26T18:44:45.771 INFO:teuthology.orchestra.run.smithi105.stdout:Total 14 MB/s | 225 MB 00:15 2024-03-26T18:44:46.316 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-03-26T18:44:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T18:44:46.587 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-03-26T18:44:46.587 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-03-26T18:44:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T18:44:46.641 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T18:44:46.672 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T18:44:46.699 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T18:44:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T18:44:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T18:44:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T18:44:47.142 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 125/198 2024-03-26T18:44:47.179 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T18:44:47.376 INFO:teuthology.orchestra.run.smithi097.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 486 kB/s | 911 kB 00:01 2024-03-26T18:44:47.639 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T18:44:47.674 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T18:44:47.770 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T18:44:47.801 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 129/198 2024-03-26T18:44:48.218 INFO:teuthology.orchestra.run.smithi097.stdout:(95/174): postgresql-13.14-1.el9.x86_64.rpm 594 kB/s | 1.6 MB 00:02 2024-03-26T18:44:48.276 INFO:teuthology.orchestra.run.smithi097.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 148 kB/s | 133 kB 00:00 2024-03-26T18:44:48.322 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-03-26T18:44:48.323 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-03-26T18:44:49.302 INFO:teuthology.orchestra.run.smithi097.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 240 kB/s | 245 kB 00:01 2024-03-26T18:44:49.990 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T18:44:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T18:44:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 240 kB/s | 248 kB 00:01 2024-03-26T18:44:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T18:44:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T18:44:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T18:44:50.674 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T18:44:50.674 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T18:44:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:44:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T18:44:51.109 INFO:teuthology.orchestra.run.smithi097.stdout:(99/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 677 kB/s | 4.1 MB 00:06 2024-03-26T18:44:51.244 INFO:teuthology.orchestra.run.smithi097.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 243 kB/s | 221 kB 00:00 2024-03-26T18:44:51.869 INFO:teuthology.orchestra.run.smithi097.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 47 kB/s | 36 kB 00:00 2024-03-26T18:44:51.978 INFO:teuthology.orchestra.run.smithi097.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 65 kB/s | 47 kB 00:00 2024-03-26T18:44:52.103 INFO:teuthology.orchestra.run.smithi097.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 180 kB/s | 22 kB 00:00 2024-03-26T18:44:52.553 INFO:teuthology.orchestra.run.smithi097.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-03-26T18:44:52.920 INFO:teuthology.orchestra.run.smithi097.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 89 kB/s | 94 kB 00:01 2024-03-26T18:44:53.229 INFO:teuthology.orchestra.run.smithi097.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 813 kB/s | 548 kB 00:00 2024-03-26T18:44:53.437 INFO:teuthology.orchestra.run.smithi097.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 599 kB/s | 309 kB 00:00 2024-03-26T18:44:53.496 INFO:teuthology.orchestra.run.smithi097.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-03-26T18:44:53.763 INFO:teuthology.orchestra.run.smithi097.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 98 kB/s | 26 kB 00:00 2024-03-26T18:44:54.122 INFO:teuthology.orchestra.run.smithi097.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 136 kB/s | 49 kB 00:00 2024-03-26T18:44:54.472 INFO:teuthology.orchestra.run.smithi097.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 193 kB/s | 67 kB 00:00 2024-03-26T18:44:54.572 INFO:teuthology.orchestra.run.smithi097.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-03-26T18:44:54.914 INFO:teuthology.orchestra.run.smithi097.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 342 kB/s | 151 kB 00:00 2024-03-26T18:44:55.264 INFO:teuthology.orchestra.run.smithi097.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-03-26T18:44:55.316 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-03-26T18:44:55.418 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T18:44:55.444 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T18:44:55.475 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T18:44:55.484 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T18:44:55.515 INFO:teuthology.orchestra.run.smithi097.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 914 kB/s | 548 kB 00:00 2024-03-26T18:44:55.525 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T18:44:55.577 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T18:44:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T18:44:55.615 INFO:teuthology.orchestra.run.smithi097.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-03-26T18:44:55.646 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T18:44:55.688 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T18:44:55.807 INFO:teuthology.orchestra.run.smithi097.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 111 kB/s | 32 kB 00:00 2024-03-26T18:44:55.882 INFO:teuthology.orchestra.run.smithi097.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-03-26T18:44:56.008 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T18:44:56.064 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T18:44:56.185 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T18:44:56.215 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T18:44:56.241 INFO:teuthology.orchestra.run.smithi097.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 397 kB/s | 172 kB 00:00 2024-03-26T18:44:56.307 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T18:44:56.336 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T18:44:56.373 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T18:44:56.483 INFO:teuthology.orchestra.run.smithi097.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 598 kB/s | 358 kB 00:00 2024-03-26T18:44:56.552 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T18:44:56.608 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T18:44:56.637 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T18:44:56.750 INFO:teuthology.orchestra.run.smithi097.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 133 kB/s | 35 kB 00:00 2024-03-26T18:44:56.765 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T18:44:56.787 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T18:44:56.833 INFO:teuthology.orchestra.run.smithi097.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 512 kB/s | 303 kB 00:00 2024-03-26T18:44:56.852 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T18:44:56.900 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T18:44:56.923 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T18:44:56.938 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T18:44:57.050 INFO:teuthology.orchestra.run.smithi097.stdout:(123/174): postgresql-server-13.14-1.el9.x86_64 670 kB/s | 5.8 MB 00:08 2024-03-26T18:44:57.080 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T18:44:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T18:44:57.152 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T18:44:57.255 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T18:44:57.275 INFO:teuthology.orchestra.run.smithi097.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 386 kB/s | 203 kB 00:00 2024-03-26T18:44:57.354 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T18:44:57.414 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T18:44:57.468 INFO:teuthology.orchestra.run.smithi097.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-03-26T18:44:57.482 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T18:44:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 327 kB/s | 144 kB 00:00 2024-03-26T18:44:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T18:44:57.693 INFO:teuthology.orchestra.run.smithi097.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-03-26T18:44:57.748 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T18:44:57.752 INFO:teuthology.orchestra.run.smithi097.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 62 kB/s | 18 kB 00:00 2024-03-26T18:44:57.777 INFO:teuthology.orchestra.run.smithi097.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 82 kB/s | 23 kB 00:00 2024-03-26T18:44:57.853 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T18:44:57.940 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T18:44:57.969 INFO:teuthology.orchestra.run.smithi097.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 71 kB/s | 19 kB 00:00 2024-03-26T18:44:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T18:44:57.997 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T18:44:58.019 INFO:teuthology.orchestra.run.smithi097.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-03-26T18:44:58.032 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T18:44:58.107 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T18:44:58.161 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T18:44:58.199 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T18:44:58.264 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T18:44:58.298 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T18:44:58.319 INFO:teuthology.orchestra.run.smithi097.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-03-26T18:44:58.342 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T18:44:58.361 INFO:teuthology.orchestra.run.smithi097.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 231 kB/s | 79 kB 00:00 2024-03-26T18:44:58.374 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T18:44:58.402 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T18:44:58.472 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T18:44:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 39/198 2024-03-26T18:44:58.537 INFO:teuthology.orchestra.run.smithi097.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 1.3 MB/s | 1.0 MB 00:00 2024-03-26T18:44:58.550 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 40/198 2024-03-26T18:44:58.579 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 41/198 2024-03-26T18:44:58.649 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T18:44:58.670 INFO:teuthology.orchestra.run.smithi097.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-03-26T18:44:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T18:44:58.771 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T18:44:58.804 INFO:teuthology.orchestra.run.smithi097.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 62 kB/s | 16 kB 00:00 2024-03-26T18:44:58.871 INFO:teuthology.orchestra.run.smithi097.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 536 kB/s | 272 kB 00:00 2024-03-26T18:44:58.909 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T18:44:58.962 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T18:44:59.029 INFO:teuthology.orchestra.run.smithi097.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 251 kB/s | 90 kB 00:00 2024-03-26T18:44:59.071 INFO:teuthology.orchestra.run.smithi097.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 106 kB/s | 28 kB 00:00 2024-03-26T18:44:59.138 INFO:teuthology.orchestra.run.smithi097.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-03-26T18:44:59.413 INFO:teuthology.orchestra.run.smithi097.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2024-03-26T18:44:59.442 INFO:teuthology.orchestra.run.smithi097.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 161 kB/s | 59 kB 00:00 2024-03-26T18:44:59.464 INFO:teuthology.orchestra.run.smithi097.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 435 kB/s | 188 kB 00:00 2024-03-26T18:44:59.485 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T18:44:59.575 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T18:44:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T18:44:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T18:44:59.703 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T18:44:59.735 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T18:44:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T18:44:59.789 INFO:teuthology.orchestra.run.smithi097.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 136 kB/s | 51 kB 00:00 2024-03-26T18:44:59.814 INFO:teuthology.orchestra.run.smithi097.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 256 kB/s | 90 kB 00:00 2024-03-26T18:44:59.821 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 53/198 2024-03-26T18:44:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T18:45:00.056 INFO:teuthology.orchestra.run.smithi097.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 372 kB/s | 230 kB 00:00 2024-03-26T18:45:00.082 INFO:teuthology.orchestra.run.smithi097.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-03-26T18:45:00.092 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T18:45:00.271 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T18:45:00.329 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T18:45:00.375 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T18:45:00.457 INFO:teuthology.orchestra.run.smithi097.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 641 kB/s | 427 kB 00:00 2024-03-26T18:45:00.508 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T18:45:00.569 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T18:45:00.577 INFO:teuthology.orchestra.run.smithi097.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 369 kB/s | 192 kB 00:00 2024-03-26T18:45:00.624 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T18:45:00.696 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T18:45:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T18:45:00.764 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T18:45:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-03-26T18:45:00.957 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T18:45:01.053 INFO:teuthology.orchestra.run.smithi097.stdout:(151/174): librados2-19.0.0-2311.g36051606.el9. 5.7 MB/s | 3.4 MB 00:00 2024-03-26T18:45:01.093 INFO:teuthology.orchestra.run.smithi097.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.5 MB/s | 321 kB 00:00 2024-03-26T18:45:01.135 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T18:45:01.245 INFO:teuthology.orchestra.run.smithi097.stdout:(153/174): librbd1-19.0.0-2311.g36051606.el9.x8 4.7 MB/s | 3.1 MB 00:00 2024-03-26T18:45:01.323 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T18:45:01.430 INFO:teuthology.orchestra.run.smithi097.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 230 kB/s | 42 kB 00:00 2024-03-26T18:45:01.435 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T18:45:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T18:45:01.613 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T18:45:01.644 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T18:45:01.654 INFO:teuthology.orchestra.run.smithi097.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.1 MB/s | 248 kB 00:00 2024-03-26T18:45:01.685 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T18:45:01.729 INFO:teuthology.orchestra.run.smithi097.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 963 kB/s | 71 kB 00:00 2024-03-26T18:45:01.777 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T18:45:01.796 INFO:teuthology.orchestra.run.smithi097.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 554 kB/s | 36 kB 00:00 2024-03-26T18:45:01.818 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T18:45:01.858 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T18:45:01.863 INFO:teuthology.orchestra.run.smithi097.stdout:(158/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.9 MB/s | 2.3 MB 00:00 2024-03-26T18:45:01.884 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T18:45:01.905 INFO:teuthology.orchestra.run.smithi097.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-03-26T18:45:01.908 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T18:45:01.964 INFO:teuthology.orchestra.run.smithi097.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 395 kB/s | 23 kB 00:00 2024-03-26T18:45:01.980 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T18:45:01.998 INFO:teuthology.orchestra.run.smithi097.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.9 MB/s | 256 kB 00:00 2024-03-26T18:45:02.031 INFO:teuthology.orchestra.run.smithi097.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-26T18:45:02.065 INFO:teuthology.orchestra.run.smithi097.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-03-26T18:45:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T18:45:02.124 INFO:teuthology.orchestra.run.smithi097.stdout:(164/174): NetworkManager-libnm-1.47.2-1.el9.x8 1.8 MB/s | 1.8 MB 00:01 2024-03-26T18:45:02.149 INFO:teuthology.orchestra.run.smithi097.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-26T18:45:02.165 INFO:teuthology.orchestra.run.smithi097.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 642 kB/s | 64 kB 00:00 2024-03-26T18:45:02.182 INFO:teuthology.orchestra.run.smithi097.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 495 kB/s | 29 kB 00:00 2024-03-26T18:45:02.199 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T18:45:02.250 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T18:45:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T18:45:02.353 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T18:45:02.491 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T18:45:02.491 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-03-26T18:45:02.491 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-03-26T18:45:02.491 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-03-26T18:45:02.492 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-03-26T18:45:02.492 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-03-26T18:45:02.492 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-03-26T18:45:02.492 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-03-26T18:45:02.492 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T18:45:02.549 INFO:teuthology.orchestra.run.smithi097.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.0 MB/s | 1.2 MB 00:00 2024-03-26T18:45:02.579 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T18:45:02.671 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T18:45:02.687 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T18:45:02.687 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:02.687 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T18:45:02.687 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:02.707 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T18:45:02.741 INFO:teuthology.orchestra.run.smithi097.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.8 MB/s | 2.2 MB 00:00 2024-03-26T18:45:02.752 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T18:45:02.775 INFO:teuthology.orchestra.run.smithi097.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.0 MB/s | 465 kB 00:00 2024-03-26T18:45:02.784 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T18:45:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T18:45:02.843 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T18:45:02.853 INFO:teuthology.orchestra.run.smithi097.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.4 MB/s | 2.3 MB 00:00 2024-03-26T18:45:02.877 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T18:45:02.884 INFO:teuthology.orchestra.run.smithi097.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 220 kB/s | 31 kB 00:00 2024-03-26T18:45:02.916 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T18:45:02.948 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T18:45:02.978 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T18:45:03.045 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T18:45:03.050 INFO:teuthology.orchestra.run.smithi097.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.9 MB/s | 1.1 MB 00:00 2024-03-26T18:45:03.080 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T18:45:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T18:45:03.154 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T18:45:03.181 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T18:45:03.269 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T18:45:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T18:45:03.404 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T18:45:03.443 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T18:45:03.490 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T18:45:03.586 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T18:45:03.636 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T18:45:03.684 INFO:teuthology.orchestra.run.smithi097.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 90 kB/s | 74 kB 00:00 2024-03-26T18:45:03.694 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-26T18:45:03.695 INFO:teuthology.orchestra.run.smithi097.stdout:Total 7.3 MB/s | 225 MB 00:30 2024-03-26T18:45:04.171 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T18:45:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T18:45:04.196 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:04.196 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T18:45:04.196 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T18:45:04.197 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T18:45:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:04.357 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T18:45:04.408 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T18:45:04.412 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T18:45:04.435 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T18:45:04.435 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:04.435 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T18:45:04.435 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T18:45:04.435 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T18:45:04.436 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:04.442 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T18:45:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T18:45:04.571 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T18:45:04.610 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T18:45:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T18:45:04.816 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T18:45:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T18:45:04.893 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T18:45:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T18:45:04.903 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:04.903 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T18:45:04.903 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T18:45:04.903 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T18:45:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:04.990 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T18:45:05.337 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-26T18:45:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T18:45:05.406 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T18:45:05.438 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 139/198 2024-03-26T18:45:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 140/198 2024-03-26T18:45:05.527 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T18:45:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T18:45:05.638 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T18:45:05.717 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-26T18:45:05.717 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-26T18:45:05.764 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T18:45:05.796 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T18:45:06.765 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T18:45:06.788 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T18:45:06.848 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T18:45:06.889 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T18:45:06.925 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T18:45:06.956 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T18:45:06.987 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T18:45:07.015 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T18:45:07.042 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 144/198 2024-03-26T18:45:07.065 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T18:45:07.089 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T18:45:07.089 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:07.089 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T18:45:07.089 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T18:45:07.089 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T18:45:07.089 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T18:45:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T18:45:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.0.0-2311.g36051606.el9.x86_64 147/198 2024-03-26T18:45:07.327 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T18:45:07.353 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T18:45:07.443 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T18:45:07.475 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 125/198 2024-03-26T18:45:07.510 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T18:45:07.971 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T18:45:07.998 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T18:45:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T18:45:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T18:45:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T18:45:08.091 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 129/198 2024-03-26T18:45:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T18:45:08.099 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:08.099 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T18:45:08.099 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T18:45:08.099 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T18:45:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T18:45:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-26T18:45:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-26T18:45:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T18:45:08.166 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:08.166 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T18:45:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:08.211 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T18:45:08.233 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T18:45:08.233 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:08.233 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T18:45:08.234 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T18:45:08.234 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T18:45:08.234 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:08.537 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T18:45:08.559 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T18:45:08.560 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:08.560 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T18:45:08.560 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T18:45:08.560 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T18:45:08.560 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T18:45:10.271 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T18:45:10.808 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T18:45:10.841 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T18:45:10.905 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T18:45:10.905 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T18:45:10.905 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T18:45:10.905 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:10.934 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T18:45:13.016 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T18:45:13.057 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 154/198 2024-03-26T18:45:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 155/198 2024-03-26T18:45:13.115 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T18:45:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 157/198 2024-03-26T18:45:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T18:45:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 159/198 2024-03-26T18:45:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T18:45:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T18:45:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T18:45:13.631 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 163/198 2024-03-26T18:45:13.671 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T18:45:13.701 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T18:45:13.755 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T18:45:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T18:45:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T18:45:14.432 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T18:45:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T18:45:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T18:45:15.334 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 170/198 2024-03-26T18:45:15.418 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T18:45:15.458 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T18:45:15.540 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T18:45:15.568 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-26T18:45:15.580 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T18:45:15.613 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T18:45:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T18:45:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T18:45:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T18:45:15.683 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T18:45:15.723 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T18:45:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T18:45:15.769 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T18:45:15.791 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T18:45:15.791 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T18:45:15.813 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T18:45:15.876 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T18:45:15.900 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T18:45:15.901 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T18:45:15.927 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T18:45:15.952 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T18:45:15.977 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T18:45:15.996 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T18:45:16.010 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T18:45:16.049 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T18:45:16.061 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T18:45:16.082 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T18:45:16.107 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T18:45:16.110 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T18:45:16.124 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T18:45:16.149 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T18:45:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T18:45:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T18:45:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T18:45:16.199 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T18:45:16.361 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T18:45:16.361 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T18:45:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T18:45:16.469 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T18:45:16.500 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T18:45:16.528 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T18:45:16.562 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T18:45:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T18:45:16.625 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T18:45:16.719 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T18:45:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T18:45:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T18:45:17.046 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T18:45:17.078 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T18:45:17.143 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T18:45:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T18:45:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T18:45:17.551 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T18:45:17.701 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T18:45:17.768 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T18:45:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T18:45:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T18:45:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T18:45:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T18:45:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T18:45:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T18:45:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T18:45:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T18:45:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T18:45:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T18:45:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T18:45:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T18:45:19.544 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T18:45:19.707 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T18:45:19.837 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T18:45:19.879 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T18:45:19.918 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T18:45:19.994 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T18:45:20.097 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T18:45:20.157 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T18:45:20.202 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T18:45:20.291 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T18:45:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T18:45:20.395 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T18:45:20.440 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T18:45:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T18:45:20.616 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T18:45:20.658 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 39/198 2024-03-26T18:45:20.710 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 40/198 2024-03-26T18:45:20.742 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 41/198 2024-03-26T18:45:20.854 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T18:45:20.947 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T18:45:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T18:45:21.238 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T18:45:21.301 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T18:45:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T18:45:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.0.0-2311.g36051606.el9.x86_64 1/198 2024-03-26T18:45:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2/198 2024-03-26T18:45:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 3/198 2024-03-26T18:45:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 4/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2311.g36051 5/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 6/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 7/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 8/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 9/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 13/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 14/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 15/198 2024-03-26T18:45:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T18:45:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 17/198 2024-03-26T18:45:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 18/198 2024-03-26T18:45:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 19/198 2024-03-26T18:45:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 20/198 2024-03-26T18:45:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 21/198 2024-03-26T18:45:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 22/198 2024-03-26T18:45:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 23/198 2024-03-26T18:45:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 24/198 2024-03-26T18:45:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 25/198 2024-03-26T18:45:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 26/198 2024-03-26T18:45:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 27/198 2024-03-26T18:45:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 28/198 2024-03-26T18:45:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 29/198 2024-03-26T18:45:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 30/198 2024-03-26T18:45:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 31/198 2024-03-26T18:45:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 32/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 33/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 34/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 35/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 36/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 37/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.0.0-2311.g36051606.el9.noarch 38/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 39/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T18:45:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T18:45:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T18:45:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T18:45:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T18:45:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T18:45:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T18:45:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T18:45:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T18:45:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T18:45:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T18:45:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T18:45:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T18:45:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T18:45:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T18:45:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T18:45:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T18:45:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T18:45:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T18:45:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T18:45:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T18:45:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T18:45:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T18:45:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T18:45:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T18:45:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T18:45:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T18:45:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T18:45:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T18:45:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T18:45:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T18:45:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T18:45:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T18:45:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T18:45:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T18:45:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T18:45:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T18:45:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T18:45:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T18:45:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T18:45:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T18:45:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T18:45:21.934 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T18:45:22.051 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T18:45:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T18:45:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T18:45:22.247 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T18:45:22.291 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T18:45:22.359 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T18:45:22.411 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 53/198 2024-03-26T18:45:22.451 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T18:45:22.831 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T18:45:22.832 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-03-26T18:45:22.832 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-03-26T18:45:22.832 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-03-26T18:45:22.832 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-03-26T18:45:22.832 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-03-26T18:45:22.832 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-03-26T18:45:22.832 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-03-26T18:45:22.832 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:22.905 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T18:45:22.988 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T18:45:23.002 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T18:45:23.003 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:23.003 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T18:45:23.003 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:23.119 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T18:45:23.184 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T18:45:23.256 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T18:45:23.466 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T18:45:23.545 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T18:45:23.624 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T18:45:23.727 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T18:45:23.762 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T18:45:23.801 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T18:45:23.869 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T18:45:23.909 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T18:45:23.971 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T18:45:24.038 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T18:45:24.251 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T18:45:24.300 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T18:45:24.337 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T18:45:24.386 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T18:45:24.429 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T18:45:24.429 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:24.429 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-03-26T18:45:24.429 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T18:45:24.429 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T18:45:24.429 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T18:45:24.429 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T18:45:24.429 INFO:teuthology.orchestra.run.smithi144.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T18:45:24.429 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T18:45:24.430 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:24.431 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.432 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.432 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.432 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:24.432 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:24.432 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:24.432 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:24.432 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:24.432 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.432 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.432 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T18:45:24.433 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: jq-1.6-16.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T18:45:24.434 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T18:45:24.435 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T18:45:24.436 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T18:45:24.437 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T18:45:24.438 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T18:45:24.439 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T18:45:24.440 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T18:45:24.441 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T18:45:24.442 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T18:45:24.442 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T18:45:24.442 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:45:24.442 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-26T18:45:24.482 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T18:45:24.507 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T18:45:24.507 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:24.507 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T18:45:24.507 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T18:45:24.507 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T18:45:24.507 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:24.513 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T18:45:24.569 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T18:45:24.617 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T18:45:24.652 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T18:45:24.677 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T18:45:24.744 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T18:45:24.766 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T18:45:24.766 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:24.766 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T18:45:24.766 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T18:45:24.766 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T18:45:24.766 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:24.824 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T18:45:24.914 DEBUG:teuthology.parallel:result is None 2024-03-26T18:45:25.006 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T18:45:25.139 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T18:45:25.196 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T18:45:25.212 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T18:45:25.234 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T18:45:25.234 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:25.234 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T18:45:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T18:45:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T18:45:25.235 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T18:45:25.331 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T18:45:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T18:45:25.569 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T18:45:25.642 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T18:45:25.647 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T18:45:25.679 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T18:45:25.711 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 139/198 2024-03-26T18:45:25.771 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 140/198 2024-03-26T18:45:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T18:45:25.814 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T18:45:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T18:45:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T18:45:25.934 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T18:45:25.974 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T18:45:26.011 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T18:45:26.035 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T18:45:26.070 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T18:45:26.077 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T18:45:26.138 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T18:45:26.179 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T18:45:26.308 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T18:45:26.353 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T18:45:26.388 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T18:45:26.442 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T18:45:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T18:45:26.633 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T18:45:27.001 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T18:45:27.031 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T18:45:27.055 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T18:45:27.062 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T18:45:27.091 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T18:45:27.148 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T18:45:27.162 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T18:45:27.284 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T18:45:27.316 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 144/198 2024-03-26T18:45:27.348 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T18:45:27.348 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T18:45:27.370 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T18:45:27.370 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:27.370 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T18:45:27.370 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T18:45:27.370 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T18:45:27.370 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:27.417 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T18:45:27.425 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T18:45:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.0.0-2311.g36051606.el9.x86_64 147/198 2024-03-26T18:45:28.287 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T18:45:28.340 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T18:45:28.347 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T18:45:28.375 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T18:45:28.381 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T18:45:28.397 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T18:45:28.397 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:28.397 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T18:45:28.398 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T18:45:28.398 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T18:45:28.398 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:28.439 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T18:45:28.459 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T18:45:28.480 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T18:45:28.481 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:28.481 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T18:45:28.481 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:28.526 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T18:45:28.548 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T18:45:28.548 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:28.548 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T18:45:28.548 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T18:45:28.548 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T18:45:28.549 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:28.549 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T18:45:28.598 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T18:45:28.831 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T18:45:28.852 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T18:45:28.862 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T18:45:28.874 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T18:45:28.874 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:28.874 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T18:45:28.874 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T18:45:28.875 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T18:45:28.875 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:28.959 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T18:45:29.777 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T18:45:29.922 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T18:45:31.424 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T18:45:31.533 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T18:45:31.572 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T18:45:31.603 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T18:45:31.644 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T18:45:32.019 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T18:45:32.049 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T18:45:32.152 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T18:45:32.240 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 125/198 2024-03-26T18:45:32.299 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T18:45:32.818 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T18:45:32.853 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T18:45:32.968 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 129/198 2024-03-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 154/198 2024-03-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 155/198 2024-03-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 157/198 2024-03-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 159/198 2024-03-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 163/198 2024-03-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T18:45:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T18:45:34.721 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T18:45:34.741 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T18:45:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 170/198 2024-03-26T18:45:35.607 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T18:45:35.624 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T18:45:35.643 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T18:45:35.665 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T18:45:35.745 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T18:45:35.795 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T18:45:35.820 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T18:45:35.845 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T18:45:35.872 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T18:45:35.872 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T18:45:35.960 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T18:45:35.990 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T18:45:35.990 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T18:45:36.074 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T18:45:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T18:45:36.134 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T18:45:36.159 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T18:45:36.184 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T18:45:36.217 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T18:45:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T18:45:36.282 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T18:45:36.306 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T18:45:36.330 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T18:45:36.331 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T18:45:36.356 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T18:45:36.381 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T18:45:36.406 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T18:45:36.412 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T18:45:36.412 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T18:45:36.499 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T18:45:36.499 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T18:45:36.499 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T18:45:36.499 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:45:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T18:45:36.593 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T18:45:36.593 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T18:45:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T18:45:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T18:45:36.715 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T18:45:36.743 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T18:45:36.768 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T18:45:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T18:45:41.703 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T18:45:41.703 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.0.0-2311.g36051606.el9.x86_64 1/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 3/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 4/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2311.g36051 5/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 6/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 7/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 8/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 9/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 13/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 14/198 2024-03-26T18:45:41.704 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 15/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 17/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 18/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 19/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 20/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 21/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 22/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 23/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 24/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 25/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 26/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 27/198 2024-03-26T18:45:41.705 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 28/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 29/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 30/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 31/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 32/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 33/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 34/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 35/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 36/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 37/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.0.0-2311.g36051606.el9.noarch 38/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 39/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T18:45:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T18:45:41.707 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T18:45:41.708 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T18:45:41.709 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T18:45:41.710 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T18:45:41.711 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T18:45:41.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T18:45:41.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T18:45:41.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T18:45:41.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T18:45:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T18:45:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T18:45:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T18:45:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T18:45:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T18:45:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T18:45:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T18:45:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T18:45:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T18:45:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T18:45:44.624 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T18:45:44.624 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:44.624 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-03-26T18:45:44.624 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.625 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T18:45:44.626 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:44.627 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T18:45:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.6-16.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T18:45:44.629 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T18:45:44.630 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T18:45:44.631 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T18:45:44.632 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T18:45:44.633 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T18:45:44.634 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T18:45:44.635 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T18:45:44.636 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T18:45:44.637 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T18:45:44.637 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T18:45:44.637 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T18:45:44.637 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T18:45:44.637 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:45:44.637 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-03-26T18:45:45.112 DEBUG:teuthology.parallel:result is None 2024-03-26T18:45:49.567 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T18:45:49.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-03-26T18:45:49.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-03-26T18:45:49.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-03-26T18:45:49.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-03-26T18:45:49.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-03-26T18:45:49.568 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-03-26T18:45:49.568 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-03-26T18:45:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:45:49.781 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T18:45:49.825 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T18:45:49.825 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:49.825 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T18:45:49.825 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:45:51.383 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T18:45:51.457 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T18:45:51.457 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:51.457 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T18:45:51.458 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T18:45:51.458 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T18:45:51.458 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:45:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T18:45:51.769 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T18:45:51.770 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:51.770 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T18:45:51.770 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T18:45:51.770 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T18:45:51.770 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:45:52.251 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T18:45:52.317 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T18:45:52.318 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:52.318 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T18:45:52.318 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T18:45:52.318 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T18:45:52.318 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:45:52.428 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T18:45:53.005 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T18:45:53.037 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T18:45:53.107 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 139/198 2024-03-26T18:45:53.208 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 140/198 2024-03-26T18:45:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T18:45:53.558 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T18:45:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T18:45:54.703 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T18:45:54.726 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T18:45:54.867 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T18:45:55.038 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 144/198 2024-03-26T18:45:55.069 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T18:45:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T18:45:55.140 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:55.140 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T18:45:55.141 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T18:45:55.141 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T18:45:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:45:55.212 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T18:45:55.252 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T18:45:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-2311.g36051606.el9.x86_64 147/198 2024-03-26T18:45:56.292 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T18:45:56.322 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T18:45:56.392 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T18:45:56.393 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:56.393 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T18:45:56.393 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T18:45:56.393 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T18:45:56.393 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:45:56.440 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T18:45:56.507 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T18:45:56.507 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:56.507 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T18:45:56.507 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:45:56.574 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T18:45:56.647 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T18:45:56.648 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:56.648 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T18:45:56.648 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T18:45:56.648 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T18:45:56.648 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:45:56.991 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T18:45:57.061 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T18:45:57.061 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T18:45:57.061 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T18:45:57.061 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T18:45:57.061 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T18:45:57.061 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:46:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T18:46:01.907 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 154/198 2024-03-26T18:46:01.940 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 155/198 2024-03-26T18:46:01.978 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T18:46:02.267 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 157/198 2024-03-26T18:46:02.393 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T18:46:02.441 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 159/198 2024-03-26T18:46:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T18:46:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T18:46:02.610 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T18:46:02.645 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 163/198 2024-03-26T18:46:02.682 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T18:46:02.734 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T18:46:02.835 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T18:46:02.946 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T18:46:02.983 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T18:46:03.893 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T18:46:03.939 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T18:46:03.995 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T18:46:05.310 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 170/198 2024-03-26T18:46:05.402 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T18:46:05.452 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T18:46:05.585 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T18:46:05.622 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T18:46:05.655 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T18:46:05.690 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T18:46:05.731 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T18:46:05.731 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T18:46:05.877 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T18:46:05.926 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T18:46:05.926 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T18:46:06.066 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T18:46:06.109 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T18:46:06.156 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T18:46:06.189 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T18:46:06.221 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T18:46:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T18:46:06.385 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T18:46:06.426 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T18:46:06.460 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T18:46:06.493 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T18:46:06.527 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T18:46:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T18:46:06.597 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T18:46:06.597 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T18:46:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T18:46:06.935 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T18:46:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T18:46:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T18:46:07.143 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T18:46:07.174 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T18:46:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T18:46:07.334 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T18:46:07.576 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T18:46:07.577 INFO:teuthology.orchestra.run.smithi097.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-26T18:46:07.577 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:46:12.616 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-2311.g36051606.el9.x86_64 1/198 2024-03-26T18:46:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2/198 2024-03-26T18:46:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 3/198 2024-03-26T18:46:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 4/198 2024-03-26T18:46:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2311.g36051 5/198 2024-03-26T18:46:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 6/198 2024-03-26T18:46:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 7/198 2024-03-26T18:46:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 8/198 2024-03-26T18:46:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 9/198 2024-03-26T18:46:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T18:46:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 13/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 14/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 15/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 17/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 18/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 19/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 20/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 21/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 22/198 2024-03-26T18:46:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 23/198 2024-03-26T18:46:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 24/198 2024-03-26T18:46:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 25/198 2024-03-26T18:46:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 26/198 2024-03-26T18:46:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 27/198 2024-03-26T18:46:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 28/198 2024-03-26T18:46:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 29/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 30/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 31/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 32/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 33/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 34/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 35/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 36/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 37/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-2311.g36051606.el9.noarch 38/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 39/198 2024-03-26T18:46:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T18:46:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T18:46:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T18:46:12.640 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T18:46:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T18:46:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T18:46:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T18:46:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T18:46:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T18:46:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T18:46:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T18:46:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T18:46:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T18:46:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T18:46:15.260 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T18:46:15.260 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:46:15.260 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-03-26T18:46:15.260 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T18:46:15.260 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T18:46:15.262 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.263 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T18:46:15.264 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-16.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T18:46:15.265 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T18:46:15.266 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T18:46:15.267 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T18:46:15.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T18:46:15.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T18:46:15.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T18:46:15.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T18:46:15.272 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T18:46:15.273 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T18:46:15.273 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T18:46:15.273 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T18:46:15.273 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:46:15.273 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-26T18:46:15.902 DEBUG:teuthology.parallel:result is None 2024-03-26T18:46:15.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:46:16.052 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T18:46:16.108 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-2311.g36051606.el9 2024-03-26T18:46:16.108 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2311.g36051606.el9 2024-03-26T18:46:16.108 INFO:teuthology.task.install:The correct ceph version 19.0.0-2311.g36051606 is installed. 2024-03-26T18:46:16.110 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:46:16.251 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T18:46:16.277 INFO:teuthology.orchestra.run.smithi105.stdout:19.0.0-2311.g36051606.el9 2024-03-26T18:46:16.277 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2311.g36051606.el9 2024-03-26T18:46:16.277 INFO:teuthology.task.install:The correct ceph version 19.0.0-2311.g36051606 is installed. 2024-03-26T18:46:16.279 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:46:16.419 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T18:46:16.444 INFO:teuthology.orchestra.run.smithi144.stdout:19.0.0-2311.g36051606.el9 2024-03-26T18:46:16.444 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2311.g36051606.el9 2024-03-26T18:46:16.444 INFO:teuthology.task.install:The correct ceph version 19.0.0-2311.g36051606 is installed. 2024-03-26T18:46:16.446 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-26T18:46:16.446 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:46:16.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T18:46:16.529 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:46:16.529 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T18:46:16.564 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:46:16.564 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T18:46:16.599 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-26T18:46:16.600 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:46:16.600 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T18:46:16.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T18:46:16.788 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:46:16.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T18:46:16.822 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T18:46:16.891 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:46:16.891 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T18:46:16.924 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T18:46:16.992 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-26T18:46:16.992 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:46:16.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T18:46:17.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T18:46:17.184 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:46:17.184 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T18:46:17.212 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T18:46:17.280 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:46:17.280 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T18:46:17.308 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T18:46:17.375 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-26T18:46:17.376 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:46:17.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T18:46:17.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T18:46:17.564 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:46:17.564 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T18:46:17.591 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T18:46:17.657 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:46:17.657 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T18:46:17.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T18:46:17.750 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-26T18:46:17.849 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', '\\(PG_DEGRADED', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'subvols': {'create': 2}} 2024-03-26T18:46:17.849 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:46:17.850 INFO:tasks.cephadm:Cluster fsid is 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:46:17.850 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-26T18:46:17.850 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.97', 'mon.b': '172.21.15.105', 'mon.c': '172.21.15.144'} 2024-03-26T18:46:17.850 INFO:tasks.cephadm:First mon is mon.a on smithi097 2024-03-26T18:46:17.850 INFO:tasks.cephadm:First mgr is x 2024-03-26T18:46:17.850 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-26T18:46:17.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-03-26T18:46:17.924 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname $(hostname -s) 2024-03-26T18:46:17.951 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-03-26T18:46:17.977 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-26T18:46:17.978 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:46:18.115 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77808/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2311-g36051606', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2311.g36051606'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-20 17:53:08.349574', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240320.145742', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-26T18:46:18.254 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-batrick-testing-20240320.145742, sha1 360516069d9393362c4cc6eb9371680fe16d66ab from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:46:18.256 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm 2024-03-26T18:46:18.256 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T18:46:18.968 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 769283 Mar 26 18:46 /home/ubuntu/cephtest/cephadm 2024-03-26T18:46:18.968 DEBUG:teuthology.orchestra.run.smithi105:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T18:46:19.680 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r--. 1 ubuntu ubuntu 769283 Mar 26 18:46 /home/ubuntu/cephtest/cephadm 2024-03-26T18:46:19.680 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T18:46:20.748 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r--. 1 ubuntu ubuntu 769283 Mar 26 18:46 /home/ubuntu/cephtest/cephadm 2024-03-26T18:46:20.748 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T18:46:20.809 DEBUG:teuthology.orchestra.run.smithi105:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T18:46:20.827 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T18:46:20.860 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab on all hosts... 2024-03-26T18:46:20.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab pull 2024-03-26T18:46:20.867 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab pull 2024-03-26T18:46:20.870 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab pull 2024-03-26T18:46:21.690 INFO:teuthology.orchestra.run.smithi105.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T18:46:21.741 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T18:46:22.030 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T18:46:49.728 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-26T18:46:49.728 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T18:46:49.728 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016", 2024-03-26T18:46:49.728 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-03-26T18:46:49.728 INFO:teuthology.orchestra.run.smithi144.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9" 2024-03-26T18:46:49.728 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-03-26T18:46:49.728 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-26T18:46:50.617 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:46:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T18:46:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "image_id": "497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016", 2024-03-26T18:46:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "repo_digests": [ 2024-03-26T18:46:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9" 2024-03-26T18:46:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T18:46:50.617 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:47:01.310 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:47:01.310 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T18:47:01.310 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016", 2024-03-26T18:47:01.310 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-03-26T18:47:01.310 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9" 2024-03-26T18:47:01.310 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-26T18:47:01.310 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:47:01.365 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-03-26T18:47:01.464 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph 2024-03-26T18:47:01.496 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-03-26T18:47:01.531 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-03-26T18:47:01.625 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /etc/ceph 2024-03-26T18:47:01.655 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-03-26T18:47:01.684 INFO:tasks.cephadm:Writing seed config... 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-26T18:47:01.686 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-26T18:47:01.687 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-26T18:47:01.688 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-26T18:47:01.689 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-26T18:47:01.689 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-26T18:47:01.689 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-26T18:47:01.689 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:47:01.690 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-26T18:47:01.744 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2172d50a-eba1-11ee-95d0-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-26T18:47:01.745 DEBUG:teuthology.orchestra.run.smithi097:mon.a> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.a.service 2024-03-26T18:47:01.791 DEBUG:teuthology.orchestra.run.smithi097:mgr.x> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.x.service 2024-03-26T18:47:01.836 INFO:tasks.cephadm:Bootstrapping... 2024-03-26T18:47:01.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab -v bootstrap --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.97 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:47:02.109 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-26T18:47:02.109 INFO:teuthology.orchestra.run.smithi097.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab', '-v', 'bootstrap', '--fsid', '2172d50a-eba1-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.97', '--skip-admin-label'] 2024-03-26T18:47:02.109 INFO:teuthology.orchestra.run.smithi097.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-26T18:47:02.109 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying podman|docker is present... 2024-03-26T18:47:02.171 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T18:47:02.171 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying lvm2 is present... 2024-03-26T18:47:02.172 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying time synchronization is in place... 2024-03-26T18:47:02.187 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T18:47:02.187 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T18:47:02.202 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T18:47:02.202 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-26T18:47:02.219 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2024-03-26T18:47:02.236 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2024-03-26T18:47:02.236 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2024-03-26T18:47:02.236 INFO:teuthology.orchestra.run.smithi097.stdout:Repeating the final host check... 2024-03-26T18:47:02.293 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T18:47:02.293 INFO:teuthology.orchestra.run.smithi097.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-26T18:47:02.293 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl is present 2024-03-26T18:47:02.293 INFO:teuthology.orchestra.run.smithi097.stdout:lvcreate is present 2024-03-26T18:47:02.311 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T18:47:02.311 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T18:47:02.331 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T18:47:02.331 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-26T18:47:02.353 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2024-03-26T18:47:02.373 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2024-03-26T18:47:02.373 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2024-03-26T18:47:02.374 INFO:teuthology.orchestra.run.smithi097.stdout:Host looks OK 2024-03-26T18:47:02.374 INFO:teuthology.orchestra.run.smithi097.stdout:Cluster fsid: 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:02.374 INFO:teuthology.orchestra.run.smithi097.stdout:Acquiring lock 140406395106256 on /run/cephadm/2172d50a-eba1-11ee-95d0-87774f69a715.lock 2024-03-26T18:47:02.375 INFO:teuthology.orchestra.run.smithi097.stdout:Lock 140406395106256 acquired on /run/cephadm/2172d50a-eba1-11ee-95d0-87774f69a715.lock 2024-03-26T18:47:02.376 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 3300 ... 2024-03-26T18:47:02.376 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 6789 ... 2024-03-26T18:47:02.377 INFO:teuthology.orchestra.run.smithi097.stdout:Base mon IP(s) is [172.21.15.97:3300, 172.21.15.97:6789], mon addrv is [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2024-03-26T18:47:02.392 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.97 metric 100 2024-03-26T18:47:02.393 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.97 metric 100 2024-03-26T18:47:02.406 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-26T18:47:02.406 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-26T18:47:02.406 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1752sec hoplimit 64 pref medium 2024-03-26T18:47:02.417 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-26T18:47:02.417 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-26T18:47:02.418 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T18:47:02.418 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-26T18:47:02.418 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-03-26T18:47:02.418 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T18:47:02.420 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2024-03-26T18:47:02.420 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2024-03-26T18:47:02.421 INFO:teuthology.orchestra.run.smithi097.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-26T18:47:02.423 INFO:teuthology.orchestra.run.smithi097.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-26T18:47:02.426 INFO:teuthology.orchestra.run.smithi097.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T18:47:03.051 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 2024-03-26T18:47:03.051 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T18:47:03.051 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Getting image source signatures 2024-03-26T18:47:03.051 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-26T18:47:03.052 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:0388617eb36698e8cd0b89da9480e32fffb3f5cb74210feb5d82e6789f473b79 2024-03-26T18:47:03.052 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying config sha256:497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 2024-03-26T18:47:03.052 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-26T18:47:13.394 INFO:teuthology.orchestra.run.smithi097.stdout:ceph: stdout ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev) 2024-03-26T18:47:13.394 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph version: ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev) 2024-03-26T18:47:13.394 INFO:teuthology.orchestra.run.smithi097.stdout:Extracting ceph user uid/gid from container image... 2024-03-26T18:47:14.551 INFO:teuthology.orchestra.run.smithi097.stdout:stat: stdout 167 167 2024-03-26T18:47:14.551 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial keys... 2024-03-26T18:47:15.591 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQAyGANmia2jOhAAnZOJoCQAhOQVs7oPWTCz3w== 2024-03-26T18:47:16.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQA0GANmAgTCAhAAfgx8bM2AcNj+6v9412u8Jg== 2024-03-26T18:47:17.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQA1GANmoigiDBAALHlCfUJeMuEO4jsDHpIkIA== 2024-03-26T18:47:17.864 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial monmap... 2024-03-26T18:47:19.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T18:47:19.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-26T18:47:19.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:19.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T18:47:19.788 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool for a [v2:172.21.15.97:3300,v1:172.21.15.97:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T18:47:19.788 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-03-26T18:47:19.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: set fsid to 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:19.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T18:47:19.788 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:47:19.788 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mon... 2024-03-26T18:47:20.892 INFO:teuthology.orchestra.run.smithi097.stdout:create mon.a on 2024-03-26T18:47:21.218 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-26T18:47:21.455 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-26T18:47:21.756 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2172d50a-eba1-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-2172d50a-eba1-11ee-95d0-87774f69a715.target. 2024-03-26T18:47:21.757 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2172d50a-eba1-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-2172d50a-eba1-11ee-95d0-87774f69a715.target. 2024-03-26T18:47:22.136 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.a 2024-03-26T18:47:22.136 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-26T18:47:22.414 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2172d50a-eba1-11ee-95d0-87774f69a715.target.wants/ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-2172d50a-eba1-11ee-95d0-87774f69a715@.service. 2024-03-26T18:47:23.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:22 smithi097 podman[31568]: 2024-03-26 18:47:22.811029154 +0000 UTC m=+0.131481439 container create 3b366d11d8807421ff4af66493563063401112fd964e1f70e2d05d49089d4550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:47:23.234 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T18:47:23.234 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-26T18:47:23.249 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T18:47:23.249 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-26T18:47:23.249 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-26T18:47:23.249 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T18:47:23.250 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon to start... 2024-03-26T18:47:23.250 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon... 2024-03-26T18:47:23.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:23 smithi097 podman[31568]: 2024-03-26 18:47:23.122620304 +0000 UTC m=+0.443072596 container init 3b366d11d8807421ff4af66493563063401112fd964e1f70e2d05d49089d4550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:47:23.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:23 smithi097 podman[31568]: 2024-03-26 18:47:23.132641884 +0000 UTC m=+0.453094176 container start 3b366d11d8807421ff4af66493563063401112fd964e1f70e2d05d49089d4550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:47:23.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:23 smithi097 bash[31568]: 3b366d11d8807421ff4af66493563063401112fd964e1f70e2d05d49089d4550 2024-03-26T18:47:23.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:23 smithi097 ceph-mon[31586]: mkfs 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:23.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:23 smithi097 ceph-mon[31586]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T18:47:23.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:23 smithi097 systemd[1]: Started Ceph mon.a for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:47:24.528 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T18:47:24.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: monmap epoch 1 2024-03-26T18:47:24.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:24.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: last_changed 2024-03-26T18:47:19.215409+0000 2024-03-26T18:47:24.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: created 2024-03-26T18:47:19.215409+0000 2024-03-26T18:47:24.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: min_mon_release 19 (squid) 2024-03-26T18:47:24.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: election_strategy: 1 2024-03-26T18:47:24.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-26T18:47:24.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: fsmap 2024-03-26T18:47:24.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T18:47:24.530 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: mgrmap e1: no daemons active 2024-03-26T18:47:24.530 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:24 smithi097 ceph-mon[31586]: from='client.? 172.21.15.97:0/2801162646' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-26T18:47:24.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout cluster: 2024-03-26T18:47:24.539 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout id: 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:24.539 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-26T18:47:24.540 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:24.540 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout services: 2024-03-26T18:47:24.540 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.704983s) 2024-03-26T18:47:24.540 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-26T18:47:24.541 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-26T18:47:24.541 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:24.541 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout data: 2024-03-26T18:47:24.541 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-26T18:47:24.542 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-26T18:47:24.542 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-26T18:47:24.542 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pgs: 2024-03-26T18:47:24.542 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:24.543 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.821+0000 7f9777c00700 1 Processor -- start 2024-03-26T18:47:24.543 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.821+0000 7f9777c00700 1 -- start start 2024-03-26T18:47:24.543 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.821+0000 7f9777c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977010a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:24.543 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.821+0000 7f9777c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f977010aad0 con 0x7f977010a0e0 2024-03-26T18:47:24.544 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.822+0000 7f9775200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977010a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:24.544 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.823+0000 7f9775200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977010a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57070/0 (socket says 172.21.15.97:57070) 2024-03-26T18:47:24.544 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.823+0000 7f9775200700 1 -- 172.21.15.97:0/1573484552 learned_addr learned my addr 172.21.15.97:0/1573484552 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:24.544 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.823+0000 7f9775200700 1 -- 172.21.15.97:0/1573484552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977010b1f0 con 0x7f977010a0e0 2024-03-26T18:47:24.544 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.824+0000 7f9775200700 1 --2- 172.21.15.97:0/1573484552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977010a500 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f97640098a0 tx=0x7f9764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2ea88b25be96fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:24.544 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.824+0000 7f976f400700 1 -- 172.21.15.97:0/1573484552 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9764004160 con 0x7f977010a0e0 2024-03-26T18:47:24.545 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.824+0000 7f976f400700 1 -- 172.21.15.97:0/1573484552 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9764004300 con 0x7f977010a0e0 2024-03-26T18:47:24.545 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.824+0000 7f976f400700 1 -- 172.21.15.97:0/1573484552 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97640036a0 con 0x7f977010a0e0 2024-03-26T18:47:24.545 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.825+0000 7f9777c00700 1 -- 172.21.15.97:0/1573484552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 msgr2=0x7f977010a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:24.546 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.825+0000 7f9777c00700 1 --2- 172.21.15.97:0/1573484552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977010a500 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f97640098a0 tx=0x7f9764009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:24.546 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.825+0000 7f9777c00700 1 -- 172.21.15.97:0/1573484552 shutdown_connections 2024-03-26T18:47:24.546 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.825+0000 7f9777c00700 1 --2- 172.21.15.97:0/1573484552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977010a500 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:24.547 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.825+0000 7f9777c00700 1 -- 172.21.15.97:0/1573484552 >> 172.21.15.97:0/1573484552 conn(0x7f977007d6a0 msgr2=0x7f977007dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:24.547 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.825+0000 7f9777c00700 1 -- 172.21.15.97:0/1573484552 shutdown_connections 2024-03-26T18:47:24.547 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.825+0000 7f9777c00700 1 -- 172.21.15.97:0/1573484552 wait complete. 2024-03-26T18:47:24.547 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.826+0000 7f9777c00700 1 Processor -- start 2024-03-26T18:47:24.547 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.826+0000 7f9777c00700 1 -- start start 2024-03-26T18:47:24.547 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.827+0000 7f9777c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:24.547 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.827+0000 7f9777c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f977010b720 con 0x7f977010a0e0 2024-03-26T18:47:24.548 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.827+0000 7f9775200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:24.548 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.827+0000 7f9775200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57072/0 (socket says 172.21.15.97:57072) 2024-03-26T18:47:24.548 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.827+0000 7f9775200700 1 -- 172.21.15.97:0/2801162646 learned_addr learned my addr 172.21.15.97:0/2801162646 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:24.548 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.828+0000 7f9775200700 1 -- 172.21.15.97:0/2801162646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977019dc00 con 0x7f977010a0e0 2024-03-26T18:47:24.548 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.828+0000 7f9775200700 1 --2- 172.21.15.97:0/2801162646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977019d6c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9764000c00 tx=0x7f9764004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:24.549 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.828+0000 7f976d600700 1 -- 172.21.15.97:0/2801162646 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9764003db0 con 0x7f977010a0e0 2024-03-26T18:47:24.549 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.828+0000 7f976d600700 1 -- 172.21.15.97:0/2801162646 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9764011630 con 0x7f977010a0e0 2024-03-26T18:47:24.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.828+0000 7f9777c00700 1 -- 172.21.15.97:0/2801162646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977019de90 con 0x7f977010a0e0 2024-03-26T18:47:24.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.828+0000 7f976d600700 1 -- 172.21.15.97:0/2801162646 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9764011950 con 0x7f977010a0e0 2024-03-26T18:47:24.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.828+0000 7f9777c00700 1 -- 172.21.15.97:0/2801162646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97701a0b80 con 0x7f977010a0e0 2024-03-26T18:47:24.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.829+0000 7f976d600700 1 -- 172.21.15.97:0/2801162646 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9764011b80 con 0x7f977010a0e0 2024-03-26T18:47:24.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.829+0000 7f976d600700 1 -- 172.21.15.97:0/2801162646 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f976401c450 con 0x7f977010a0e0 2024-03-26T18:47:24.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.829+0000 7f9777c00700 1 -- 172.21.15.97:0/2801162646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97700639e0 con 0x7f977010a0e0 2024-03-26T18:47:24.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.833+0000 7f976d600700 1 -- 172.21.15.97:0/2801162646 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9764032020 con 0x7f977010a0e0 2024-03-26T18:47:24.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.912+0000 7f9777c00700 1 -- 172.21.15.97:0/2801162646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f977002dd10 con 0x7f977010a0e0 2024-03-26T18:47:24.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.912+0000 7f976d600700 1 -- 172.21.15.97:0/2801162646 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f9764004260 con 0x7f977010a0e0 2024-03-26T18:47:24.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.913+0000 7f9777c00700 1 -- 172.21.15.97:0/2801162646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 msgr2=0x7f977019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:24.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.913+0000 7f9777c00700 1 --2- 172.21.15.97:0/2801162646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977019d6c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9764000c00 tx=0x7f9764004750 comp rx=0 tx=0).stop 2024-03-26T18:47:24.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.914+0000 7f9777c00700 1 -- 172.21.15.97:0/2801162646 shutdown_connections 2024-03-26T18:47:24.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.914+0000 7f9777c00700 1 --2- 172.21.15.97:0/2801162646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f977010a0e0 0x7f977019d6c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:24.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.914+0000 7f9777c00700 1 -- 172.21.15.97:0/2801162646 >> 172.21.15.97:0/2801162646 conn(0x7f977007d6a0 msgr2=0x7f97701942b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:24.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.914+0000 7f9777c00700 1 -- 172.21.15.97:0/2801162646 shutdown_connections 2024-03-26T18:47:24.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:23.914+0000 7f9777c00700 1 -- 172.21.15.97:0/2801162646 wait complete. 2024-03-26T18:47:24.552 INFO:teuthology.orchestra.run.smithi097.stdout:mon is available 2024-03-26T18:47:24.552 INFO:teuthology.orchestra.run.smithi097.stdout:Assimilating anything we can from ceph.conf... 2024-03-26T18:47:25.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:25 smithi097 ceph-mon[31586]: from='client.? 172.21.15.97:0/139317048' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T18:47:25.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:25 smithi097 ceph-mon[31586]: from='client.? 172.21.15.97:0/139317048' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-26T18:47:25.847 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:25.847 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T18:47:25.847 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:25.847 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2024-03-26T18:47:25.847 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T18:47:25.847 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T18:47:25.847 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T18:47:25.847 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T18:47:25.847 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.133+0000 7fbd55200700 1 Processor -- start 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.133+0000 7fbd55200700 1 -- start start 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.133+0000 7fbd55200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd50108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.133+0000 7fbd55200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbd501087d0 con 0x7fbd50107de0 2024-03-26T18:47:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.134+0000 7fbd4de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd50108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.134+0000 7fbd4de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd50108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57074/0 (socket says 172.21.15.97:57074) 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.134+0000 7fbd4de00700 1 -- 172.21.15.97:0/838677360 learned_addr learned my addr 172.21.15.97:0/838677360 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.134+0000 7fbd4de00700 1 -- 172.21.15.97:0/838677360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd50108fe0 con 0x7fbd50107de0 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.135+0000 7fbd4de00700 1 --2- 172.21.15.97:0/838677360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd50108200 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbd400098a0 tx=0x7fbd40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4f9c0de9dfbee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.135+0000 7fbd4ca00700 1 -- 172.21.15.97:0/838677360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd40004160 con 0x7fbd50107de0 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.135+0000 7fbd4ca00700 1 -- 172.21.15.97:0/838677360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd40004300 con 0x7fbd50107de0 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.135+0000 7fbd4ca00700 1 -- 172.21.15.97:0/838677360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd400036a0 con 0x7fbd50107de0 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.135+0000 7fbd55200700 1 -- 172.21.15.97:0/838677360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 msgr2=0x7fbd50108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.135+0000 7fbd55200700 1 --2- 172.21.15.97:0/838677360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd50108200 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbd400098a0 tx=0x7fbd40009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.136+0000 7fbd55200700 1 -- 172.21.15.97:0/838677360 shutdown_connections 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.136+0000 7fbd55200700 1 --2- 172.21.15.97:0/838677360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd50108200 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.136+0000 7fbd55200700 1 -- 172.21.15.97:0/838677360 >> 172.21.15.97:0/838677360 conn(0x7fbd501033e0 msgr2=0x7fbd50105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.136+0000 7fbd55200700 1 -- 172.21.15.97:0/838677360 shutdown_connections 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.136+0000 7fbd55200700 1 -- 172.21.15.97:0/838677360 wait complete. 2024-03-26T18:47:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.137+0000 7fbd55200700 1 Processor -- start 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.137+0000 7fbd55200700 1 -- start start 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.137+0000 7fbd55200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd5019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.137+0000 7fbd55200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbd50109510 con 0x7fbd50107de0 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.138+0000 7fbd4de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd5019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.138+0000 7fbd4de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd5019d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57076/0 (socket says 172.21.15.97:57076) 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.138+0000 7fbd4de00700 1 -- 172.21.15.97:0/139317048 learned_addr learned my addr 172.21.15.97:0/139317048 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.138+0000 7fbd4de00700 1 -- 172.21.15.97:0/139317048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5019dc10 con 0x7fbd50107de0 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.138+0000 7fbd4de00700 1 --2- 172.21.15.97:0/139317048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd5019d6d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbd40000c00 tx=0x7fbd40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.138+0000 7fbd46a00700 1 -- 172.21.15.97:0/139317048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd40003db0 con 0x7fbd50107de0 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.139+0000 7fbd46a00700 1 -- 172.21.15.97:0/139317048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd40011630 con 0x7fbd50107de0 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.139+0000 7fbd55200700 1 -- 172.21.15.97:0/139317048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5019dea0 con 0x7fbd50107de0 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.139+0000 7fbd46a00700 1 -- 172.21.15.97:0/139317048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd40011950 con 0x7fbd50107de0 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.139+0000 7fbd55200700 1 -- 172.21.15.97:0/139317048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd501a0b90 con 0x7fbd50107de0 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.139+0000 7fbd46a00700 1 -- 172.21.15.97:0/139317048 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd40011af0 con 0x7fbd50107de0 2024-03-26T18:47:25.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.139+0000 7fbd46a00700 1 -- 172.21.15.97:0/139317048 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd4001c450 con 0x7fbd50107de0 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.140+0000 7fbd55200700 1 -- 172.21.15.97:0/139317048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd500639e0 con 0x7fbd50107de0 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.143+0000 7fbd46a00700 1 -- 172.21.15.97:0/139317048 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbd40032020 con 0x7fbd50107de0 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.219+0000 7fbd55200700 1 -- 172.21.15.97:0/139317048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbd501a0d60 con 0x7fbd50107de0 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.227+0000 7fbd46a00700 1 -- 172.21.15.97:0/139317048 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fbd40004260 con 0x7fbd50107de0 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.228+0000 7fbd55200700 1 -- 172.21.15.97:0/139317048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 msgr2=0x7fbd5019d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.228+0000 7fbd55200700 1 --2- 172.21.15.97:0/139317048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd5019d6d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbd40000c00 tx=0x7fbd40004750 comp rx=0 tx=0).stop 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.228+0000 7fbd55200700 1 -- 172.21.15.97:0/139317048 shutdown_connections 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.228+0000 7fbd55200700 1 --2- 172.21.15.97:0/139317048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50107de0 0x7fbd5019d6d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.228+0000 7fbd55200700 1 -- 172.21.15.97:0/139317048 >> 172.21.15.97:0/139317048 conn(0x7fbd501033e0 msgr2=0x7fbd5007daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.229+0000 7fbd55200700 1 -- 172.21.15.97:0/139317048 shutdown_connections 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:25.229+0000 7fbd55200700 1 -- 172.21.15.97:0/139317048 wait complete. 2024-03-26T18:47:25.851 INFO:teuthology.orchestra.run.smithi097.stdout:Generating new minimal ceph.conf... 2024-03-26T18:47:26.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:26 smithi097 ceph-mon[31586]: from='client.? 172.21.15.97:0/2424804284' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:47:27.218 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.472+0000 7f00b3200700 1 Processor -- start 2024-03-26T18:47:27.218 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.473+0000 7f00b3200700 1 -- start start 2024-03-26T18:47:27.218 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.473+0000 7f00b3200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b41080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:27.218 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.473+0000 7f00b3200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4108670 con 0x7f00b4107c80 2024-03-26T18:47:27.218 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.473+0000 7f00b1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b41080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:27.218 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.473+0000 7f00b1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b41080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57088/0 (socket says 172.21.15.97:57088) 2024-03-26T18:47:27.218 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.474+0000 7f00b1e00700 1 -- 172.21.15.97:0/2589763337 learned_addr learned my addr 172.21.15.97:0/2589763337 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:27.218 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.474+0000 7f00b1e00700 1 -- 172.21.15.97:0/2589763337 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b4108e30 con 0x7f00b4107c80 2024-03-26T18:47:27.218 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.474+0000 7f00b1e00700 1 --2- 172.21.15.97:0/2589763337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b41080a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f00a40098a0 tx=0x7f00a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c2728214ceff2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:27.218 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.475+0000 7f00b0a00700 1 -- 172.21.15.97:0/2589763337 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00a4004160 con 0x7f00b4107c80 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.475+0000 7f00b0a00700 1 -- 172.21.15.97:0/2589763337 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f00a4004300 con 0x7f00b4107c80 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.475+0000 7f00b0a00700 1 -- 172.21.15.97:0/2589763337 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00a4011b50 con 0x7f00b4107c80 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.475+0000 7f00b3200700 1 -- 172.21.15.97:0/2589763337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 msgr2=0x7f00b41080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.475+0000 7f00b3200700 1 --2- 172.21.15.97:0/2589763337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b41080a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f00a40098a0 tx=0x7f00a4009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.475+0000 7f00b3200700 1 -- 172.21.15.97:0/2589763337 shutdown_connections 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.475+0000 7f00b3200700 1 --2- 172.21.15.97:0/2589763337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b41080a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.475+0000 7f00b3200700 1 -- 172.21.15.97:0/2589763337 >> 172.21.15.97:0/2589763337 conn(0x7f00b4103260 msgr2=0x7f00b4105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.476+0000 7f00b3200700 1 -- 172.21.15.97:0/2589763337 shutdown_connections 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.476+0000 7f00b3200700 1 -- 172.21.15.97:0/2589763337 wait complete. 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.477+0000 7f00b3200700 1 Processor -- start 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.477+0000 7f00b3200700 1 -- start start 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.477+0000 7f00b3200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.477+0000 7f00b3200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00b410c1e0 con 0x7f00b4107c80 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.477+0000 7f00b1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.477+0000 7f00b1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57102/0 (socket says 172.21.15.97:57102) 2024-03-26T18:47:27.219 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.477+0000 7f00b1e00700 1 -- 172.21.15.97:0/2424804284 learned_addr learned my addr 172.21.15.97:0/2424804284 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.478+0000 7f00b1e00700 1 -- 172.21.15.97:0/2424804284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b419e350 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.478+0000 7f00b1e00700 1 --2- 172.21.15.97:0/2424804284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b419de10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f00a4011850 tx=0x7f00a4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.478+0000 7f00aaa00700 1 -- 172.21.15.97:0/2424804284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00a4003af0 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.478+0000 7f00aaa00700 1 -- 172.21.15.97:0/2424804284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f00a4020070 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.478+0000 7f00b3200700 1 -- 172.21.15.97:0/2424804284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b419e5e0 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.478+0000 7f00aaa00700 1 -- 172.21.15.97:0/2424804284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00a40195a0 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.478+0000 7f00b3200700 1 -- 172.21.15.97:0/2424804284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b41a12d0 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.479+0000 7f00aaa00700 1 -- 172.21.15.97:0/2424804284 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f00a4019c30 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.479+0000 7f00aaa00700 1 -- 172.21.15.97:0/2424804284 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f00a4011e00 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.480+0000 7f00b3200700 1 -- 172.21.15.97:0/2424804284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b40639e0 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.482+0000 7f00aaa00700 1 -- 172.21.15.97:0/2424804284 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f00a4003c90 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.558+0000 7f00b3200700 1 -- 172.21.15.97:0/2424804284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f00b41a14a0 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.559+0000 7f00aaa00700 1 -- 172.21.15.97:0/2424804284 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f00a401a430 con 0x7f00b4107c80 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.560+0000 7f00b3200700 1 -- 172.21.15.97:0/2424804284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 msgr2=0x7f00b419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.560+0000 7f00b3200700 1 --2- 172.21.15.97:0/2424804284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b419de10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f00a4011850 tx=0x7f00a4011c10 comp rx=0 tx=0).stop 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.560+0000 7f00b3200700 1 -- 172.21.15.97:0/2424804284 shutdown_connections 2024-03-26T18:47:27.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.560+0000 7f00b3200700 1 --2- 172.21.15.97:0/2424804284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b4107c80 0x7f00b419de10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:27.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.560+0000 7f00b3200700 1 -- 172.21.15.97:0/2424804284 >> 172.21.15.97:0/2424804284 conn(0x7f00b4103260 msgr2=0x7f00b4103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:27.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.561+0000 7f00b3200700 1 -- 172.21.15.97:0/2424804284 shutdown_connections 2024-03-26T18:47:27.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:26.561+0000 7f00b3200700 1 -- 172.21.15.97:0/2424804284 wait complete. 2024-03-26T18:47:27.221 INFO:teuthology.orchestra.run.smithi097.stdout:Restarting the monitor... 2024-03-26T18:47:27.601 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:27 smithi097 systemd[1]: Stopping Ceph mon.a for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:47:28.049 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a[31582]: 2024-03-26T18:47:27.700+0000 7ff77be00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:47:28.049 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a[31582]: 2024-03-26T18:47:27.700+0000 7ff77be00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-26T18:47:28.049 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:27 smithi097 podman[31827]: 2024-03-26 18:47:27.782544146 +0000 UTC m=+0.221746190 container died 3b366d11d8807421ff4af66493563063401112fd964e1f70e2d05d49089d4550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T18:47:28.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:28 smithi097 podman[31827]: 2024-03-26 18:47:28.193061322 +0000 UTC m=+0.632263363 container cleanup 3b366d11d8807421ff4af66493563063401112fd964e1f70e2d05d49089d4550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True) 2024-03-26T18:47:28.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:28 smithi097 bash[31827]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a 2024-03-26T18:47:28.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:28 smithi097 podman[31841]: 2024-03-26 18:47:28.37706381 +0000 UTC m=+0.576745607 container remove 3b366d11d8807421ff4af66493563063401112fd964e1f70e2d05d49089d4550 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-26T18:47:29.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:28 smithi097 systemd[1]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-26T18:47:29.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:28 smithi097 systemd[1]: Stopped Ceph mon.a for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:47:29.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:28 smithi097 systemd[1]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.a.service: Consumed 1.837s CPU time. 2024-03-26T18:47:29.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:28 smithi097 systemd[1]: Starting Ceph mon.a for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:47:29.464 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 podman[31935]: 2024-03-26 18:47:29.133083471 +0000 UTC m=+0.031140683 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T18:47:29.464 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 podman[31935]: 2024-03-26 18:47:29.344868396 +0000 UTC m=+0.242925627 container create 67e023dc813fba1d66fcb80bb4e19d8ca4fca3a0445c139afa7d12f574757fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T18:47:29.624 INFO:teuthology.orchestra.run.smithi097.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-26T18:47:29.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 podman[31935]: 2024-03-26 18:47:29.549337274 +0000 UTC m=+0.447394481 container init 67e023dc813fba1d66fcb80bb4e19d8ca4fca3a0445c139afa7d12f574757fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:47:29.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 podman[31935]: 2024-03-26 18:47:29.560373417 +0000 UTC m=+0.458430621 container start 67e023dc813fba1d66fcb80bb4e19d8ca4fca3a0445c139afa7d12f574757fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:47:29.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev), process ceph-mon, pid 2 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: pidfile_write: ignore empty --pid-file 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: load: jerasure load: lrc 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: RocksDB version: 7.9.2 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Git sha 0 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Compile date 2024-03-20 16:36:06 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: DB SUMMARY 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: DB Session ID: 72FLRMON81LVZREOSYCF 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: CURRENT file: CURRENT 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-26T18:47:29.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105194 ; 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.error_if_exists: 0 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.create_if_missing: 0 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.paranoid_checks: 1 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.env: 0x5586d4aa5c60 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.info_log: 0x5586d6f9db20 2024-03-26T18:47:29.780 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T18:47:29.781 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.statistics: (nil) 2024-03-26T18:47:29.781 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.use_fsync: 0 2024-03-26T18:47:29.781 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_log_file_size: 0 2024-03-26T18:47:29.781 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T18:47:29.781 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T18:47:29.781 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T18:47:29.781 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.allow_fallocate: 1 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.use_direct_reads: 0 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.db_log_dir: 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.wal_dir: 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.write_buffer_manager: 0x5586d6f701e0 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.unordered_write: 0 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T18:47:29.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.row_cache: None 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.wal_filter: None 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.two_write_queues: 0 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.wal_compression: 0 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.atomic_flush: 0 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.log_readahead_size: 0 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T18:47:29.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_background_jobs: 2 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_background_compactions: -1 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_subcompactions: 1 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T18:47:29.785 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_open_files: -1 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_background_flushes: -1 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Compression algorithms supported: 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: kZSTD supported: 0 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: kXpressCompression supported: 0 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: kZlibCompression supported: 1 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: kSnappyCompression supported: 1 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: kLZ4Compression supported: 1 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: kBZip2Compression supported: 0 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T18:47:29.786 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.merge_operator: 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_filter: None 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5586d6f9dc40) 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T18:47:29.787 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_type: 0 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_index_type: 0 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_shortening: 1 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: checksum: 4 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: no_block_cache: 0 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache: 0x5586d6fa3090 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_options: 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: capacity : 536870912 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_shard_bits : 4 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: strict_capacity_limit : 0 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_compressed: (nil) 2024-03-26T18:47:29.788 INFO:journalctl@ceph.mon.a.smithi097.stdout: persistent_cache: (nil) 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size: 4096 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size_deviation: 10 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_restart_interval: 16 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_block_restart_interval: 1 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: metadata_block_size: 4096 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: partition_filters: 0 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: use_delta_encoding: 1 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: filter_policy: bloomfilter 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: whole_key_filtering: 1 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: verify_compression: 0 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: format_version: 5 2024-03-26T18:47:29.789 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_index_compression: 1 2024-03-26T18:47:29.790 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_align: 0 2024-03-26T18:47:29.790 INFO:journalctl@ceph.mon.a.smithi097.stdout: max_auto_readahead_size: 262144 2024-03-26T18:47:29.790 INFO:journalctl@ceph.mon.a.smithi097.stdout: prepopulate_block_cache: 0 2024-03-26T18:47:29.790 INFO:journalctl@ceph.mon.a.smithi097.stdout: initial_auto_readahead_size: 8192 2024-03-26T18:47:29.790 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T18:47:29.790 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T18:47:29.790 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T18:47:29.790 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compression: NoCompression 2024-03-26T18:47:29.793 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.num_levels: 7 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T18:47:29.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T18:47:29.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T18:47:29.796 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T18:47:29.796 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T18:47:29.796 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T18:47:29.796 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T18:47:29.796 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T18:47:29.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T18:47:29.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T18:47:29.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T18:47:29.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T18:47:29.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T18:47:29.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T18:47:29.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T18:47:29.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T18:47:29.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T18:47:29.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.table_properties_collectors: 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.inplace_update_support: 0 2024-03-26T18:47:29.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.bloom_locality: 0 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.max_successive_merges: 0 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.ttl: 2592000 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T18:47:29.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.enable_blob_files: false 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.min_blob_size: 0 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d59a1a72-5c66-48f2-a53a-5bfbbae1e94e 2024-03-26T18:47:29.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711478849613257, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711478849615064, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98217, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14195, "raw_average_key_size": 49, "raw_value_size": 90504, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711478849, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d59a1a72-5c66-48f2-a53a-5bfbbae1e94e", "db_session_id": "72FLRMON81LVZREOSYCF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711478849615209, "job": 1, "event": "recovery_finished"} 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5586d7096000 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: DB pointer 0x5586d7080000 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** DB Stats ** 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T18:47:29.801 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: L0 2/0 99.79 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: Sum 2/0 99.79 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 79.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-26T18:47:29.802 INFO:journalctl@ceph.mon.a.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative compaction: 0.00 GB write, 13.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval compaction: 0.00 GB write, 13.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache BinnedLRUCache@0x5586d6fa3090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.2e-05 secs_since: 0 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: starting mon.a rank 0 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: mon.a@-1(???) e1 preinit fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: mon.a@-1(???).mds e1 new map 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: mon.a@-1(???).mds e1 print_map 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout: e1 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout: legacy client fscid: -1 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout: No filesystems configured 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 bash[31935]: 67e023dc813fba1d66fcb80bb4e19d8ca4fca3a0445c139afa7d12f574757fbd 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 systemd[1]: Started Ceph mon.a for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:47:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T18:47:29.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: monmap epoch 1 2024-03-26T18:47:29.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:29.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: last_changed 2024-03-26T18:47:19.215409+0000 2024-03-26T18:47:29.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: created 2024-03-26T18:47:19.215409+0000 2024-03-26T18:47:29.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: min_mon_release 19 (squid) 2024-03-26T18:47:29.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: election_strategy: 1 2024-03-26T18:47:29.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-26T18:47:29.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: fsmap 2024-03-26T18:47:29.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T18:47:29.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:29 smithi097 ceph-mon[31955]: mgrmap e1: no daemons active 2024-03-26T18:47:31.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.283+0000 7fe524200700 1 Processor -- start 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.284+0000 7fe524200700 1 -- start start 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.284+0000 7fe524200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.284+0000 7fe524200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c07c840 con 0x7fe51c07de00 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.284+0000 7fe521800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.285+0000 7fe521800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57104/0 (socket says 172.21.15.97:57104) 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.285+0000 7fe521800700 1 -- 172.21.15.97:0/2547405093 learned_addr learned my addr 172.21.15.97:0/2547405093 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.285+0000 7fe521800700 1 -- 172.21.15.97:0/2547405093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c07c9c0 con 0x7fe51c07de00 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.285+0000 7fe521800700 1 --2- 172.21.15.97:0/2547405093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c07c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe5100098a0 tx=0x7fe510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0eb90d2ef63585 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.286+0000 7fe51be00700 1 -- 172.21.15.97:0/2547405093 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe510004160 con 0x7fe51c07de00 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.286+0000 7fe51be00700 1 -- 172.21.15.97:0/2547405093 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe510004300 con 0x7fe51c07de00 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.286+0000 7fe51be00700 1 -- 172.21.15.97:0/2547405093 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe510011b50 con 0x7fe51c07de00 2024-03-26T18:47:31.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.286+0000 7fe524200700 1 -- 172.21.15.97:0/2547405093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 msgr2=0x7fe51c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.286+0000 7fe524200700 1 --2- 172.21.15.97:0/2547405093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c07c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe5100098a0 tx=0x7fe510009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.287+0000 7fe524200700 1 -- 172.21.15.97:0/2547405093 shutdown_connections 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.287+0000 7fe524200700 1 --2- 172.21.15.97:0/2547405093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c07c300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.287+0000 7fe524200700 1 -- 172.21.15.97:0/2547405093 >> 172.21.15.97:0/2547405093 conn(0x7fe51c1034a0 msgr2=0x7fe51c1058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.287+0000 7fe524200700 1 -- 172.21.15.97:0/2547405093 shutdown_connections 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.287+0000 7fe524200700 1 -- 172.21.15.97:0/2547405093 wait complete. 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.288+0000 7fe524200700 1 Processor -- start 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.288+0000 7fe524200700 1 -- start start 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.288+0000 7fe524200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.288+0000 7fe524200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c10e370 con 0x7fe51c07de00 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.289+0000 7fe521800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.289+0000 7fe521800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c19df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57110/0 (socket says 172.21.15.97:57110) 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.289+0000 7fe521800700 1 -- 172.21.15.97:0/792417818 learned_addr learned my addr 172.21.15.97:0/792417818 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.289+0000 7fe521800700 1 -- 172.21.15.97:0/792417818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c19e470 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.289+0000 7fe521800700 1 --2- 172.21.15.97:0/792417818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c19df30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe510011b60 tx=0x7fe510011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.290+0000 7fe51a000700 1 -- 172.21.15.97:0/792417818 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe510003b20 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.290+0000 7fe51a000700 1 -- 172.21.15.97:0/792417818 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe510003cc0 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.290+0000 7fe524200700 1 -- 172.21.15.97:0/792417818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c19e700 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.290+0000 7fe524200700 1 -- 172.21.15.97:0/792417818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c1a13f0 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.290+0000 7fe51a000700 1 -- 172.21.15.97:0/792417818 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe510024860 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.290+0000 7fe51a000700 1 -- 172.21.15.97:0/792417818 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe510022020 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.290+0000 7fe51a000700 1 -- 172.21.15.97:0/792417818 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe510011e00 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.291+0000 7fe524200700 1 -- 172.21.15.97:0/792417818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe51c0639e0 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.294+0000 7fe51a000700 1 -- 172.21.15.97:0/792417818 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe510043dd0 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.370+0000 7fe524200700 1 -- 172.21.15.97:0/792417818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe51c1a1a80 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.374+0000 7fe51a000700 1 -- 172.21.15.97:0/792417818 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe510032030 con 0x7fe51c07de00 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.376+0000 7fe524200700 1 -- 172.21.15.97:0/792417818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 msgr2=0x7fe51c19df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.376+0000 7fe524200700 1 --2- 172.21.15.97:0/792417818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c19df30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe510011b60 tx=0x7fe510011c40 comp rx=0 tx=0).stop 2024-03-26T18:47:31.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.376+0000 7fe524200700 1 -- 172.21.15.97:0/792417818 shutdown_connections 2024-03-26T18:47:31.026 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.376+0000 7fe524200700 1 --2- 172.21.15.97:0/792417818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c07de00 0x7fe51c19df30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:31.026 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.376+0000 7fe524200700 1 -- 172.21.15.97:0/792417818 >> 172.21.15.97:0/792417818 conn(0x7fe51c1034a0 msgr2=0x7fe51c104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:31.026 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.376+0000 7fe524200700 1 -- 172.21.15.97:0/792417818 shutdown_connections 2024-03-26T18:47:31.026 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:30.376+0000 7fe524200700 1 -- 172.21.15.97:0/792417818 wait complete. 2024-03-26T18:47:31.026 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-26T18:47:31.060 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:47:31.060 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mgr... 2024-03-26T18:47:31.062 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-26T18:47:31.063 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-26T18:47:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.x 2024-03-26T18:47:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-26T18:47:31.691 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2172d50a-eba1-11ee-95d0-87774f69a715.target.wants/ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-2172d50a-eba1-11ee-95d0-87774f69a715@.service. 2024-03-26T18:47:31.724 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:31 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/792417818' entity='client.admin' 2024-03-26T18:47:32.473 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T18:47:32.473 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-26T18:47:32.495 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T18:47:32.495 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-26T18:47:32.496 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-26T18:47:32.496 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T18:47:32.522 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T18:47:32.523 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-26T18:47:32.542 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T18:47:32.542 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-26T18:47:32.542 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-26T18:47:32.543 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-26T18:47:32.543 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr to start... 2024-03-26T18:47:32.543 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr... 2024-03-26T18:47:32.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:32 smithi097 systemd[1]: Started Ceph mgr.x for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:47:33.263 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:33 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:33.104+0000 7fbbb9597200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T18:47:33.263 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:33 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:33.227+0000 7fbbb9597200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T18:47:33.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:33 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3503326612' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T18:47:34.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:34.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-26T18:47:34.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:47:34.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T18:47:34.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T18:47:34.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T18:47:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:34.072 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T18:47:34.072 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T18:47:34.072 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:34.072 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T18:47:34.072 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:34.072 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T18:47:34.072 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T18:47:34.072 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T18:47:34.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T18:47:34.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T18:47:34.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T18:47:34.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:34.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T18:47:34.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T18:47:34.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T18:47:34.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:34.080 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T18:47:34.081 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T18:47:23.210190+0000", 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.163+0000 7fdaeaa00700 1 Processor -- start 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.164+0000 7fdaeaa00700 1 -- start start 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.164+0000 7fdaeaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae41082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.164+0000 7fdaeaa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdae4108800 con 0x7fdae4107ea0 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.165+0000 7fdae3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae41082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.165+0000 7fdae3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae41082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53792/0 (socket says 172.21.15.97:53792) 2024-03-26T18:47:34.082 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.165+0000 7fdae3e00700 1 -- 172.21.15.97:0/2712221577 learned_addr learned my addr 172.21.15.97:0/2712221577 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.165+0000 7fdae3e00700 1 -- 172.21.15.97:0/2712221577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae4108f60 con 0x7fdae4107ea0 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.166+0000 7fdae3e00700 1 --2- 172.21.15.97:0/2712221577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae41082c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdad40098a0 tx=0x7fdad4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf903fca307137c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.166+0000 7fdae2a00700 1 -- 172.21.15.97:0/2712221577 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdad4004160 con 0x7fdae4107ea0 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.166+0000 7fdae2a00700 1 -- 172.21.15.97:0/2712221577 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdad4004300 con 0x7fdae4107ea0 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.166+0000 7fdae2a00700 1 -- 172.21.15.97:0/2712221577 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdad4011b50 con 0x7fdae4107ea0 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.167+0000 7fdaeaa00700 1 -- 172.21.15.97:0/2712221577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 msgr2=0x7fdae41082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.167+0000 7fdaeaa00700 1 --2- 172.21.15.97:0/2712221577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae41082c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdad40098a0 tx=0x7fdad4009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.167+0000 7fdaeaa00700 1 -- 172.21.15.97:0/2712221577 shutdown_connections 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.167+0000 7fdaeaa00700 1 --2- 172.21.15.97:0/2712221577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae41082c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.167+0000 7fdaeaa00700 1 -- 172.21.15.97:0/2712221577 >> 172.21.15.97:0/2712221577 conn(0x7fdae41034a0 msgr2=0x7fdae41058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:34.083 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.167+0000 7fdaeaa00700 1 -- 172.21.15.97:0/2712221577 shutdown_connections 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.167+0000 7fdaeaa00700 1 -- 172.21.15.97:0/2712221577 wait complete. 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.168+0000 7fdaeaa00700 1 Processor -- start 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.168+0000 7fdaeaa00700 1 -- start start 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.169+0000 7fdaeaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.169+0000 7fdaeaa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410c220 con 0x7fdae4107ea0 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.169+0000 7fdae3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.169+0000 7fdae3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae419bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53798/0 (socket says 172.21.15.97:53798) 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.169+0000 7fdae3e00700 1 -- 172.21.15.97:0/3503326612 learned_addr learned my addr 172.21.15.97:0/3503326612 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.169+0000 7fdae3e00700 1 -- 172.21.15.97:0/3503326612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae419c270 con 0x7fdae4107ea0 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.170+0000 7fdae3e00700 1 --2- 172.21.15.97:0/3503326612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae419bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdad4011b60 tx=0x7fdad4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.170+0000 7fdae0c00700 1 -- 172.21.15.97:0/3503326612 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdad4003b20 con 0x7fdae4107ea0 2024-03-26T18:47:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.170+0000 7fdae0c00700 1 -- 172.21.15.97:0/3503326612 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdad4003cc0 con 0x7fdae4107ea0 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.170+0000 7fdae0c00700 1 -- 172.21.15.97:0/3503326612 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdad4024840 con 0x7fdae4107ea0 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.170+0000 7fdaeaa00700 1 -- 172.21.15.97:0/3503326612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae419c500 con 0x7fdae4107ea0 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.170+0000 7fdaeaa00700 1 -- 172.21.15.97:0/3503326612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae419f1f0 con 0x7fdae4107ea0 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.170+0000 7fdae0c00700 1 -- 172.21.15.97:0/3503326612 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdad4022020 con 0x7fdae4107ea0 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.170+0000 7fdae0c00700 1 -- 172.21.15.97:0/3503326612 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdad4011e00 con 0x7fdae4107ea0 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.171+0000 7fdaeaa00700 1 -- 172.21.15.97:0/3503326612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae410c3a0 con 0x7fdae4107ea0 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.174+0000 7fdae0c00700 1 -- 172.21.15.97:0/3503326612 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdad4043dd0 con 0x7fdae4107ea0 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.253+0000 7fdaeaa00700 1 -- 172.21.15.97:0/3503326612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdae419f3c0 con 0x7fdae4107ea0 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.254+0000 7fdae0c00700 1 -- 172.21.15.97:0/3503326612 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdad4032030 con 0x7fdae4107ea0 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.255+0000 7fdaeaa00700 1 -- 172.21.15.97:0/3503326612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 msgr2=0x7fdae419bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.255+0000 7fdaeaa00700 1 --2- 172.21.15.97:0/3503326612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae419bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdad4011b60 tx=0x7fdad4011c40 comp rx=0 tx=0).stop 2024-03-26T18:47:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.256+0000 7fdaeaa00700 1 -- 172.21.15.97:0/3503326612 shutdown_connections 2024-03-26T18:47:34.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.256+0000 7fdaeaa00700 1 --2- 172.21.15.97:0/3503326612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae4107ea0 0x7fdae419bd30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:34.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.256+0000 7fdaeaa00700 1 -- 172.21.15.97:0/3503326612 >> 172.21.15.97:0/3503326612 conn(0x7fdae41034a0 msgr2=0x7fdae4103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:34.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.256+0000 7fdaeaa00700 1 -- 172.21.15.97:0/3503326612 shutdown_connections 2024-03-26T18:47:34.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:33.256+0000 7fdaeaa00700 1 -- 172.21.15.97:0/3503326612 wait complete. 2024-03-26T18:47:34.086 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (1/15)... 2024-03-26T18:47:34.351 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:34.074+0000 7fbbb9597200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T18:47:34.351 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:34.195+0000 7fbbb9597200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T18:47:34.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:34.368+0000 7fbbb9597200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T18:47:34.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:34.490+0000 7fbbb9597200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T18:47:34.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:34.598+0000 7fbbb9597200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T18:47:35.291 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:34.933+0000 7fbbb9597200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T18:47:35.291 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:35.071+0000 7fbbb9597200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T18:47:35.291 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:35.179+0000 7fbbb9597200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T18:47:35.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:35.300+0000 7fbbb9597200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T18:47:35.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:35.514+0000 7fbbb9597200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T18:47:36.991 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:36 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/79887251' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T18:47:36.992 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:36 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:36.829+0000 7fbbb9597200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T18:47:36.992 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:37 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:37.000+0000 7fbbb9597200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T18:47:37.420 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:37.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-26T18:47:37.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:47:37.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T18:47:37.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T18:47:37.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T18:47:37.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:37.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T18:47:37.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T18:47:37.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T18:47:23.210190+0000", 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.643+0000 7fc117e00700 1 Processor -- start 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.643+0000 7fc117e00700 1 -- start start 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.643+0000 7fc117e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc1181080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.643+0000 7fc117e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc118108690 con 0x7fc118107ca0 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.644+0000 7fc116a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc1181080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:37.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.644+0000 7fc116a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc1181080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53812/0 (socket says 172.21.15.97:53812) 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.644+0000 7fc116a00700 1 -- 172.21.15.97:0/2345552386 learned_addr learned my addr 172.21.15.97:0/2345552386 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.644+0000 7fc116a00700 1 -- 172.21.15.97:0/2345552386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc118108e00 con 0x7fc118107ca0 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.645+0000 7fc116a00700 1 --2- 172.21.15.97:0/2345552386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc1181080c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1080098a0 tx=0x7fc108009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2700f462c6d783 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.645+0000 7fc115600700 1 -- 172.21.15.97:0/2345552386 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc108004160 con 0x7fc118107ca0 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.645+0000 7fc115600700 1 -- 172.21.15.97:0/2345552386 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc108004300 con 0x7fc118107ca0 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.645+0000 7fc115600700 1 -- 172.21.15.97:0/2345552386 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc108011b50 con 0x7fc118107ca0 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.646+0000 7fc117e00700 1 -- 172.21.15.97:0/2345552386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 msgr2=0x7fc1181080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.646+0000 7fc117e00700 1 --2- 172.21.15.97:0/2345552386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc1181080c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1080098a0 tx=0x7fc108009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.646+0000 7fc117e00700 1 -- 172.21.15.97:0/2345552386 shutdown_connections 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.646+0000 7fc117e00700 1 --2- 172.21.15.97:0/2345552386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc1181080c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.646+0000 7fc117e00700 1 -- 172.21.15.97:0/2345552386 >> 172.21.15.97:0/2345552386 conn(0x7fc118103240 msgr2=0x7fc118105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.646+0000 7fc117e00700 1 -- 172.21.15.97:0/2345552386 shutdown_connections 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.646+0000 7fc117e00700 1 -- 172.21.15.97:0/2345552386 wait complete. 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.647+0000 7fc117e00700 1 Processor -- start 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.647+0000 7fc117e00700 1 -- start start 2024-03-26T18:47:37.436 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.647+0000 7fc117e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc11819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.647+0000 7fc117e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc11810c1b0 con 0x7fc118107ca0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.648+0000 7fc116a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc11819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.648+0000 7fc116a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc11819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53818/0 (socket says 172.21.15.97:53818) 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.648+0000 7fc116a00700 1 -- 172.21.15.97:0/79887251 learned_addr learned my addr 172.21.15.97:0/79887251 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.648+0000 7fc116a00700 1 -- 172.21.15.97:0/79887251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11819e360 con 0x7fc118107ca0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.648+0000 7fc116a00700 1 --2- 172.21.15.97:0/79887251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc11819de20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc108011670 tx=0x7fc108011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.649+0000 7fc10f400700 1 -- 172.21.15.97:0/79887251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1080039c0 con 0x7fc118107ca0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.649+0000 7fc10f400700 1 -- 172.21.15.97:0/79887251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc108011e50 con 0x7fc118107ca0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.649+0000 7fc10f400700 1 -- 172.21.15.97:0/79887251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc10801a5c0 con 0x7fc118107ca0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.649+0000 7fc117e00700 1 -- 172.21.15.97:0/79887251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11819e5f0 con 0x7fc118107ca0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.649+0000 7fc117e00700 1 -- 172.21.15.97:0/79887251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1181a12e0 con 0x7fc118107ca0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.649+0000 7fc10f400700 1 -- 172.21.15.97:0/79887251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc10801ac00 con 0x7fc118107ca0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.649+0000 7fc10f400700 1 -- 172.21.15.97:0/79887251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc108019700 con 0x7fc118107ca0 2024-03-26T18:47:37.437 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.650+0000 7fc117e00700 1 -- 172.21.15.97:0/79887251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1180639e0 con 0x7fc118107ca0 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.652+0000 7fc10f400700 1 -- 172.21.15.97:0/79887251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc108003b60 con 0x7fc118107ca0 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.732+0000 7fc117e00700 1 -- 172.21.15.97:0/79887251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc1181a14b0 con 0x7fc118107ca0 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.733+0000 7fc10f400700 1 -- 172.21.15.97:0/79887251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc108019900 con 0x7fc118107ca0 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.734+0000 7fc117e00700 1 -- 172.21.15.97:0/79887251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 msgr2=0x7fc11819de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.734+0000 7fc117e00700 1 --2- 172.21.15.97:0/79887251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc11819de20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc108011670 tx=0x7fc108011b60 comp rx=0 tx=0).stop 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.735+0000 7fc117e00700 1 -- 172.21.15.97:0/79887251 shutdown_connections 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.735+0000 7fc117e00700 1 --2- 172.21.15.97:0/79887251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc118107ca0 0x7fc11819de20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.735+0000 7fc117e00700 1 -- 172.21.15.97:0/79887251 >> 172.21.15.97:0/79887251 conn(0x7fc118103240 msgr2=0x7fc118103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.735+0000 7fc117e00700 1 -- 172.21.15.97:0/79887251 shutdown_connections 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:36.735+0000 7fc117e00700 1 -- 172.21.15.97:0/79887251 wait complete. 2024-03-26T18:47:37.438 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (2/15)... 2024-03-26T18:47:37.781 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:37 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:37.417+0000 7fbbb9597200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T18:47:37.781 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:37 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:37.537+0000 7fbbb9597200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T18:47:37.781 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:37 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:37.788+0000 7fbbb9597200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:47:38.101 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:38 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:38.029+0000 7fbbb9597200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T18:47:38.705 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:38 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:38.401+0000 7fbbb9597200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:47:38.705 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:38 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:38.712+0000 7fbbb9597200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T18:47:39.351 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:39 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:39.014+0000 7fbbb9597200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T18:47:40.152 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:40 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:40.052+0000 7fbbb9597200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T18:47:40.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:40 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/252745524' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T18:47:40.843 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:40 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:40.757+0000 7fbbb9597200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-26T18:47:40.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T18:47:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T18:47:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T18:47:40.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T18:47:23.210190+0000", 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.051+0000 7fbc8ac00700 1 Processor -- start 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.052+0000 7fbc8ac00700 1 -- start start 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.052+0000 7fbc8ac00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc841082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:40.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.052+0000 7fbc8ac00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84108800 con 0x7fbc84107ea0 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.052+0000 7fbc83e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc841082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.052+0000 7fbc83e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc841082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53828/0 (socket says 172.21.15.97:53828) 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.053+0000 7fbc83e00700 1 -- 172.21.15.97:0/1863027282 learned_addr learned my addr 172.21.15.97:0/1863027282 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.053+0000 7fbc83e00700 1 -- 172.21.15.97:0/1863027282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc84108f60 con 0x7fbc84107ea0 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.053+0000 7fbc83e00700 1 --2- 172.21.15.97:0/1863027282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc841082c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc740098a0 tx=0x7fbc74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34ce3a14cb8885b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.054+0000 7fbc82a00700 1 -- 172.21.15.97:0/1863027282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74004160 con 0x7fbc84107ea0 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.054+0000 7fbc82a00700 1 -- 172.21.15.97:0/1863027282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc74004300 con 0x7fbc84107ea0 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.054+0000 7fbc82a00700 1 -- 172.21.15.97:0/1863027282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74011b50 con 0x7fbc84107ea0 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.054+0000 7fbc8ac00700 1 -- 172.21.15.97:0/1863027282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 msgr2=0x7fbc841082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.054+0000 7fbc8ac00700 1 --2- 172.21.15.97:0/1863027282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc841082c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc740098a0 tx=0x7fbc74009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.054+0000 7fbc8ac00700 1 -- 172.21.15.97:0/1863027282 shutdown_connections 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.054+0000 7fbc8ac00700 1 --2- 172.21.15.97:0/1863027282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc841082c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.054+0000 7fbc8ac00700 1 -- 172.21.15.97:0/1863027282 >> 172.21.15.97:0/1863027282 conn(0x7fbc841034a0 msgr2=0x7fbc841058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.054+0000 7fbc8ac00700 1 -- 172.21.15.97:0/1863027282 shutdown_connections 2024-03-26T18:47:40.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.054+0000 7fbc8ac00700 1 -- 172.21.15.97:0/1863027282 wait complete. 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.056+0000 7fbc8ac00700 1 Processor -- start 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.056+0000 7fbc8ac00700 1 -- start start 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.056+0000 7fbc8ac00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc8407db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.056+0000 7fbc8ac00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8410c220 con 0x7fbc84107ea0 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.057+0000 7fbc83e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc8407db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.057+0000 7fbc83e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc8407db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53830/0 (socket says 172.21.15.97:53830) 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.057+0000 7fbc83e00700 1 -- 172.21.15.97:0/252745524 learned_addr learned my addr 172.21.15.97:0/252745524 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.057+0000 7fbc83e00700 1 -- 172.21.15.97:0/252745524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8407e090 con 0x7fbc84107ea0 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.057+0000 7fbc83e00700 1 --2- 172.21.15.97:0/252745524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc8407db50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc74011670 tx=0x7fbc74011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.058+0000 7fbc80c00700 1 -- 172.21.15.97:0/252745524 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc740039c0 con 0x7fbc84107ea0 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.058+0000 7fbc80c00700 1 -- 172.21.15.97:0/252745524 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc74011e50 con 0x7fbc84107ea0 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.058+0000 7fbc8ac00700 1 -- 172.21.15.97:0/252745524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8407c280 con 0x7fbc84107ea0 2024-03-26T18:47:40.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.058+0000 7fbc8ac00700 1 -- 172.21.15.97:0/252745524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8407c7c0 con 0x7fbc84107ea0 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.058+0000 7fbc80c00700 1 -- 172.21.15.97:0/252745524 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc7401a5c0 con 0x7fbc84107ea0 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.058+0000 7fbc80c00700 1 -- 172.21.15.97:0/252745524 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc7401ac20 con 0x7fbc84107ea0 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.058+0000 7fbc80c00700 1 -- 172.21.15.97:0/252745524 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc74019700 con 0x7fbc84107ea0 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.059+0000 7fbc8ac00700 1 -- 172.21.15.97:0/252745524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8410c3a0 con 0x7fbc84107ea0 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.061+0000 7fbc80c00700 1 -- 172.21.15.97:0/252745524 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbc74003b60 con 0x7fbc84107ea0 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.141+0000 7fbc8ac00700 1 -- 172.21.15.97:0/252745524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbc8407cb00 con 0x7fbc84107ea0 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.142+0000 7fbc80c00700 1 -- 172.21.15.97:0/252745524 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fbc740199a0 con 0x7fbc84107ea0 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.143+0000 7fbc8ac00700 1 -- 172.21.15.97:0/252745524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 msgr2=0x7fbc8407db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.143+0000 7fbc8ac00700 1 --2- 172.21.15.97:0/252745524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc8407db50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc74011670 tx=0x7fbc74011b60 comp rx=0 tx=0).stop 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.143+0000 7fbc8ac00700 1 -- 172.21.15.97:0/252745524 shutdown_connections 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.143+0000 7fbc8ac00700 1 --2- 172.21.15.97:0/252745524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc84107ea0 0x7fbc8407db50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.144+0000 7fbc8ac00700 1 -- 172.21.15.97:0/252745524 >> 172.21.15.97:0/252745524 conn(0x7fbc841034a0 msgr2=0x7fbc84103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.144+0000 7fbc8ac00700 1 -- 172.21.15.97:0/252745524 shutdown_connections 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:40.144+0000 7fbc8ac00700 1 -- 172.21.15.97:0/252745524 wait complete. 2024-03-26T18:47:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (3/15)... 2024-03-26T18:47:41.101 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:40 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:40.887+0000 7fbbb9597200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T18:47:41.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:41 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:41.163+0000 7fbbb9597200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T18:47:41.601 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:41 smithi097 ceph-mon[31955]: Activating manager daemon x 2024-03-26T18:47:41.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:41 smithi097 ceph-mon[31955]: mgrmap e2: x(active, starting, since 0.00968375s) 2024-03-26T18:47:41.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:41 smithi097 ceph-mon[31955]: from='mgr.14100 172.21.15.97:0/2439783425' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T18:47:41.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:41 smithi097 ceph-mon[31955]: from='mgr.14100 172.21.15.97:0/2439783425' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T18:47:41.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:41 smithi097 ceph-mon[31955]: from='mgr.14100 172.21.15.97:0/2439783425' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T18:47:41.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:41 smithi097 ceph-mon[31955]: from='mgr.14100 172.21.15.97:0/2439783425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:47:41.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:41 smithi097 ceph-mon[31955]: from='mgr.14100 172.21.15.97:0/2439783425' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T18:47:41.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:41 smithi097 ceph-mon[31955]: Manager daemon x is now available 2024-03-26T18:47:42.601 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:42 smithi097 ceph-mon[31955]: from='mgr.14100 172.21.15.97:0/2439783425' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T18:47:42.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:42 smithi097 ceph-mon[31955]: from='mgr.14100 172.21.15.97:0/2439783425' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T18:47:42.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:42 smithi097 ceph-mon[31955]: from='mgr.14100 172.21.15.97:0/2439783425' entity='mgr.x' 2024-03-26T18:47:42.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:42 smithi097 ceph-mon[31955]: from='mgr.14100 172.21.15.97:0/2439783425' entity='mgr.x' 2024-03-26T18:47:42.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:42 smithi097 ceph-mon[31955]: from='mgr.14100 172.21.15.97:0/2439783425' entity='mgr.x' 2024-03-26T18:47:42.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:42 smithi097 ceph-mon[31955]: mgrmap e3: x(active, since 1.01875s) 2024-03-26T18:47:44.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:44 smithi097 ceph-mon[31955]: mgrmap e4: x(active, since 2s) 2024-03-26T18:47:44.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:44 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1235356574' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T18:47:44.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:44.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-26T18:47:44.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:47:44.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T18:47:44.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T18:47:44.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T18:47:44.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T18:47:44.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:44.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T18:47:44.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T18:47:44.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-26T18:47:44.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:44.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T18:47:44.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T18:47:44.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T18:47:44.417 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T18:47:44.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T18:47:44.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T18:47:44.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T18:47:44.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:44.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T18:47:44.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T18:47:44.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T18:47:44.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T18:47:44.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T18:47:23.210190+0000", 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T18:47:44.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.429+0000 7fd442000700 1 Processor -- start 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.429+0000 7fd442000700 1 -- start start 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.430+0000 7fd442000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c107170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.430+0000 7fd442000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c107740 con 0x7fd43c106d50 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.430+0000 7fd43b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.430+0000 7fd43b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c107170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46924/0 (socket says 172.21.15.97:46924) 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.430+0000 7fd43b400700 1 -- 172.21.15.97:0/3765612645 learned_addr learned my addr 172.21.15.97:0/3765612645 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.431+0000 7fd43b400700 1 -- 172.21.15.97:0/3765612645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c107f50 con 0x7fd43c106d50 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.431+0000 7fd43b400700 1 --2- 172.21.15.97:0/3765612645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c107170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00bb40 tx=0x7fd42c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6f684510933694 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.431+0000 7fd43a000700 1 -- 172.21.15.97:0/3765612645 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42c00dbe0 con 0x7fd43c106d50 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.431+0000 7fd43a000700 1 -- 172.21.15.97:0/3765612645 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd42c0044f0 con 0x7fd43c106d50 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.431+0000 7fd43a000700 1 -- 172.21.15.97:0/3765612645 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42c010470 con 0x7fd43c106d50 2024-03-26T18:47:44.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.432+0000 7fd442000700 1 -- 172.21.15.97:0/3765612645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 msgr2=0x7fd43c107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.432+0000 7fd442000700 1 --2- 172.21.15.97:0/3765612645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c107170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00bb40 tx=0x7fd42c00be50 comp rx=0 tx=0).stop 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.432+0000 7fd442000700 1 -- 172.21.15.97:0/3765612645 shutdown_connections 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.432+0000 7fd442000700 1 --2- 172.21.15.97:0/3765612645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c107170 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.432+0000 7fd442000700 1 -- 172.21.15.97:0/3765612645 >> 172.21.15.97:0/3765612645 conn(0x7fd43c102310 msgr2=0x7fd43c104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.432+0000 7fd442000700 1 -- 172.21.15.97:0/3765612645 shutdown_connections 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.432+0000 7fd442000700 1 -- 172.21.15.97:0/3765612645 wait complete. 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.433+0000 7fd442000700 1 Processor -- start 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.433+0000 7fd442000700 1 -- start start 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.434+0000 7fd442000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.434+0000 7fd442000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c07afb0 con 0x7fd43c106d50 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.434+0000 7fd43b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.434+0000 7fd43b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46936/0 (socket says 172.21.15.97:46936) 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.434+0000 7fd43b400700 1 -- 172.21.15.97:0/1235356574 learned_addr learned my addr 172.21.15.97:0/1235356574 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:44.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.434+0000 7fd43b400700 1 -- 172.21.15.97:0/1235356574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c19e510 con 0x7fd43c106d50 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.435+0000 7fd43b400700 1 --2- 172.21.15.97:0/1235356574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c19dfd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00b340 tx=0x7fd42c0040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.435+0000 7fd433e00700 1 -- 172.21.15.97:0/1235356574 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42c010720 con 0x7fd43c106d50 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.435+0000 7fd433e00700 1 -- 172.21.15.97:0/1235356574 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd42c01d070 con 0x7fd43c106d50 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.435+0000 7fd442000700 1 -- 172.21.15.97:0/1235356574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c19e7a0 con 0x7fd43c106d50 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.435+0000 7fd433e00700 1 -- 172.21.15.97:0/1235356574 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42c021c20 con 0x7fd43c106d50 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.435+0000 7fd442000700 1 -- 172.21.15.97:0/1235356574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c1a1490 con 0x7fd43c106d50 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.436+0000 7fd433e00700 1 -- 172.21.15.97:0/1235356574 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46283+0+0 (secure 0 0 0) 0x7fd42c01f030 con 0x7fd43c106d50 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.436+0000 7fd442000700 1 -- 172.21.15.97:0/1235356574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c0639e0 con 0x7fd43c106d50 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.436+0000 7fd433e00700 1 --2- 172.21.15.97:0/1235356574 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fd4200390e0 0x7fd42003b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.437+0000 7fd433e00700 1 -- 172.21.15.97:0/1235356574 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd42c04d130 con 0x7fd43c106d50 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.437+0000 7fd43aa00700 1 --2- 172.21.15.97:0/1235356574 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fd4200390e0 0x7fd42003b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.437+0000 7fd43aa00700 1 --2- 172.21.15.97:0/1235356574 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fd4200390e0 0x7fd42003b5a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd428006fd0 tx=0x7fd428006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.443+0000 7fd433e00700 1 -- 172.21.15.97:0/1235356574 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd42c015070 con 0x7fd43c106d50 2024-03-26T18:47:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.749+0000 7fd442000700 1 -- 172.21.15.97:0/1235356574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd43c02dfd0 con 0x7fd43c106d50 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.750+0000 7fd433e00700 1 -- 172.21.15.97:0/1235356574 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd42c026210 con 0x7fd43c106d50 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.753+0000 7fd442000700 1 -- 172.21.15.97:0/1235356574 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fd4200390e0 msgr2=0x7fd42003b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.753+0000 7fd442000700 1 --2- 172.21.15.97:0/1235356574 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fd4200390e0 0x7fd42003b5a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd428006fd0 tx=0x7fd428006e40 comp rx=0 tx=0).stop 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.753+0000 7fd442000700 1 -- 172.21.15.97:0/1235356574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 msgr2=0x7fd43c19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.753+0000 7fd442000700 1 --2- 172.21.15.97:0/1235356574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c19dfd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00b340 tx=0x7fd42c0040b0 comp rx=0 tx=0).stop 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.753+0000 7fd442000700 1 -- 172.21.15.97:0/1235356574 shutdown_connections 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.753+0000 7fd442000700 1 --2- 172.21.15.97:0/1235356574 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fd4200390e0 0x7fd42003b5a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.753+0000 7fd442000700 1 --2- 172.21.15.97:0/1235356574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c106d50 0x7fd43c19dfd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.753+0000 7fd442000700 1 -- 172.21.15.97:0/1235356574 >> 172.21.15.97:0/1235356574 conn(0x7fd43c102310 msgr2=0x7fd43c194b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.753+0000 7fd442000700 1 -- 172.21.15.97:0/1235356574 shutdown_connections 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:43.754+0000 7fd442000700 1 -- 172.21.15.97:0/1235356574 wait complete. 2024-03-26T18:47:44.433 INFO:teuthology.orchestra.run.smithi097.stdout:mgr is available 2024-03-26T18:47:45.698 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:45 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3505960449' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T18:47:45.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:45.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T18:47:45.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:47:45.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T18:47:45.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T18:47:45.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T18:47:45.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T18:47:45.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:45.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.139+0000 7f149ce00700 1 Processor -- start 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.139+0000 7f149ce00700 1 -- start start 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.139+0000 7f149ce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149810a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.140+0000 7f149ce00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f149810aad0 con 0x7f149810a0e0 2024-03-26T18:47:45.998 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.140+0000 7f1495e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149810a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.140+0000 7f1495e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149810a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46942/0 (socket says 172.21.15.97:46942) 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.140+0000 7f1495e00700 1 -- 172.21.15.97:0/2810989032 learned_addr learned my addr 172.21.15.97:0/2810989032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.140+0000 7f1495e00700 1 -- 172.21.15.97:0/2810989032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149810b220 con 0x7f149810a0e0 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.141+0000 7f1495e00700 1 --2- 172.21.15.97:0/2810989032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149810a500 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f14880098a0 tx=0x7f1488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9faec29f161e4ff2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.141+0000 7f1495400700 1 -- 172.21.15.97:0/2810989032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1488004160 con 0x7f149810a0e0 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.141+0000 7f1495400700 1 -- 172.21.15.97:0/2810989032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1488004300 con 0x7f149810a0e0 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.141+0000 7f1495400700 1 -- 172.21.15.97:0/2810989032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1488011b50 con 0x7f149810a0e0 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.141+0000 7f149ce00700 1 -- 172.21.15.97:0/2810989032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 msgr2=0x7f149810a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.141+0000 7f149ce00700 1 --2- 172.21.15.97:0/2810989032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149810a500 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f14880098a0 tx=0x7f1488009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.142+0000 7f149ce00700 1 -- 172.21.15.97:0/2810989032 shutdown_connections 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.142+0000 7f149ce00700 1 --2- 172.21.15.97:0/2810989032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149810a500 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.142+0000 7f149ce00700 1 -- 172.21.15.97:0/2810989032 >> 172.21.15.97:0/2810989032 conn(0x7f149807d510 msgr2=0x7f149807d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.142+0000 7f149ce00700 1 -- 172.21.15.97:0/2810989032 shutdown_connections 2024-03-26T18:47:45.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.142+0000 7f149ce00700 1 -- 172.21.15.97:0/2810989032 wait complete. 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.143+0000 7f149ce00700 1 Processor -- start 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.143+0000 7f149ce00700 1 -- start start 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.143+0000 7f149ce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149819df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.143+0000 7f149ce00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f149810e5d0 con 0x7f149810a0e0 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.143+0000 7f1495e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149819df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.143+0000 7f1495e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149819df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46958/0 (socket says 172.21.15.97:46958) 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.143+0000 7f1495e00700 1 -- 172.21.15.97:0/3505960449 learned_addr learned my addr 172.21.15.97:0/3505960449 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.144+0000 7f1495e00700 1 -- 172.21.15.97:0/3505960449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149819e450 con 0x7f149810a0e0 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.144+0000 7f1495e00700 1 --2- 172.21.15.97:0/3505960449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149819df10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1488011b60 tx=0x7f1488011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.144+0000 7f148ea00700 1 -- 172.21.15.97:0/3505960449 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1488003b20 con 0x7f149810a0e0 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.144+0000 7f148ea00700 1 -- 172.21.15.97:0/3505960449 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1488003cc0 con 0x7f149810a0e0 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.144+0000 7f149ce00700 1 -- 172.21.15.97:0/3505960449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149819e6e0 con 0x7f149810a0e0 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.144+0000 7f148ea00700 1 -- 172.21.15.97:0/3505960449 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1488024840 con 0x7f149810a0e0 2024-03-26T18:47:46.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.144+0000 7f149ce00700 1 -- 172.21.15.97:0/3505960449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14981a13d0 con 0x7f149810a0e0 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.145+0000 7f148ea00700 1 -- 172.21.15.97:0/3505960449 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46283+0+0 (secure 0 0 0) 0x7f1488022020 con 0x7f149810a0e0 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.146+0000 7f149ce00700 1 -- 172.21.15.97:0/3505960449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14980639e0 con 0x7f149810a0e0 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.146+0000 7f148ea00700 1 --2- 172.21.15.97:0/3505960449 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f147c039180 0x7f147c03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.146+0000 7f148ea00700 1 -- 172.21.15.97:0/3505960449 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1488011e00 con 0x7f149810a0e0 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.146+0000 7f148fe00700 1 --2- 172.21.15.97:0/3505960449 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f147c039180 0x7f147c03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.147+0000 7f148fe00700 1 --2- 172.21.15.97:0/3505960449 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f147c039180 0x7f147c03b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1480006fd0 tx=0x7f1480006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.151+0000 7f148ea00700 1 -- 172.21.15.97:0/3505960449 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f148800b3e0 con 0x7f149810a0e0 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.355+0000 7f149ce00700 1 -- 172.21.15.97:0/3505960449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f14981a15a0 con 0x7f149810a0e0 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.357+0000 7f148ea00700 1 -- 172.21.15.97:0/3505960449 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f14981a15a0 con 0x7f149810a0e0 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.359+0000 7f149ce00700 1 -- 172.21.15.97:0/3505960449 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f147c039180 msgr2=0x7f147c03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.359+0000 7f149ce00700 1 --2- 172.21.15.97:0/3505960449 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f147c039180 0x7f147c03b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1480006fd0 tx=0x7f1480006e40 comp rx=0 tx=0).stop 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.359+0000 7f149ce00700 1 -- 172.21.15.97:0/3505960449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 msgr2=0x7f149819df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.359+0000 7f149ce00700 1 --2- 172.21.15.97:0/3505960449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149819df10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1488011b60 tx=0x7f1488011c40 comp rx=0 tx=0).stop 2024-03-26T18:47:46.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.360+0000 7f149ce00700 1 -- 172.21.15.97:0/3505960449 shutdown_connections 2024-03-26T18:47:46.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.360+0000 7f149ce00700 1 --2- 172.21.15.97:0/3505960449 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f147c039180 0x7f147c03b640 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:46.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.360+0000 7f149ce00700 1 --2- 172.21.15.97:0/3505960449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f149810a0e0 0x7f149819df10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:46.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.360+0000 7f149ce00700 1 -- 172.21.15.97:0/3505960449 >> 172.21.15.97:0/3505960449 conn(0x7f149807d510 msgr2=0x7f1498107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:46.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.360+0000 7f149ce00700 1 -- 172.21.15.97:0/3505960449 shutdown_connections 2024-03-26T18:47:46.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:45.360+0000 7f149ce00700 1 -- 172.21.15.97:0/3505960449 wait complete. 2024-03-26T18:47:46.002 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling cephadm module... 2024-03-26T18:47:47.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:46 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3902605072' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-26T18:47:47.351 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:47 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: ignoring --setuser ceph since I am not root 2024-03-26T18:47:47.351 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:47 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: ignoring --setgroup ceph since I am not root 2024-03-26T18:47:47.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.630+0000 7fc372600700 1 Processor -- start 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.630+0000 7fc372600700 1 -- start start 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.630+0000 7fc372600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.631+0000 7fc372600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c108800 con 0x7fc36c107e10 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.631+0000 7fc36b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.631+0000 7fc36b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46964/0 (socket says 172.21.15.97:46964) 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.631+0000 7fc36b400700 1 -- 172.21.15.97:0/1322515038 learned_addr learned my addr 172.21.15.97:0/1322515038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.631+0000 7fc36b400700 1 -- 172.21.15.97:0/1322515038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c108fc0 con 0x7fc36c107e10 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.632+0000 7fc36b400700 1 --2- 172.21.15.97:0/1322515038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c108230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0098a0 tx=0x7fc35c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcde69962855fc0d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.632+0000 7fc36a000700 1 -- 172.21.15.97:0/1322515038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc35c004160 con 0x7fc36c107e10 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.632+0000 7fc36a000700 1 -- 172.21.15.97:0/1322515038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc35c004300 con 0x7fc36c107e10 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.632+0000 7fc36a000700 1 -- 172.21.15.97:0/1322515038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc35c011b50 con 0x7fc36c107e10 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.632+0000 7fc372600700 1 -- 172.21.15.97:0/1322515038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 msgr2=0x7fc36c108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:47.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.632+0000 7fc372600700 1 --2- 172.21.15.97:0/1322515038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c108230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0098a0 tx=0x7fc35c009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.633+0000 7fc372600700 1 -- 172.21.15.97:0/1322515038 shutdown_connections 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.633+0000 7fc372600700 1 --2- 172.21.15.97:0/1322515038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c108230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.633+0000 7fc372600700 1 -- 172.21.15.97:0/1322515038 >> 172.21.15.97:0/1322515038 conn(0x7fc36c103410 msgr2=0x7fc36c105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.633+0000 7fc372600700 1 -- 172.21.15.97:0/1322515038 shutdown_connections 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.633+0000 7fc372600700 1 -- 172.21.15.97:0/1322515038 wait complete. 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.634+0000 7fc372600700 1 Processor -- start 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.634+0000 7fc372600700 1 -- start start 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.634+0000 7fc372600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.634+0000 7fc372600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c10c370 con 0x7fc36c107e10 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.635+0000 7fc36b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.635+0000 7fc36b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46974/0 (socket says 172.21.15.97:46974) 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.635+0000 7fc36b400700 1 -- 172.21.15.97:0/3902605072 learned_addr learned my addr 172.21.15.97:0/3902605072 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.635+0000 7fc36b400700 1 -- 172.21.15.97:0/3902605072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c19e4b0 con 0x7fc36c107e10 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.635+0000 7fc36b400700 1 --2- 172.21.15.97:0/3902605072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c19df70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc35c011670 tx=0x7fc35c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:47.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.635+0000 7fc363e00700 1 -- 172.21.15.97:0/3902605072 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc35c0039c0 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.635+0000 7fc363e00700 1 -- 172.21.15.97:0/3902605072 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc35c011e50 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.635+0000 7fc363e00700 1 -- 172.21.15.97:0/3902605072 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc35c01a5c0 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.635+0000 7fc372600700 1 -- 172.21.15.97:0/3902605072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36c19e740 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.636+0000 7fc372600700 1 -- 172.21.15.97:0/3902605072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36c1a1430 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.637+0000 7fc372600700 1 -- 172.21.15.97:0/3902605072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc36c0639e0 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.637+0000 7fc363e00700 1 -- 172.21.15.97:0/3902605072 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46283+0+0 (secure 0 0 0) 0x7fc35c024430 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.637+0000 7fc363e00700 1 --2- 172.21.15.97:0/3902605072 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fc350039180 0x7fc35003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.637+0000 7fc363e00700 1 -- 172.21.15.97:0/3902605072 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc35c04d4f0 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.637+0000 7fc36aa00700 1 --2- 172.21.15.97:0/3902605072 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fc350039180 0x7fc35003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.638+0000 7fc36aa00700 1 --2- 172.21.15.97:0/3902605072 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fc350039180 0x7fc35003b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc358006fd0 tx=0x7fc358006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.643+0000 7fc363e00700 1 -- 172.21.15.97:0/3902605072 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc35c00f2e0 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.921+0000 7fc372600700 1 -- 172.21.15.97:0/3902605072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc36c1a1600 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.975+0000 7fc363e00700 1 -- 172.21.15.97:0/3902605072 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc36c1a1600 con 0x7fc36c107e10 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.978+0000 7fc372600700 1 -- 172.21.15.97:0/3902605072 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fc350039180 msgr2=0x7fc35003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.978+0000 7fc372600700 1 --2- 172.21.15.97:0/3902605072 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fc350039180 0x7fc35003b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc358006fd0 tx=0x7fc358006e40 comp rx=0 tx=0).stop 2024-03-26T18:47:47.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.978+0000 7fc372600700 1 -- 172.21.15.97:0/3902605072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 msgr2=0x7fc36c19df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:47.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.978+0000 7fc363e00700 1 -- 172.21.15.97:0/3902605072 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 5) ==== 46294+0+0 (secure 0 0 0) 0x7fc35c02d3f0 con 0x7fc36c107e10 2024-03-26T18:47:47.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.980+0000 7fc372600700 1 --2- 172.21.15.97:0/3902605072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c19df70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc35c011670 tx=0x7fc35c011b60 comp rx=0 tx=0).stop 2024-03-26T18:47:47.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.980+0000 7fc372600700 1 -- 172.21.15.97:0/3902605072 shutdown_connections 2024-03-26T18:47:47.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.980+0000 7fc372600700 1 --2- 172.21.15.97:0/3902605072 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fc350039180 0x7fc35003b640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:47.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.980+0000 7fc372600700 1 --2- 172.21.15.97:0/3902605072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c107e10 0x7fc36c19df70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:47.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.980+0000 7fc372600700 1 -- 172.21.15.97:0/3902605072 >> 172.21.15.97:0/3902605072 conn(0x7fc36c103410 msgr2=0x7fc36c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:47.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.981+0000 7fc372600700 1 -- 172.21.15.97:0/3902605072 shutdown_connections 2024-03-26T18:47:47.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:46.981+0000 7fc372600700 1 -- 172.21.15.97:0/3902605072 wait complete. 2024-03-26T18:47:47.966 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:47 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:47.734+0000 7f1f901b7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T18:47:47.966 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:47 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:47.859+0000 7f1f901b7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T18:47:48.266 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:47 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3902605072' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-26T18:47:48.266 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:47 smithi097 ceph-mon[31955]: mgrmap e5: x(active, since 5s) 2024-03-26T18:47:48.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:48 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:48.666+0000 7f1f901b7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T18:47:48.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:48 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:48.793+0000 7f1f901b7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T18:47:49.216 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:48 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2096144276' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T18:47:49.216 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:48 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:48.982+0000 7f1f901b7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T18:47:49.217 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:49.111+0000 7f1f901b7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T18:47:49.217 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:49.224+0000 7f1f901b7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T18:47:49.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-26T18:47:49.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-26T18:47:49.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T18:47:49.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T18:47:49.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T18:47:49.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-26T18:47:49.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.274+0000 7f471b000700 1 Processor -- start 2024-03-26T18:47:49.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.274+0000 7f471b000700 1 -- start start 2024-03-26T18:47:49.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.274+0000 7f471b000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.275+0000 7f471b000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f471407c800 con 0x7f471407ddc0 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.275+0000 7f4713e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.275+0000 7f4713e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46996/0 (socket says 172.21.15.97:46996) 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.275+0000 7f4713e00700 1 -- 172.21.15.97:0/3761705082 learned_addr learned my addr 172.21.15.97:0/3761705082 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.275+0000 7f4713e00700 1 -- 172.21.15.97:0/3761705082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471407c980 con 0x7f471407ddc0 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.276+0000 7f4713e00700 1 --2- 172.21.15.97:0/3761705082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471407c2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47040098a0 tx=0x7f4704009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f433e595257f68b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.276+0000 7f4712a00700 1 -- 172.21.15.97:0/3761705082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4704004160 con 0x7f471407ddc0 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.276+0000 7f4712a00700 1 -- 172.21.15.97:0/3761705082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4704004300 con 0x7f471407ddc0 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.276+0000 7f4712a00700 1 -- 172.21.15.97:0/3761705082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4704011b50 con 0x7f471407ddc0 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.277+0000 7f471b000700 1 -- 172.21.15.97:0/3761705082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 msgr2=0x7f471407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.277+0000 7f471b000700 1 --2- 172.21.15.97:0/3761705082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471407c2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47040098a0 tx=0x7f4704009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:49.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.277+0000 7f471b000700 1 -- 172.21.15.97:0/3761705082 shutdown_connections 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.277+0000 7f471b000700 1 --2- 172.21.15.97:0/3761705082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471407c2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.277+0000 7f471b000700 1 -- 172.21.15.97:0/3761705082 >> 172.21.15.97:0/3761705082 conn(0x7f47141034a0 msgr2=0x7f47141058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.277+0000 7f471b000700 1 -- 172.21.15.97:0/3761705082 shutdown_connections 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.277+0000 7f471b000700 1 -- 172.21.15.97:0/3761705082 wait complete. 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.279+0000 7f471b000700 1 Processor -- start 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.279+0000 7f471b000700 1 -- start start 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.279+0000 7f471b000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.279+0000 7f471b000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f471410e350 con 0x7f471407ddc0 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.279+0000 7f4713e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.279+0000 7f4713e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471419df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47006/0 (socket says 172.21.15.97:47006) 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.280+0000 7f4713e00700 1 -- 172.21.15.97:0/2096144276 learned_addr learned my addr 172.21.15.97:0/2096144276 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:49.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.280+0000 7f4713e00700 1 -- 172.21.15.97:0/2096144276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471419e470 con 0x7f471407ddc0 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.280+0000 7f4713e00700 1 --2- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471419df30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4704011b60 tx=0x7f4704011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.280+0000 7f4710c00700 1 -- 172.21.15.97:0/2096144276 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4704003b20 con 0x7f471407ddc0 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.280+0000 7f4710c00700 1 -- 172.21.15.97:0/2096144276 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4704003cc0 con 0x7f471407ddc0 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.280+0000 7f4710c00700 1 -- 172.21.15.97:0/2096144276 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4704024840 con 0x7f471407ddc0 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.280+0000 7f471b000700 1 -- 172.21.15.97:0/2096144276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471419e700 con 0x7f471407ddc0 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.281+0000 7f471b000700 1 -- 172.21.15.97:0/2096144276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47141a13f0 con 0x7f471407ddc0 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.282+0000 7f4710c00700 1 -- 172.21.15.97:0/2096144276 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 46294+0+0 (secure 0 0 0) 0x7f4704022020 con 0x7f471407ddc0 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.282+0000 7f471b000700 1 -- 172.21.15.97:0/2096144276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47140639e0 con 0x7f471407ddc0 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.282+0000 7f4710c00700 1 --2- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f46f8039180 0x7f46f803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.282+0000 7f4710c00700 1 -- 172.21.15.97:0/2096144276 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4704011e00 con 0x7f471407ddc0 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.282+0000 7f4713400700 1 -- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f46f8039180 msgr2=0x7f46f803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/350003366 2024-03-26T18:47:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.283+0000 7f4713400700 1 --2- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f46f8039180 0x7f46f803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.288+0000 7f4710c00700 1 -- 172.21.15.97:0/2096144276 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f470404cd40 con 0x7f471407ddc0 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.484+0000 7f4713400700 1 -- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f46f8039180 msgr2=0x7f46f803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/350003366 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.484+0000 7f4713400700 1 --2- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f46f8039180 0x7f46f803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.566+0000 7f471b000700 1 -- 172.21.15.97:0/2096144276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f47141a15c0 con 0x7f471407ddc0 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.567+0000 7f4710c00700 1 -- 172.21.15.97:0/2096144276 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f47141a15c0 con 0x7f471407ddc0 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.570+0000 7f471b000700 1 -- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f46f8039180 msgr2=0x7f46f803b640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.570+0000 7f471b000700 1 --2- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f46f8039180 0x7f46f803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.570+0000 7f471b000700 1 -- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 msgr2=0x7f471419df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.570+0000 7f471b000700 1 --2- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471419df30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4704011b60 tx=0x7f4704011c40 comp rx=0 tx=0).stop 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.570+0000 7f471b000700 1 -- 172.21.15.97:0/2096144276 shutdown_connections 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.570+0000 7f471b000700 1 --2- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7f46f8039180 0x7f46f803b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:49.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.570+0000 7f471b000700 1 --2- 172.21.15.97:0/2096144276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471407ddc0 0x7f471419df30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:49.363 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.570+0000 7f471b000700 1 -- 172.21.15.97:0/2096144276 >> 172.21.15.97:0/2096144276 conn(0x7f47141034a0 msgr2=0x7f4714103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:49.363 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.570+0000 7f471b000700 1 -- 172.21.15.97:0/2096144276 shutdown_connections 2024-03-26T18:47:49.363 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:48.570+0000 7f471b000700 1 -- 172.21.15.97:0/2096144276 wait complete. 2024-03-26T18:47:49.363 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2024-03-26T18:47:49.363 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 5... 2024-03-26T18:47:49.755 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:49.559+0000 7f1f901b7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T18:47:49.755 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:49.703+0000 7f1f901b7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T18:47:50.101 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:49.817+0000 7f1f901b7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T18:47:50.102 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:49.942+0000 7f1f901b7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T18:47:50.513 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:50 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:50.159+0000 7f1f901b7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T18:47:51.566 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:51 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:51.404+0000 7f1f901b7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T18:47:51.566 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:51 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:51.573+0000 7f1f901b7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T18:47:52.343 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:51 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:51.985+0000 7f1f901b7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T18:47:52.343 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:52 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:52.100+0000 7f1f901b7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T18:47:52.343 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:52 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:52.350+0000 7f1f901b7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:47:52.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:52 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:52.591+0000 7f1f901b7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T18:47:53.266 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:52 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:52.959+0000 7f1f901b7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:47:53.266 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:53 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:53.273+0000 7f1f901b7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T18:47:53.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:53 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:53.573+0000 7f1f901b7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T18:47:54.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:54 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:54.578+0000 7f1f901b7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T18:47:55.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:55.263+0000 7f1f901b7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T18:47:55.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:55.390+0000 7f1f901b7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T18:47:56.101 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:47:55.668+0000 7f1f901b7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T18:47:56.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: Active manager daemon x restarted 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: Activating manager daemon x 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: osdmap e2: 0 total, 0 up, 0 in 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: mgrmap e6: x(active, starting, since 0.00494075s) 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: Manager daemon x is now available 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:47:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:55 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:47:57.038 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:56 smithi097 ceph-mon[31955]: Found migration_current of "None". Setting to last migration. 2024-03-26T18:47:57.038 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:56 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:47:57.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:56 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:47:57.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:56 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T18:47:57.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:56 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T18:47:57.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:56 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:47:57.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:56 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:47:57.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:56 smithi097 ceph-mon[31955]: mgrmap e7: x(active, since 1.0134s) 2024-03-26T18:47:57.368 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-26T18:47:57.368 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-26T18:47:57.369 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T18:47:57.369 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-26T18:47:57.369 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.911+0000 7fcce9600700 1 Processor -- start 2024-03-26T18:47:57.369 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.911+0000 7fcce9600700 1 -- start start 2024-03-26T18:47:57.369 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.912+0000 7fcce9600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce41082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:57.369 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.912+0000 7fcce9600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4108800 con 0x7fcce4107ea0 2024-03-26T18:47:57.376 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.912+0000 7fcce2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce41082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:57.376 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.912+0000 7fcce2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce41082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47012/0 (socket says 172.21.15.97:47012) 2024-03-26T18:47:57.376 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.912+0000 7fcce2a00700 1 -- 172.21.15.97:0/3437366147 learned_addr learned my addr 172.21.15.97:0/3437366147 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:57.376 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.913+0000 7fcce2a00700 1 -- 172.21.15.97:0/3437366147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce4108f60 con 0x7fcce4107ea0 2024-03-26T18:47:57.376 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.913+0000 7fcce2a00700 1 --2- 172.21.15.97:0/3437366147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce41082c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098a0 tx=0x7fccd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82fb939e997a702 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.913+0000 7fcce1600700 1 -- 172.21.15.97:0/3437366147 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd4004160 con 0x7fcce4107ea0 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.913+0000 7fcce1600700 1 -- 172.21.15.97:0/3437366147 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fccd4004300 con 0x7fcce4107ea0 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.913+0000 7fcce1600700 1 -- 172.21.15.97:0/3437366147 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd4011b50 con 0x7fcce4107ea0 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.914+0000 7fcce9600700 1 -- 172.21.15.97:0/3437366147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 msgr2=0x7fcce41082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.914+0000 7fcce9600700 1 --2- 172.21.15.97:0/3437366147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce41082c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098a0 tx=0x7fccd4009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.914+0000 7fcce9600700 1 -- 172.21.15.97:0/3437366147 shutdown_connections 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.914+0000 7fcce9600700 1 --2- 172.21.15.97:0/3437366147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce41082c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.914+0000 7fcce9600700 1 -- 172.21.15.97:0/3437366147 >> 172.21.15.97:0/3437366147 conn(0x7fcce41034a0 msgr2=0x7fcce41058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.914+0000 7fcce9600700 1 -- 172.21.15.97:0/3437366147 shutdown_connections 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.914+0000 7fcce9600700 1 -- 172.21.15.97:0/3437366147 wait complete. 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.915+0000 7fcce9600700 1 Processor -- start 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.915+0000 7fcce9600700 1 -- start start 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.915+0000 7fcce9600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:57.377 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.915+0000 7fcce9600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcce410c220 con 0x7fcce4107ea0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.915+0000 7fcce2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.916+0000 7fcce2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce419df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47024/0 (socket says 172.21.15.97:47024) 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.916+0000 7fcce2a00700 1 -- 172.21.15.97:0/4260207393 learned_addr learned my addr 172.21.15.97:0/4260207393 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.916+0000 7fcce2a00700 1 -- 172.21.15.97:0/4260207393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce419e490 con 0x7fcce4107ea0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.916+0000 7fcce2a00700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce419df50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fccd4011b60 tx=0x7fccd4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.916+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd4003b20 con 0x7fcce4107ea0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.916+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fccd4003cc0 con 0x7fcce4107ea0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.916+0000 7fcce9600700 1 -- 172.21.15.97:0/4260207393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce419e720 con 0x7fcce4107ea0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.916+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd4024840 con 0x7fcce4107ea0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.917+0000 7fcce9600700 1 -- 172.21.15.97:0/4260207393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce41a1410 con 0x7fcce4107ea0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.918+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 46294+0+0 (secure 0 0 0) 0x7fccd4022020 con 0x7fcce4107ea0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.918+0000 7fccdb400700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 0x7fccc803b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.918+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 --> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fccc803bda0 con 0x7fccc80391d0 2024-03-26T18:47:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.918+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fccd4011e00 con 0x7fcce4107ea0 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.918+0000 7fcce2000700 1 -- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 msgr2=0x7fccc803b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/350003366 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:49.918+0000 7fcce2000700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 0x7fccc803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:50.119+0000 7fcce2000700 1 -- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 msgr2=0x7fccc803b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/350003366 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:50.119+0000 7fcce2000700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 0x7fccc803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:50.520+0000 7fcce2000700 1 -- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 msgr2=0x7fccc803b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/350003366 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:50.520+0000 7fcce2000700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 0x7fccc803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:51.321+0000 7fcce2000700 1 -- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 msgr2=0x7fccc803b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/350003366 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:51.321+0000 7fcce2000700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 0x7fccc803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:52.923+0000 7fcce2000700 1 -- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 msgr2=0x7fccc803b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/350003366 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:52.923+0000 7fcce2000700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 0x7fccc803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:55.675+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mgrmap(e 6) ==== 46061+0+0 (secure 0 0 0) 0x7fccd4024c00 con 0x7fcce4107ea0 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:55.675+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 msgr2=0x7fccc803b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:47:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:55.675+0000 7fccdb400700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 0x7fccc803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.679+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 7) ==== 46188+0+0 (secure 0 0 0) 0x7fccd404eb10 con 0x7fcce4107ea0 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.680+0000 7fccdb400700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fccc803cac0 0x7fccc803eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.680+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 --> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fccc803bda0 con 0x7fccc803cac0 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.681+0000 7fcce2000700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fccc803cac0 0x7fccc803eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.684+0000 7fcce2000700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fccc803cac0 0x7fccc803eeb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fccd0003a10 tx=0x7fccd00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.686+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 <== mgr.14120 v2:172.21.15.97:6800/3462603610 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fccc803bda0 con 0x7fccc803cac0 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.693+0000 7fccd8c00700 1 -- 172.21.15.97:0/4260207393 --> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fcce410c3a0 con 0x7fccc803cac0 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.694+0000 7fccdb400700 1 -- 172.21.15.97:0/4260207393 <== mgr.14120 v2:172.21.15.97:6800/3462603610 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fcce410c3a0 con 0x7fccc803cac0 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.694+0000 7fccd8c00700 1 -- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fccc803cac0 msgr2=0x7fccc803eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.694+0000 7fccd8c00700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fccc803cac0 0x7fccc803eeb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fccd0003a10 tx=0x7fccd00092b0 comp rx=0 tx=0).stop 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.694+0000 7fccd8c00700 1 -- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 msgr2=0x7fcce419df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.694+0000 7fccd8c00700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce419df50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fccd4011b60 tx=0x7fccd4011c40 comp rx=0 tx=0).stop 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.694+0000 7fccd8c00700 1 -- 172.21.15.97:0/4260207393 shutdown_connections 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.694+0000 7fccd8c00700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fccc803cac0 0x7fccc803eeb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.694+0000 7fccd8c00700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4107ea0 0x7fcce419df50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:57.381 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.694+0000 7fccd8c00700 1 --2- 172.21.15.97:0/4260207393 >> [v2:172.21.15.97:6800/350003366,v1:172.21.15.97:6801/350003366] conn(0x7fccc80391d0 0x7fccc803b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:57.381 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.694+0000 7fccd8c00700 1 -- 172.21.15.97:0/4260207393 >> 172.21.15.97:0/4260207393 conn(0x7fcce41034a0 msgr2=0x7fcce4103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:57.381 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.695+0000 7fccd8c00700 1 -- 172.21.15.97:0/4260207393 shutdown_connections 2024-03-26T18:47:57.381 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:56.695+0000 7fccd8c00700 1 -- 172.21.15.97:0/4260207393 wait complete. 2024-03-26T18:47:57.381 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 5 is available 2024-03-26T18:47:57.381 INFO:teuthology.orchestra.run.smithi097.stdout:Setting orchestrator backend to cephadm... 2024-03-26T18:47:57.961 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:57 smithi097 ceph-mon[31955]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T18:47:57.961 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:57 smithi097 ceph-mon[31955]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T18:47:58.914 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:58 smithi097 ceph-mon[31955]: [26/Mar/2024:18:47:57] ENGINE Bus STARTING 2024-03-26T18:47:58.914 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:58 smithi097 ceph-mon[31955]: [26/Mar/2024:18:47:57] ENGINE Serving on https://172.21.15.97:7150 2024-03-26T18:47:58.914 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:58 smithi097 ceph-mon[31955]: mgrmap e8: x(active, since 2s) 2024-03-26T18:47:58.915 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:58 smithi097 ceph-mon[31955]: [26/Mar/2024:18:47:57] ENGINE Serving on http://172.21.15.97:8765 2024-03-26T18:47:58.915 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:58 smithi097 ceph-mon[31955]: [26/Mar/2024:18:47:57] ENGINE Bus STARTED 2024-03-26T18:47:58.915 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:58 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:47:58.915 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:58 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:47:58.915 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:58 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:47:58.929 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.968+0000 7f77d5200700 1 Processor -- start 2024-03-26T18:47:58.929 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.968+0000 7f77d5200700 1 -- start start 2024-03-26T18:47:58.930 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.969+0000 7f77d5200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d0108750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:58.930 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.969+0000 7f77d5200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0108d20 con 0x7f77d0108330 2024-03-26T18:47:58.930 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.969+0000 7f77cde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d0108750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:58.930 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.969+0000 7f77cde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d0108750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56942/0 (socket says 172.21.15.97:56942) 2024-03-26T18:47:58.930 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.969+0000 7f77cde00700 1 -- 172.21.15.97:0/1927098537 learned_addr learned my addr 172.21.15.97:0/1927098537 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:58.931 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.970+0000 7f77cde00700 1 -- 172.21.15.97:0/1927098537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d0106970 con 0x7f77d0108330 2024-03-26T18:47:58.932 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.970+0000 7f77cde00700 1 --2- 172.21.15.97:0/1927098537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d0108750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f77c00098a0 tx=0x7f77c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140ab84047629a5b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:58.932 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.970+0000 7f77cca00700 1 -- 172.21.15.97:0/1927098537 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0004160 con 0x7f77d0108330 2024-03-26T18:47:58.932 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.970+0000 7f77cca00700 1 -- 172.21.15.97:0/1927098537 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77c0004300 con 0x7f77d0108330 2024-03-26T18:47:58.932 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.970+0000 7f77cca00700 1 -- 172.21.15.97:0/1927098537 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0011b50 con 0x7f77d0108330 2024-03-26T18:47:58.932 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.971+0000 7f77d5200700 1 -- 172.21.15.97:0/1927098537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 msgr2=0x7f77d0108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:58.932 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.971+0000 7f77d5200700 1 --2- 172.21.15.97:0/1927098537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d0108750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f77c00098a0 tx=0x7f77c0009bb0 comp rx=0 tx=0).stop 2024-03-26T18:47:58.936 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.971+0000 7f77d5200700 1 -- 172.21.15.97:0/1927098537 shutdown_connections 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.971+0000 7f77d5200700 1 --2- 172.21.15.97:0/1927098537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d0108750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.971+0000 7f77d5200700 1 -- 172.21.15.97:0/1927098537 >> 172.21.15.97:0/1927098537 conn(0x7f77d0102310 msgr2=0x7f77d0104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.971+0000 7f77d5200700 1 -- 172.21.15.97:0/1927098537 shutdown_connections 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.971+0000 7f77d5200700 1 -- 172.21.15.97:0/1927098537 wait complete. 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.972+0000 7f77d5200700 1 Processor -- start 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.973+0000 7f77d5200700 1 -- start start 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.973+0000 7f77d5200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.973+0000 7f77d5200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77d007b130 con 0x7f77d0108330 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.973+0000 7f77cde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.973+0000 7f77cde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d019bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56958/0 (socket says 172.21.15.97:56958) 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.973+0000 7f77cde00700 1 -- 172.21.15.97:0/2534505867 learned_addr learned my addr 172.21.15.97:0/2534505867 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.974+0000 7f77cde00700 1 -- 172.21.15.97:0/2534505867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d019c2e0 con 0x7f77d0108330 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.974+0000 7f77cde00700 1 --2- 172.21.15.97:0/2534505867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d019bda0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f77c0000c00 tx=0x7f77c0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:58.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.974+0000 7f77c6a00700 1 -- 172.21.15.97:0/2534505867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0003b20 con 0x7f77d0108330 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.974+0000 7f77c6a00700 1 -- 172.21.15.97:0/2534505867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77c0003cc0 con 0x7f77d0108330 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.974+0000 7f77d5200700 1 -- 172.21.15.97:0/2534505867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d019c570 con 0x7f77d0108330 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.974+0000 7f77c6a00700 1 -- 172.21.15.97:0/2534505867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0024910 con 0x7f77d0108330 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.974+0000 7f77d5200700 1 -- 172.21.15.97:0/2534505867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d019f260 con 0x7f77d0108330 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.975+0000 7f77c6a00700 1 -- 172.21.15.97:0/2534505867 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f77c0022020 con 0x7f77d0108330 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.975+0000 7f77d5200700 1 -- 172.21.15.97:0/2534505867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d00639e0 con 0x7f77d0108330 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.976+0000 7f77c6a00700 1 --2- 172.21.15.97:0/2534505867 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f77b40391d0 0x7f77b403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.976+0000 7f77c6a00700 1 -- 172.21.15.97:0/2534505867 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f77c0011e00 con 0x7f77d0108330 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.976+0000 7f77cd400700 1 --2- 172.21.15.97:0/2534505867 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f77b40391d0 0x7f77b403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.977+0000 7f77cd400700 1 --2- 172.21.15.97:0/2534505867 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f77b40391d0 0x7f77b403b690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77b8006fd0 tx=0x7f77b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:57.982+0000 7f77c6a00700 1 -- 172.21.15.97:0/2534505867 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f77c00172b0 con 0x7f77d0108330 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.218+0000 7f77d5200700 1 -- 172.21.15.97:0/2534505867 --> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f77d00008d0 con 0x7f77b40391d0 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.229+0000 7f77c6a00700 1 -- 172.21.15.97:0/2534505867 <== mgr.14120 v2:172.21.15.97:6800/3462603610 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f77d00008d0 con 0x7f77b40391d0 2024-03-26T18:47:58.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.233+0000 7f77d5200700 1 -- 172.21.15.97:0/2534505867 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f77b40391d0 msgr2=0x7f77b403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:58.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.233+0000 7f77d5200700 1 --2- 172.21.15.97:0/2534505867 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f77b40391d0 0x7f77b403b690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77b8006fd0 tx=0x7f77b8006e40 comp rx=0 tx=0).stop 2024-03-26T18:47:58.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.233+0000 7f77d5200700 1 -- 172.21.15.97:0/2534505867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 msgr2=0x7f77d019bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:47:58.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.233+0000 7f77d5200700 1 --2- 172.21.15.97:0/2534505867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d019bda0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f77c0000c00 tx=0x7f77c0011c10 comp rx=0 tx=0).stop 2024-03-26T18:47:58.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.233+0000 7f77d5200700 1 -- 172.21.15.97:0/2534505867 shutdown_connections 2024-03-26T18:47:58.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.233+0000 7f77d5200700 1 --2- 172.21.15.97:0/2534505867 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f77b40391d0 0x7f77b403b690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:58.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.233+0000 7f77d5200700 1 --2- 172.21.15.97:0/2534505867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108330 0x7f77d019bda0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:47:58.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.233+0000 7f77d5200700 1 -- 172.21.15.97:0/2534505867 >> 172.21.15.97:0/2534505867 conn(0x7f77d0102310 msgr2=0x7f77d0192930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:47:58.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.233+0000 7f77d5200700 1 -- 172.21.15.97:0/2534505867 shutdown_connections 2024-03-26T18:47:58.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:58.233+0000 7f77d5200700 1 -- 172.21.15.97:0/2534505867 wait complete. 2024-03-26T18:47:59.830 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:47:59 smithi097 ceph-mon[31955]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-26T18:48:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.578+0000 7f4418000700 1 Processor -- start 2024-03-26T18:48:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.579+0000 7f4418000700 1 -- start start 2024-03-26T18:48:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.579+0000 7f4418000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f4410108240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.579+0000 7f4418000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4410108810 con 0x7f4410107e20 2024-03-26T18:48:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.579+0000 7f4415600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f4410108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.580+0000 7f4415600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f4410108240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56960/0 (socket says 172.21.15.97:56960) 2024-03-26T18:48:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.580+0000 7f4415600700 1 -- 172.21.15.97:0/167347667 learned_addr learned my addr 172.21.15.97:0/167347667 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.580+0000 7f4415600700 1 -- 172.21.15.97:0/167347667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4410108fd0 con 0x7f4410107e20 2024-03-26T18:48:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.580+0000 7f4415600700 1 --2- 172.21.15.97:0/167347667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f4410108240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44040098a0 tx=0x7f4404009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84de24bc1505262d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.581+0000 7f440fe00700 1 -- 172.21.15.97:0/167347667 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4404004160 con 0x7f4410107e20 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.581+0000 7f440fe00700 1 -- 172.21.15.97:0/167347667 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4404004300 con 0x7f4410107e20 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.581+0000 7f440fe00700 1 -- 172.21.15.97:0/167347667 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4404011b50 con 0x7f4410107e20 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.581+0000 7f4418000700 1 -- 172.21.15.97:0/167347667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 msgr2=0x7f4410108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.581+0000 7f4418000700 1 --2- 172.21.15.97:0/167347667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f4410108240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44040098a0 tx=0x7f4404009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.582+0000 7f4418000700 1 -- 172.21.15.97:0/167347667 shutdown_connections 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.582+0000 7f4418000700 1 --2- 172.21.15.97:0/167347667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f4410108240 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.582+0000 7f4418000700 1 -- 172.21.15.97:0/167347667 >> 172.21.15.97:0/167347667 conn(0x7f4410103440 msgr2=0x7f4410105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.582+0000 7f4418000700 1 -- 172.21.15.97:0/167347667 shutdown_connections 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.582+0000 7f4418000700 1 -- 172.21.15.97:0/167347667 wait complete. 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.583+0000 7f4418000700 1 Processor -- start 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.583+0000 7f4418000700 1 -- start start 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.583+0000 7f4418000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f441019df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:00.477 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.583+0000 7f4418000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f441010c380 con 0x7f4410107e20 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.583+0000 7f4415600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f441019df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.583+0000 7f4415600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f441019df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56972/0 (socket says 172.21.15.97:56972) 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.583+0000 7f4415600700 1 -- 172.21.15.97:0/2236978385 learned_addr learned my addr 172.21.15.97:0/2236978385 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.584+0000 7f4415600700 1 -- 172.21.15.97:0/2236978385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441019e470 con 0x7f4410107e20 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.584+0000 7f4415600700 1 --2- 172.21.15.97:0/2236978385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f441019df30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4404000c00 tx=0x7f4404011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.584+0000 7f440e000700 1 -- 172.21.15.97:0/2236978385 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4404003b20 con 0x7f4410107e20 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.584+0000 7f440e000700 1 -- 172.21.15.97:0/2236978385 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4404003cc0 con 0x7f4410107e20 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.584+0000 7f4418000700 1 -- 172.21.15.97:0/2236978385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441019e700 con 0x7f4410107e20 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.584+0000 7f440e000700 1 -- 172.21.15.97:0/2236978385 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4404024910 con 0x7f4410107e20 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.585+0000 7f4418000700 1 -- 172.21.15.97:0/2236978385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44101a13f0 con 0x7f4410107e20 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.586+0000 7f440e000700 1 -- 172.21.15.97:0/2236978385 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f4404022020 con 0x7f4410107e20 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.586+0000 7f4418000700 1 -- 172.21.15.97:0/2236978385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44100639e0 con 0x7f4410107e20 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.586+0000 7f440e000700 1 --2- 172.21.15.97:0/2236978385 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f43f80391d0 0x7f43f803b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.586+0000 7f440e000700 1 -- 172.21.15.97:0/2236978385 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4404011e00 con 0x7f4410107e20 2024-03-26T18:48:00.478 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.586+0000 7f4414c00700 1 --2- 172.21.15.97:0/2236978385 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f43f80391d0 0x7f43f803b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.587+0000 7f4414c00700 1 --2- 172.21.15.97:0/2236978385 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f43f80391d0 0x7f43f803b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f43fc006fd0 tx=0x7f43fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.592+0000 7f440e000700 1 -- 172.21.15.97:0/2236978385 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f440401ad80 con 0x7f4410107e20 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.815+0000 7f4418000700 1 -- 172.21.15.97:0/2236978385 --> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f44101045e0 con 0x7f43f80391d0 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.817+0000 7f440e000700 1 -- 172.21.15.97:0/2236978385 <== mgr.14120 v2:172.21.15.97:6800/3462603610 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f44101045e0 con 0x7f43f80391d0 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.820+0000 7f4418000700 1 -- 172.21.15.97:0/2236978385 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f43f80391d0 msgr2=0x7f43f803b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.820+0000 7f4418000700 1 --2- 172.21.15.97:0/2236978385 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f43f80391d0 0x7f43f803b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f43fc006fd0 tx=0x7f43fc006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.820+0000 7f4418000700 1 -- 172.21.15.97:0/2236978385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 msgr2=0x7f441019df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.820+0000 7f4418000700 1 --2- 172.21.15.97:0/2236978385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f441019df30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4404000c00 tx=0x7f4404011c10 comp rx=0 tx=0).stop 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.820+0000 7f4418000700 1 -- 172.21.15.97:0/2236978385 shutdown_connections 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.820+0000 7f4418000700 1 --2- 172.21.15.97:0/2236978385 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f43f80391d0 0x7f43f803b690 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.820+0000 7f4418000700 1 --2- 172.21.15.97:0/2236978385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410107e20 0x7f441019df30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.820+0000 7f4418000700 1 -- 172.21.15.97:0/2236978385 >> 172.21.15.97:0/2236978385 conn(0x7f4410103440 msgr2=0x7f4410103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.820+0000 7f4418000700 1 -- 172.21.15.97:0/2236978385 shutdown_connections 2024-03-26T18:48:00.479 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:47:59.821+0000 7f4418000700 1 -- 172.21.15.97:0/2236978385 wait complete. 2024-03-26T18:48:00.480 INFO:teuthology.orchestra.run.smithi097.stdout:Generating ssh key... 2024-03-26T18:48:01.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:00 smithi097 ceph-mon[31955]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:01.602 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: Generating public/private rsa key pair. 2024-03-26T18:48:01.602 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: Your identification has been saved in /tmp/tmpr9unqt86/key. 2024-03-26T18:48:01.602 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: Your public key has been saved in /tmp/tmpr9unqt86/key.pub. 2024-03-26T18:48:01.602 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: The key fingerprint is: 2024-03-26T18:48:01.602 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: SHA256:imLST8mBFkvDwmhZchUhO8JNhl5dzPerds1OdhZhr6I ceph-2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:48:01.602 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: The key's randomart image is: 2024-03-26T18:48:01.602 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: +---[RSA 3072]----+ 2024-03-26T18:48:01.602 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: | ..*o+*. | 2024-03-26T18:48:01.602 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: |+oO.o. o . | 2024-03-26T18:48:01.602 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: |+**+ . . o | 2024-03-26T18:48:01.603 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: |.+.=. . . o| 2024-03-26T18:48:01.603 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: | + . S . ..| 2024-03-26T18:48:01.603 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: | o . + . . ..| 2024-03-26T18:48:01.603 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: |. + = . . +o.o | 2024-03-26T18:48:01.603 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: | o + o oo+o | 2024-03-26T18:48:01.603 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: | . . E .. | 2024-03-26T18:48:01.603 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:01 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: +----[SHA256]-----+ 2024-03-26T18:48:02.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.103+0000 7f7c5a400700 1 Processor -- start 2024-03-26T18:48:02.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.103+0000 7f7c5a400700 1 -- start start 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.104+0000 7f7c5a400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.104+0000 7f7c5a400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5407c800 con 0x7f7c5407ddc0 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.104+0000 7f7c59000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.104+0000 7f7c59000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37826/0 (socket says 172.21.15.97:37826) 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.104+0000 7f7c59000700 1 -- 172.21.15.97:0/814267873 learned_addr learned my addr 172.21.15.97:0/814267873 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.105+0000 7f7c59000700 1 -- 172.21.15.97:0/814267873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5407c980 con 0x7f7c5407ddc0 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.105+0000 7f7c59000700 1 --2- 172.21.15.97:0/814267873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5407c2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009e80 tx=0x7f7c4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195bf777e4384c5a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.105+0000 7f7c53400700 1 -- 172.21.15.97:0/814267873 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c48017070 con 0x7f7c5407ddc0 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.106+0000 7f7c53400700 1 -- 172.21.15.97:0/814267873 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c4800b8c0 con 0x7f7c5407ddc0 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.106+0000 7f7c53400700 1 -- 172.21.15.97:0/814267873 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c480045c0 con 0x7f7c5407ddc0 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.106+0000 7f7c5a400700 1 -- 172.21.15.97:0/814267873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 msgr2=0x7f7c5407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.106+0000 7f7c5a400700 1 --2- 172.21.15.97:0/814267873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5407c2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009e80 tx=0x7f7c4800b330 comp rx=0 tx=0).stop 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.106+0000 7f7c5a400700 1 -- 172.21.15.97:0/814267873 shutdown_connections 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.106+0000 7f7c5a400700 1 --2- 172.21.15.97:0/814267873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5407c2c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:02.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.106+0000 7f7c5a400700 1 -- 172.21.15.97:0/814267873 >> 172.21.15.97:0/814267873 conn(0x7f7c54103240 msgr2=0x7f7c54105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.107+0000 7f7c5a400700 1 -- 172.21.15.97:0/814267873 shutdown_connections 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.107+0000 7f7c5a400700 1 -- 172.21.15.97:0/814267873 wait complete. 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.108+0000 7f7c5a400700 1 Processor -- start 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.108+0000 7f7c5a400700 1 -- start start 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.108+0000 7f7c5a400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.108+0000 7f7c5a400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5410e150 con 0x7f7c5407ddc0 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.108+0000 7f7c59000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.108+0000 7f7c59000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37840/0 (socket says 172.21.15.97:37840) 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.108+0000 7f7c59000700 1 -- 172.21.15.97:0/2094174904 learned_addr learned my addr 172.21.15.97:0/2094174904 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.109+0000 7f7c59000700 1 -- 172.21.15.97:0/2094174904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5419e260 con 0x7f7c5407ddc0 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.109+0000 7f7c59000700 1 --2- 172.21.15.97:0/2094174904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5419dd20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7c480072a0 tx=0x7f7c4800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.109+0000 7f7c51600700 1 -- 172.21.15.97:0/2094174904 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c48020020 con 0x7f7c5407ddc0 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.109+0000 7f7c51600700 1 -- 172.21.15.97:0/2094174904 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c48003b50 con 0x7f7c5407ddc0 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.109+0000 7f7c51600700 1 -- 172.21.15.97:0/2094174904 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c48017040 con 0x7f7c5407ddc0 2024-03-26T18:48:02.277 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.110+0000 7f7c5a400700 1 -- 172.21.15.97:0/2094174904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5419e4f0 con 0x7f7c5407ddc0 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.110+0000 7f7c5a400700 1 -- 172.21.15.97:0/2094174904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c541a11d0 con 0x7f7c5407ddc0 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.111+0000 7f7c5a400700 1 -- 172.21.15.97:0/2094174904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c540639e0 con 0x7f7c5407ddc0 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.111+0000 7f7c51600700 1 -- 172.21.15.97:0/2094174904 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f7c48023020 con 0x7f7c5407ddc0 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.111+0000 7f7c51600700 1 --2- 172.21.15.97:0/2094174904 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f7c3c039130 0x7f7c3c03b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.111+0000 7f7c51600700 1 -- 172.21.15.97:0/2094174904 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c4800f050 con 0x7f7c5407ddc0 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.112+0000 7f7c53e00700 1 --2- 172.21.15.97:0/2094174904 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f7c3c039130 0x7f7c3c03b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.112+0000 7f7c53e00700 1 --2- 172.21.15.97:0/2094174904 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f7c3c039130 0x7f7c3c03b5f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006fd0 tx=0x7f7c40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.117+0000 7f7c51600700 1 -- 172.21.15.97:0/2094174904 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7c4802a020 con 0x7f7c5407ddc0 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.341+0000 7f7c5a400700 1 -- 172.21.15.97:0/2094174904 --> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7c54104530 con 0x7f7c3c039130 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.562+0000 7f7c51600700 1 -- 172.21.15.97:0/2094174904 <== mgr.14120 v2:172.21.15.97:6800/3462603610 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7c54104530 con 0x7f7c3c039130 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.565+0000 7f7c5a400700 1 -- 172.21.15.97:0/2094174904 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f7c3c039130 msgr2=0x7f7c3c03b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.565+0000 7f7c5a400700 1 --2- 172.21.15.97:0/2094174904 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f7c3c039130 0x7f7c3c03b5f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006fd0 tx=0x7f7c40006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.565+0000 7f7c5a400700 1 -- 172.21.15.97:0/2094174904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 msgr2=0x7f7c5419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.565+0000 7f7c5a400700 1 --2- 172.21.15.97:0/2094174904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5419dd20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7c480072a0 tx=0x7f7c4800bd50 comp rx=0 tx=0).stop 2024-03-26T18:48:02.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.565+0000 7f7c5a400700 1 -- 172.21.15.97:0/2094174904 shutdown_connections 2024-03-26T18:48:02.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.565+0000 7f7c5a400700 1 --2- 172.21.15.97:0/2094174904 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f7c3c039130 0x7f7c3c03b5f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:02.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.565+0000 7f7c5a400700 1 --2- 172.21.15.97:0/2094174904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5407ddc0 0x7f7c5419dd20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:02.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.565+0000 7f7c5a400700 1 -- 172.21.15.97:0/2094174904 >> 172.21.15.97:0/2094174904 conn(0x7f7c54103240 msgr2=0x7f7c54103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:02.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.565+0000 7f7c5a400700 1 -- 172.21.15.97:0/2094174904 shutdown_connections 2024-03-26T18:48:02.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:01.565+0000 7f7c5a400700 1 -- 172.21.15.97:0/2094174904 wait complete. 2024-03-26T18:48:02.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:02 smithi097 ceph-mon[31955]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:02.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:02 smithi097 ceph-mon[31955]: Generating ssh key... 2024-03-26T18:48:02.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:02 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:48:02.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:02 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:48:03.816 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD6wr2B0ZNNqNIANPkfHHAQuw4/mKkq4IBbSCA5LG7xLkunFKKTw3LqrakJOlCJs7XxSY49PB0Idv57MuXrHrwLh1jJRDjvbJz0+5i+ldX4gZ1/RS9YhoXsict9Do/xDLTkfgmb3ihCprxArg/8E6N0Zn8Zs6VQvOoWsqFDY3pDgqkb0GkQzmDyL/JbceSoUYfxPd/6tl5F0MU9UhSTJi306gL3ppEm1pU0SnuuMFPUwc4aDIF9+cZYHKwZRVpmm++kG9Xst00gII+UqDMMLQqnn5nD82v2h8rQASYRPRWbIzUWphzSr+3S3JSVj4ZQjSMLJSjsqubEkr0FWwreQ996nvAXvhBYYhwQSq9Y5HUkmpUwyvPYmXxeNVrz3BNU/oMuIpB2DmJhPkxUqKvPjNQks08AjP87wYffIXOgKqUb08YHJ9EzsrtUDg/v4Hum1giB9trI4jm9ve1G4dZP1a3bjIVdCVsqrvLE6oh/oTcgnRGUg55uf6BFO26bx3C94U= ceph-2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:48:03.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.883+0000 7fbc42400700 1 Processor -- start 2024-03-26T18:48:03.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.884+0000 7fbc42400700 1 -- start start 2024-03-26T18:48:03.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.884+0000 7fbc42400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c106af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.884+0000 7fbc42400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c07c6a0 con 0x7fbc3c1066d0 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.885+0000 7fbc3b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.885+0000 7fbc3b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37850/0 (socket says 172.21.15.97:37850) 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.885+0000 7fbc3b400700 1 -- 172.21.15.97:0/2354057677 learned_addr learned my addr 172.21.15.97:0/2354057677 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.885+0000 7fbc3b400700 1 -- 172.21.15.97:0/2354057677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c107030 con 0x7fbc3c1066d0 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.885+0000 7fbc3b400700 1 --2- 172.21.15.97:0/2354057677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c106af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009e80 tx=0x7fbc2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794dab9b4f45f21f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.885+0000 7fbc3a000700 1 -- 172.21.15.97:0/2354057677 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc2c017070 con 0x7fbc3c1066d0 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.886+0000 7fbc3a000700 1 -- 172.21.15.97:0/2354057677 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc2c00b8c0 con 0x7fbc3c1066d0 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.886+0000 7fbc3a000700 1 -- 172.21.15.97:0/2354057677 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc2c0045c0 con 0x7fbc3c1066d0 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.886+0000 7fbc42400700 1 -- 172.21.15.97:0/2354057677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 msgr2=0x7fbc3c106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.886+0000 7fbc42400700 1 --2- 172.21.15.97:0/2354057677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c106af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009e80 tx=0x7fbc2c00b330 comp rx=0 tx=0).stop 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.886+0000 7fbc42400700 1 -- 172.21.15.97:0/2354057677 shutdown_connections 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.886+0000 7fbc42400700 1 --2- 172.21.15.97:0/2354057677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c106af0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:03.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.886+0000 7fbc42400700 1 -- 172.21.15.97:0/2354057677 >> 172.21.15.97:0/2354057677 conn(0x7fbc3c102350 msgr2=0x7fbc3c104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.886+0000 7fbc42400700 1 -- 172.21.15.97:0/2354057677 shutdown_connections 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.886+0000 7fbc42400700 1 -- 172.21.15.97:0/2354057677 wait complete. 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.887+0000 7fbc42400700 1 Processor -- start 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.888+0000 7fbc42400700 1 -- start start 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.888+0000 7fbc42400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c19ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.888+0000 7fbc42400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c108050 con 0x7fbc3c1066d0 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.888+0000 7fbc3b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c19ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.888+0000 7fbc3b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c19ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37852/0 (socket says 172.21.15.97:37852) 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.888+0000 7fbc3b400700 1 -- 172.21.15.97:0/548812004 learned_addr learned my addr 172.21.15.97:0/548812004 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.889+0000 7fbc3b400700 1 -- 172.21.15.97:0/548812004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c19e410 con 0x7fbc3c1066d0 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.889+0000 7fbc3b400700 1 --2- 172.21.15.97:0/548812004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c19ded0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0072a0 tx=0x7fbc2c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.889+0000 7fbc33e00700 1 -- 172.21.15.97:0/548812004 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc2c020020 con 0x7fbc3c1066d0 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.889+0000 7fbc33e00700 1 -- 172.21.15.97:0/548812004 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc2c003b50 con 0x7fbc3c1066d0 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.889+0000 7fbc42400700 1 -- 172.21.15.97:0/548812004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c19e6a0 con 0x7fbc3c1066d0 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.889+0000 7fbc33e00700 1 -- 172.21.15.97:0/548812004 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc2c017040 con 0x7fbc3c1066d0 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.889+0000 7fbc42400700 1 -- 172.21.15.97:0/548812004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c1a1390 con 0x7fbc3c1066d0 2024-03-26T18:48:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.890+0000 7fbc33e00700 1 -- 172.21.15.97:0/548812004 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7fbc2c023020 con 0x7fbc3c1066d0 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.890+0000 7fbc42400700 1 -- 172.21.15.97:0/548812004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3c0639e0 con 0x7fbc3c1066d0 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.891+0000 7fbc33e00700 1 --2- 172.21.15.97:0/548812004 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fbc20039130 0x7fbc2003b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.891+0000 7fbc33e00700 1 -- 172.21.15.97:0/548812004 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbc2c00f050 con 0x7fbc3c1066d0 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.891+0000 7fbc3aa00700 1 --2- 172.21.15.97:0/548812004 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fbc20039130 0x7fbc2003b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.892+0000 7fbc3aa00700 1 --2- 172.21.15.97:0/548812004 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fbc20039130 0x7fbc2003b5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc28006fd0 tx=0x7fbc28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:02.896+0000 7fbc33e00700 1 -- 172.21.15.97:0/548812004 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbc2c02a020 con 0x7fbc3c1066d0 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.117+0000 7fbc42400700 1 -- 172.21.15.97:0/548812004 --> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fbc3c02dfe0 con 0x7fbc20039130 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.118+0000 7fbc33e00700 1 -- 172.21.15.97:0/548812004 <== mgr.14120 v2:172.21.15.97:6800/3462603610 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fbc3c02dfe0 con 0x7fbc20039130 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.121+0000 7fbc42400700 1 -- 172.21.15.97:0/548812004 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fbc20039130 msgr2=0x7fbc2003b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.121+0000 7fbc42400700 1 --2- 172.21.15.97:0/548812004 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fbc20039130 0x7fbc2003b5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc28006fd0 tx=0x7fbc28006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.121+0000 7fbc42400700 1 -- 172.21.15.97:0/548812004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 msgr2=0x7fbc3c19ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.121+0000 7fbc42400700 1 --2- 172.21.15.97:0/548812004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c19ded0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0072a0 tx=0x7fbc2c00bd50 comp rx=0 tx=0).stop 2024-03-26T18:48:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.121+0000 7fbc42400700 1 -- 172.21.15.97:0/548812004 shutdown_connections 2024-03-26T18:48:03.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.121+0000 7fbc42400700 1 --2- 172.21.15.97:0/548812004 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fbc20039130 0x7fbc2003b5f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:03.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.121+0000 7fbc42400700 1 --2- 172.21.15.97:0/548812004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c1066d0 0x7fbc3c19ded0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:03.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.121+0000 7fbc42400700 1 -- 172.21.15.97:0/548812004 >> 172.21.15.97:0/548812004 conn(0x7fbc3c102350 msgr2=0x7fbc3c194b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:03.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.122+0000 7fbc42400700 1 -- 172.21.15.97:0/548812004 shutdown_connections 2024-03-26T18:48:03.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:03.122+0000 7fbc42400700 1 -- 172.21.15.97:0/548812004 wait complete. 2024-03-26T18:48:03.825 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-26T18:48:03.825 INFO:teuthology.orchestra.run.smithi097.stdout:Adding key to root@localhost authorized_keys... 2024-03-26T18:48:03.826 INFO:teuthology.orchestra.run.smithi097.stdout:Adding host smithi097... 2024-03-26T18:48:04.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:04 smithi097 ceph-mon[31955]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:05.336 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:05 smithi097 ceph-mon[31955]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:06.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:06 smithi097 ceph-mon[31955]: Deploying cephadm binary to smithi097 2024-03-26T18:48:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Added host 'smithi097' with addr '172.21.15.97' 2024-03-26T18:48:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.451+0000 7fa177c00700 1 Processor -- start 2024-03-26T18:48:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.451+0000 7fa177c00700 1 -- start start 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.452+0000 7fa177c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17010a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.452+0000 7fa177c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa17010aad0 con 0x7fa17010a0e0 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.452+0000 7fa175200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17010a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.452+0000 7fa175200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17010a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37854/0 (socket says 172.21.15.97:37854) 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.452+0000 7fa175200700 1 -- 172.21.15.97:0/929024785 learned_addr learned my addr 172.21.15.97:0/929024785 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.453+0000 7fa175200700 1 -- 172.21.15.97:0/929024785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17010b1f0 con 0x7fa17010a0e0 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.453+0000 7fa175200700 1 --2- 172.21.15.97:0/929024785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17010a500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa1640098a0 tx=0x7fa164009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36e80d609e218d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.454+0000 7fa16f400700 1 -- 172.21.15.97:0/929024785 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa164004160 con 0x7fa17010a0e0 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.454+0000 7fa16f400700 1 -- 172.21.15.97:0/929024785 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa164004300 con 0x7fa17010a0e0 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.454+0000 7fa16f400700 1 -- 172.21.15.97:0/929024785 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa164011b50 con 0x7fa17010a0e0 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.454+0000 7fa177c00700 1 -- 172.21.15.97:0/929024785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 msgr2=0x7fa17010a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.454+0000 7fa177c00700 1 --2- 172.21.15.97:0/929024785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17010a500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa1640098a0 tx=0x7fa164009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.454+0000 7fa177c00700 1 -- 172.21.15.97:0/929024785 shutdown_connections 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.455+0000 7fa177c00700 1 --2- 172.21.15.97:0/929024785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17010a500 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.455+0000 7fa177c00700 1 -- 172.21.15.97:0/929024785 >> 172.21.15.97:0/929024785 conn(0x7fa17007d6a0 msgr2=0x7fa17007dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.455+0000 7fa177c00700 1 -- 172.21.15.97:0/929024785 shutdown_connections 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.455+0000 7fa177c00700 1 -- 172.21.15.97:0/929024785 wait complete. 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.456+0000 7fa177c00700 1 Processor -- start 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.456+0000 7fa177c00700 1 -- start start 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.456+0000 7fa177c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.456+0000 7fa177c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa17010e5a0 con 0x7fa17010a0e0 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.456+0000 7fa175200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.456+0000 7fa175200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17019dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37860/0 (socket says 172.21.15.97:37860) 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.456+0000 7fa175200700 1 -- 172.21.15.97:0/2521633892 learned_addr learned my addr 172.21.15.97:0/2521633892 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.457+0000 7fa175200700 1 -- 172.21.15.97:0/2521633892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17019e4e0 con 0x7fa17010a0e0 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.457+0000 7fa175200700 1 --2- 172.21.15.97:0/2521633892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17019dfa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa164011b60 tx=0x7fa164011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.457+0000 7fa16d600700 1 -- 172.21.15.97:0/2521633892 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa164003b20 con 0x7fa17010a0e0 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.457+0000 7fa16d600700 1 -- 172.21.15.97:0/2521633892 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa164003cc0 con 0x7fa17010a0e0 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.457+0000 7fa177c00700 1 -- 172.21.15.97:0/2521633892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17019e770 con 0x7fa17010a0e0 2024-03-26T18:48:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.457+0000 7fa16d600700 1 -- 172.21.15.97:0/2521633892 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa164024840 con 0x7fa17010a0e0 2024-03-26T18:48:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.458+0000 7fa177c00700 1 -- 172.21.15.97:0/2521633892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1701a1460 con 0x7fa17010a0e0 2024-03-26T18:48:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.459+0000 7fa16d600700 1 -- 172.21.15.97:0/2521633892 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7fa164022020 con 0x7fa17010a0e0 2024-03-26T18:48:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.459+0000 7fa177c00700 1 -- 172.21.15.97:0/2521633892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1700639e0 con 0x7fa17010a0e0 2024-03-26T18:48:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.459+0000 7fa16d600700 1 --2- 172.21.15.97:0/2521633892 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fa1580391d0 0x7fa15803b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.459+0000 7fa16d600700 1 -- 172.21.15.97:0/2521633892 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa164011e00 con 0x7fa17010a0e0 2024-03-26T18:48:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.460+0000 7fa16fe00700 1 --2- 172.21.15.97:0/2521633892 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fa1580391d0 0x7fa15803b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.460+0000 7fa16fe00700 1 --2- 172.21.15.97:0/2521633892 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fa1580391d0 0x7fa15803b690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa160006fd0 tx=0x7fa160006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.465+0000 7fa16d600700 1 -- 172.21.15.97:0/2521633892 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa16404ccd0 con 0x7fa17010a0e0 2024-03-26T18:48:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:04.691+0000 7fa177c00700 1 -- 172.21.15.97:0/2521633892 --> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}) -- 0x7fa170108470 con 0x7fa1580391d0 2024-03-26T18:48:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.943+0000 7fa16d600700 1 -- 172.21.15.97:0/2521633892 <== mgr.14120 v2:172.21.15.97:6800/3462603610 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa170108470 con 0x7fa1580391d0 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.947+0000 7fa177c00700 1 -- 172.21.15.97:0/2521633892 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fa1580391d0 msgr2=0x7fa15803b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.948+0000 7fa177c00700 1 --2- 172.21.15.97:0/2521633892 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fa1580391d0 0x7fa15803b690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa160006fd0 tx=0x7fa160006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.948+0000 7fa177c00700 1 -- 172.21.15.97:0/2521633892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 msgr2=0x7fa17019dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.948+0000 7fa177c00700 1 --2- 172.21.15.97:0/2521633892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17019dfa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa164011b60 tx=0x7fa164011c40 comp rx=0 tx=0).stop 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.948+0000 7fa177c00700 1 -- 172.21.15.97:0/2521633892 shutdown_connections 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.948+0000 7fa177c00700 1 --2- 172.21.15.97:0/2521633892 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fa1580391d0 0x7fa15803b690 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.948+0000 7fa177c00700 1 --2- 172.21.15.97:0/2521633892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17010a0e0 0x7fa17019dfa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.948+0000 7fa177c00700 1 -- 172.21.15.97:0/2521633892 >> 172.21.15.97:0/2521633892 conn(0x7fa17007d6a0 msgr2=0x7fa170107d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.948+0000 7fa177c00700 1 -- 172.21.15.97:0/2521633892 shutdown_connections 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:10.948+0000 7fa177c00700 1 -- 172.21.15.97:0/2521633892 wait complete. 2024-03-26T18:48:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mon service... 2024-03-26T18:48:12.195 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:11 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:48:12.196 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:11 smithi097 ceph-mon[31955]: Added host smithi097 2024-03-26T18:48:12.196 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:11 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:48:13.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-26T18:48:13.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.371+0000 7fb9fea00700 1 Processor -- start 2024-03-26T18:48:13.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.371+0000 7fb9fea00700 1 -- start start 2024-03-26T18:48:13.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.372+0000 7fb9fea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f8106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:13.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.372+0000 7fb9fea00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f807c650 con 0x7fb9f8106560 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.372+0000 7fb9fd600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f8106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.372+0000 7fb9fd600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f8106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46064/0 (socket says 172.21.15.97:46064) 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.372+0000 7fb9fd600700 1 -- 172.21.15.97:0/2472565831 learned_addr learned my addr 172.21.15.97:0/2472565831 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.373+0000 7fb9fd600700 1 -- 172.21.15.97:0/2472565831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f8106ec0 con 0x7fb9f8106560 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.373+0000 7fb9fd600700 1 --2- 172.21.15.97:0/2472565831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f8106980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0098a0 tx=0x7fb9ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcdc798d6a061fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.373+0000 7fb9f7e00700 1 -- 172.21.15.97:0/2472565831 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9ec004160 con 0x7fb9f8106560 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.373+0000 7fb9f7e00700 1 -- 172.21.15.97:0/2472565831 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb9ec004300 con 0x7fb9f8106560 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.373+0000 7fb9f7e00700 1 -- 172.21.15.97:0/2472565831 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9ec011b50 con 0x7fb9f8106560 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.374+0000 7fb9fea00700 1 -- 172.21.15.97:0/2472565831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 msgr2=0x7fb9f8106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.374+0000 7fb9fea00700 1 --2- 172.21.15.97:0/2472565831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f8106980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0098a0 tx=0x7fb9ec009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.374+0000 7fb9fea00700 1 -- 172.21.15.97:0/2472565831 shutdown_connections 2024-03-26T18:48:13.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.374+0000 7fb9fea00700 1 --2- 172.21.15.97:0/2472565831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f8106980 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:13.513 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.374+0000 7fb9fea00700 1 -- 172.21.15.97:0/2472565831 >> 172.21.15.97:0/2472565831 conn(0x7fb9f8102160 msgr2=0x7fb9f8104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:13.513 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.374+0000 7fb9fea00700 1 -- 172.21.15.97:0/2472565831 shutdown_connections 2024-03-26T18:48:13.513 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.374+0000 7fb9fea00700 1 -- 172.21.15.97:0/2472565831 wait complete. 2024-03-26T18:48:13.513 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.376+0000 7fb9fea00700 1 Processor -- start 2024-03-26T18:48:13.513 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.376+0000 7fb9fea00700 1 -- start start 2024-03-26T18:48:13.513 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.376+0000 7fb9fea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.376+0000 7fb9fea00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f8107d60 con 0x7fb9f8106560 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.376+0000 7fb9fd600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.377+0000 7fb9fd600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46072/0 (socket says 172.21.15.97:46072) 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.377+0000 7fb9fd600700 1 -- 172.21.15.97:0/101157743 learned_addr learned my addr 172.21.15.97:0/101157743 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.377+0000 7fb9fd600700 1 -- 172.21.15.97:0/101157743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f819e320 con 0x7fb9f8106560 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.377+0000 7fb9fd600700 1 --2- 172.21.15.97:0/101157743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f819dde0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec011b60 tx=0x7fb9ec011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.377+0000 7fb9f6000700 1 -- 172.21.15.97:0/101157743 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9ec003970 con 0x7fb9f8106560 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.377+0000 7fb9f6000700 1 -- 172.21.15.97:0/101157743 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb9ec003b10 con 0x7fb9f8106560 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.377+0000 7fb9f6000700 1 -- 172.21.15.97:0/101157743 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9ec024610 con 0x7fb9f8106560 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.378+0000 7fb9fea00700 1 -- 172.21.15.97:0/101157743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f819e5b0 con 0x7fb9f8106560 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.378+0000 7fb9fea00700 1 -- 172.21.15.97:0/101157743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f81a12a0 con 0x7fb9f8106560 2024-03-26T18:48:13.514 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.379+0000 7fb9f6000700 1 -- 172.21.15.97:0/101157743 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7fb9ec01a9f0 con 0x7fb9f8106560 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.379+0000 7fb9fea00700 1 -- 172.21.15.97:0/101157743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f80639e0 con 0x7fb9f8106560 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.379+0000 7fb9f6000700 1 --2- 172.21.15.97:0/101157743 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb9e00391d0 0x7fb9e003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.379+0000 7fb9f6000700 1 -- 172.21.15.97:0/101157743 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb9ec011e00 con 0x7fb9f8106560 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.380+0000 7fb9fcc00700 1 --2- 172.21.15.97:0/101157743 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb9e00391d0 0x7fb9e003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.380+0000 7fb9fcc00700 1 --2- 172.21.15.97:0/101157743 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb9e00391d0 0x7fb9e003b690 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4006fd0 tx=0x7fb9e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.385+0000 7fb9f6000700 1 -- 172.21.15.97:0/101157743 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb9ec00edb0 con 0x7fb9f8106560 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.629+0000 7fb9fea00700 1 -- 172.21.15.97:0/101157743 --> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb9f80008d0 con 0x7fb9e00391d0 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.637+0000 7fb9f6000700 1 -- 172.21.15.97:0/101157743 <== mgr.14120 v2:172.21.15.97:6800/3462603610 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb9f80008d0 con 0x7fb9e00391d0 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.639+0000 7fb9fea00700 1 -- 172.21.15.97:0/101157743 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb9e00391d0 msgr2=0x7fb9e003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.639+0000 7fb9fea00700 1 --2- 172.21.15.97:0/101157743 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb9e00391d0 0x7fb9e003b690 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4006fd0 tx=0x7fb9e4006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.639+0000 7fb9fea00700 1 -- 172.21.15.97:0/101157743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 msgr2=0x7fb9f819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.639+0000 7fb9fea00700 1 --2- 172.21.15.97:0/101157743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f819dde0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec011b60 tx=0x7fb9ec011c40 comp rx=0 tx=0).stop 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.640+0000 7fb9fea00700 1 -- 172.21.15.97:0/101157743 shutdown_connections 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.640+0000 7fb9fea00700 1 --2- 172.21.15.97:0/101157743 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb9e00391d0 0x7fb9e003b690 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:13.515 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.640+0000 7fb9fea00700 1 --2- 172.21.15.97:0/101157743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f8106560 0x7fb9f819dde0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:13.516 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.640+0000 7fb9fea00700 1 -- 172.21.15.97:0/101157743 >> 172.21.15.97:0/101157743 conn(0x7fb9f8102160 msgr2=0x7fb9f81949c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:13.516 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.640+0000 7fb9fea00700 1 -- 172.21.15.97:0/101157743 shutdown_connections 2024-03-26T18:48:13.516 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:12.640+0000 7fb9fea00700 1 -- 172.21.15.97:0/101157743 wait complete. 2024-03-26T18:48:13.516 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mgr service... 2024-03-26T18:48:13.758 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:13 smithi097 ceph-mon[31955]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:13.758 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:13 smithi097 ceph-mon[31955]: Saving service mon spec with placement count:5 2024-03-26T18:48:13.758 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:13 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:48:15.712 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:15 smithi097 ceph-mon[31955]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:15.712 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:15 smithi097 ceph-mon[31955]: Saving service mgr spec with placement count:2 2024-03-26T18:48:15.713 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:15 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:48:15.713 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:15 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.354+0000 7ff5de800700 1 Processor -- start 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.354+0000 7ff5de800700 1 -- start start 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.355+0000 7ff5de800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d81082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.355+0000 7ff5de800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d8108800 con 0x7ff5d8107ea0 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.355+0000 7ff5d7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d81082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.355+0000 7ff5d7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d81082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46086/0 (socket says 172.21.15.97:46086) 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.355+0000 7ff5d7400700 1 -- 172.21.15.97:0/3606490606 learned_addr learned my addr 172.21.15.97:0/3606490606 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.356+0000 7ff5d7400700 1 -- 172.21.15.97:0/3606490606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d8108f60 con 0x7ff5d8107ea0 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.356+0000 7ff5d7400700 1 --2- 172.21.15.97:0/3606490606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d81082c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098a0 tx=0x7ff5c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3257246726da1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.356+0000 7ff5d6000700 1 -- 172.21.15.97:0/3606490606 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5c8004160 con 0x7ff5d8107ea0 2024-03-26T18:48:15.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.356+0000 7ff5d6000700 1 -- 172.21.15.97:0/3606490606 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5c8004300 con 0x7ff5d8107ea0 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.356+0000 7ff5d6000700 1 -- 172.21.15.97:0/3606490606 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5c8011b50 con 0x7ff5d8107ea0 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.357+0000 7ff5de800700 1 -- 172.21.15.97:0/3606490606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 msgr2=0x7ff5d81082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.357+0000 7ff5de800700 1 --2- 172.21.15.97:0/3606490606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d81082c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098a0 tx=0x7ff5c8009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.357+0000 7ff5de800700 1 -- 172.21.15.97:0/3606490606 shutdown_connections 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.357+0000 7ff5de800700 1 --2- 172.21.15.97:0/3606490606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d81082c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.357+0000 7ff5de800700 1 -- 172.21.15.97:0/3606490606 >> 172.21.15.97:0/3606490606 conn(0x7ff5d81034a0 msgr2=0x7ff5d81058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.357+0000 7ff5de800700 1 -- 172.21.15.97:0/3606490606 shutdown_connections 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.357+0000 7ff5de800700 1 -- 172.21.15.97:0/3606490606 wait complete. 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.358+0000 7ff5de800700 1 Processor -- start 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.358+0000 7ff5de800700 1 -- start start 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.359+0000 7ff5de800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d807e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.359+0000 7ff5de800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d810c220 con 0x7ff5d8107ea0 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.359+0000 7ff5d7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d807e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.359+0000 7ff5d7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d807e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46096/0 (socket says 172.21.15.97:46096) 2024-03-26T18:48:15.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.359+0000 7ff5d7400700 1 -- 172.21.15.97:0/12693357 learned_addr learned my addr 172.21.15.97:0/12693357 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.359+0000 7ff5d7400700 1 -- 172.21.15.97:0/12693357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d807e920 con 0x7ff5d8107ea0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.359+0000 7ff5d7400700 1 --2- 172.21.15.97:0/12693357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d807e3e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8011b60 tx=0x7ff5c8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.360+0000 7ff5cfe00700 1 -- 172.21.15.97:0/12693357 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5c8003b20 con 0x7ff5d8107ea0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.360+0000 7ff5de800700 1 -- 172.21.15.97:0/12693357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d807cb10 con 0x7ff5d8107ea0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.360+0000 7ff5cfe00700 1 -- 172.21.15.97:0/12693357 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5c8003cc0 con 0x7ff5d8107ea0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.360+0000 7ff5cfe00700 1 -- 172.21.15.97:0/12693357 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5c8024840 con 0x7ff5d8107ea0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.360+0000 7ff5de800700 1 -- 172.21.15.97:0/12693357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d807d050 con 0x7ff5d8107ea0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.361+0000 7ff5cfe00700 1 -- 172.21.15.97:0/12693357 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7ff5c8022020 con 0x7ff5d8107ea0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.361+0000 7ff5de800700 1 -- 172.21.15.97:0/12693357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d80639e0 con 0x7ff5d8107ea0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.361+0000 7ff5cfe00700 1 --2- 172.21.15.97:0/12693357 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7ff5bc0391d0 0x7ff5bc03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.362+0000 7ff5cfe00700 1 -- 172.21.15.97:0/12693357 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff5c8011e00 con 0x7ff5d8107ea0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.362+0000 7ff5d6a00700 1 --2- 172.21.15.97:0/12693357 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7ff5bc0391d0 0x7ff5bc03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:15.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.363+0000 7ff5d6a00700 1 --2- 172.21.15.97:0/12693357 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7ff5bc0391d0 0x7ff5bc03b690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4006fd0 tx=0x7ff5c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.367+0000 7ff5cfe00700 1 -- 172.21.15.97:0/12693357 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff5c804ccd0 con 0x7ff5d8107ea0 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.605+0000 7ff5de800700 1 -- 172.21.15.97:0/12693357 --> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff5d8104680 con 0x7ff5bc0391d0 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.614+0000 7ff5cfe00700 1 -- 172.21.15.97:0/12693357 <== mgr.14120 v2:172.21.15.97:6800/3462603610 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff5d8104680 con 0x7ff5bc0391d0 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.617+0000 7ff5de800700 1 -- 172.21.15.97:0/12693357 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7ff5bc0391d0 msgr2=0x7ff5bc03b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.617+0000 7ff5de800700 1 --2- 172.21.15.97:0/12693357 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7ff5bc0391d0 0x7ff5bc03b690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4006fd0 tx=0x7ff5c4006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.617+0000 7ff5de800700 1 -- 172.21.15.97:0/12693357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 msgr2=0x7ff5d807e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.617+0000 7ff5de800700 1 --2- 172.21.15.97:0/12693357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d807e3e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8011b60 tx=0x7ff5c8011c40 comp rx=0 tx=0).stop 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.617+0000 7ff5de800700 1 -- 172.21.15.97:0/12693357 shutdown_connections 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.617+0000 7ff5de800700 1 --2- 172.21.15.97:0/12693357 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7ff5bc0391d0 0x7ff5bc03b690 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.617+0000 7ff5de800700 1 --2- 172.21.15.97:0/12693357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d8107ea0 0x7ff5d807e3e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.617+0000 7ff5de800700 1 -- 172.21.15.97:0/12693357 >> 172.21.15.97:0/12693357 conn(0x7ff5d81034a0 msgr2=0x7ff5d8103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.617+0000 7ff5de800700 1 -- 172.21.15.97:0/12693357 shutdown_connections 2024-03-26T18:48:15.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:14.617+0000 7ff5de800700 1 -- 172.21.15.97:0/12693357 wait complete. 2024-03-26T18:48:16.984 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:16 smithi097 ceph-mon[31955]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:48:16.984 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:16 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:48:16.984 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:16 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/986551153' entity='client.admin' 2024-03-26T18:48:17.305 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.361+0000 7fc874c00700 1 Processor -- start 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.361+0000 7fc874c00700 1 -- start start 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.362+0000 7fc874c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87010ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.362+0000 7fc874c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc87007bdd0 con 0x7fc870108980 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.362+0000 7fc86de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87010ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.362+0000 7fc86de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87010ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46110/0 (socket says 172.21.15.97:46110) 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.362+0000 7fc86de00700 1 -- 172.21.15.97:0/180523379 learned_addr learned my addr 172.21.15.97:0/180523379 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.363+0000 7fc86de00700 1 -- 172.21.15.97:0/180523379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87010b2b0 con 0x7fc870108980 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.363+0000 7fc86de00700 1 --2- 172.21.15.97:0/180523379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87010ad70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc8600098a0 tx=0x7fc860009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2510488d0bd5c913 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.363+0000 7fc86ca00700 1 -- 172.21.15.97:0/180523379 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc860004160 con 0x7fc870108980 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.363+0000 7fc86ca00700 1 -- 172.21.15.97:0/180523379 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc860004300 con 0x7fc870108980 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.364+0000 7fc86ca00700 1 -- 172.21.15.97:0/180523379 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc860011b20 con 0x7fc870108980 2024-03-26T18:48:17.306 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.364+0000 7fc874c00700 1 -- 172.21.15.97:0/180523379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 msgr2=0x7fc87010ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.364+0000 7fc874c00700 1 --2- 172.21.15.97:0/180523379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87010ad70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc8600098a0 tx=0x7fc860009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.365+0000 7fc874c00700 1 -- 172.21.15.97:0/180523379 shutdown_connections 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.365+0000 7fc874c00700 1 --2- 172.21.15.97:0/180523379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87010ad70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.365+0000 7fc874c00700 1 -- 172.21.15.97:0/180523379 >> 172.21.15.97:0/180523379 conn(0x7fc870102310 msgr2=0x7fc870104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.365+0000 7fc874c00700 1 -- 172.21.15.97:0/180523379 shutdown_connections 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.365+0000 7fc874c00700 1 -- 172.21.15.97:0/180523379 wait complete. 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.366+0000 7fc874c00700 1 Processor -- start 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.366+0000 7fc874c00700 1 -- start start 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.366+0000 7fc874c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.366+0000 7fc874c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc87010c260 con 0x7fc870108980 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.366+0000 7fc86de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.366+0000 7fc86de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87019df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46122/0 (socket says 172.21.15.97:46122) 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.366+0000 7fc86de00700 1 -- 172.21.15.97:0/986551153 learned_addr learned my addr 172.21.15.97:0/986551153 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.367+0000 7fc86de00700 1 -- 172.21.15.97:0/986551153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87019e480 con 0x7fc870108980 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.367+0000 7fc86de00700 1 --2- 172.21.15.97:0/986551153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87019df40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc8600117e0 tx=0x7fc860011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:17.307 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.367+0000 7fc866a00700 1 -- 172.21.15.97:0/986551153 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc860003ba0 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.367+0000 7fc866a00700 1 -- 172.21.15.97:0/986551153 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86001a910 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.367+0000 7fc866a00700 1 -- 172.21.15.97:0/986551153 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8600248f0 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.367+0000 7fc874c00700 1 -- 172.21.15.97:0/986551153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87019e710 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.368+0000 7fc874c00700 1 -- 172.21.15.97:0/986551153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87019eb70 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.369+0000 7fc874c00700 1 -- 172.21.15.97:0/986551153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8700639e0 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.369+0000 7fc866a00700 1 -- 172.21.15.97:0/986551153 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7fc860022020 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.369+0000 7fc866a00700 1 --2- 172.21.15.97:0/986551153 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fc8540391d0 0x7fc85403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.370+0000 7fc866a00700 1 -- 172.21.15.97:0/986551153 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc86004d590 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.370+0000 7fc86d400700 1 --2- 172.21.15.97:0/986551153 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fc8540391d0 0x7fc85403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.371+0000 7fc86d400700 1 --2- 172.21.15.97:0/986551153 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fc8540391d0 0x7fc85403b690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc85c006fd0 tx=0x7fc85c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.375+0000 7fc866a00700 1 -- 172.21.15.97:0/986551153 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc86000b3e0 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.613+0000 7fc874c00700 1 -- 172.21.15.97:0/986551153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fc8701a1950 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.618+0000 7fc866a00700 1 -- 172.21.15.97:0/986551153 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fc86001ab70 con 0x7fc870108980 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.621+0000 7fc874c00700 1 -- 172.21.15.97:0/986551153 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fc8540391d0 msgr2=0x7fc85403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.621+0000 7fc874c00700 1 --2- 172.21.15.97:0/986551153 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fc8540391d0 0x7fc85403b690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc85c006fd0 tx=0x7fc85c006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.621+0000 7fc874c00700 1 -- 172.21.15.97:0/986551153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 msgr2=0x7fc87019df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:17.309 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.621+0000 7fc874c00700 1 --2- 172.21.15.97:0/986551153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87019df40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc8600117e0 tx=0x7fc860011ba0 comp rx=0 tx=0).stop 2024-03-26T18:48:17.309 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.621+0000 7fc874c00700 1 -- 172.21.15.97:0/986551153 shutdown_connections 2024-03-26T18:48:17.309 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.621+0000 7fc874c00700 1 --2- 172.21.15.97:0/986551153 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fc8540391d0 0x7fc85403b690 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:17.309 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.621+0000 7fc874c00700 1 --2- 172.21.15.97:0/986551153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc870108980 0x7fc87019df40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:17.309 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.621+0000 7fc874c00700 1 -- 172.21.15.97:0/986551153 >> 172.21.15.97:0/986551153 conn(0x7fc870102310 msgr2=0x7fc870102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:17.309 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.622+0000 7fc874c00700 1 -- 172.21.15.97:0/986551153 shutdown_connections 2024-03-26T18:48:17.309 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:16.622+0000 7fc874c00700 1 -- 172.21.15.97:0/986551153 wait complete. 2024-03-26T18:48:19.038 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:18 smithi097 ceph-mon[31955]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:48:19.038 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:18 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3259480536' entity='client.admin' 2024-03-26T18:48:19.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.941+0000 7f006b400700 1 Processor -- start 2024-03-26T18:48:19.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.941+0000 7f006b400700 1 -- start start 2024-03-26T18:48:19.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.941+0000 7f006b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006410a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:19.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.941+0000 7f006b400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f006410aad0 con 0x7f006410a0e0 2024-03-26T18:48:19.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.941+0000 7f0068a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006410a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:19.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.942+0000 7f0068a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006410a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46126/0 (socket says 172.21.15.97:46126) 2024-03-26T18:48:19.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.942+0000 7f0068a00700 1 -- 172.21.15.97:0/1152798785 learned_addr learned my addr 172.21.15.97:0/1152798785 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:19.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.942+0000 7f0068a00700 1 -- 172.21.15.97:0/1152798785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006410b1f0 con 0x7f006410a0e0 2024-03-26T18:48:19.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.942+0000 7f0068a00700 1 --2- 172.21.15.97:0/1152798785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006410a500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f00580098a0 tx=0x7f0058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2923de51bf22a58e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:19.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.942+0000 7f0063400700 1 -- 172.21.15.97:0/1152798785 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0058004160 con 0x7f006410a0e0 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.942+0000 7f0063400700 1 -- 172.21.15.97:0/1152798785 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0058004300 con 0x7f006410a0e0 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.943+0000 7f0063400700 1 -- 172.21.15.97:0/1152798785 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0058011b20 con 0x7f006410a0e0 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.943+0000 7f006b400700 1 -- 172.21.15.97:0/1152798785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 msgr2=0x7f006410a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.943+0000 7f006b400700 1 --2- 172.21.15.97:0/1152798785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006410a500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f00580098a0 tx=0x7f0058009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.943+0000 7f006b400700 1 -- 172.21.15.97:0/1152798785 shutdown_connections 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.943+0000 7f006b400700 1 --2- 172.21.15.97:0/1152798785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006410a500 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.943+0000 7f006b400700 1 -- 172.21.15.97:0/1152798785 >> 172.21.15.97:0/1152798785 conn(0x7f006407d6a0 msgr2=0x7f006407dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.943+0000 7f006b400700 1 -- 172.21.15.97:0/1152798785 shutdown_connections 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.943+0000 7f006b400700 1 -- 172.21.15.97:0/1152798785 wait complete. 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.944+0000 7f006b400700 1 Processor -- start 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.945+0000 7f006b400700 1 -- start start 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.945+0000 7f006b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.945+0000 7f006b400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f006410e5a0 con 0x7f006410a0e0 2024-03-26T18:48:19.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.945+0000 7f0068a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.945+0000 7f0068a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006419df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46142/0 (socket says 172.21.15.97:46142) 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.945+0000 7f0068a00700 1 -- 172.21.15.97:0/3259480536 learned_addr learned my addr 172.21.15.97:0/3259480536 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.946+0000 7f0068a00700 1 -- 172.21.15.97:0/3259480536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006419e490 con 0x7f006410a0e0 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.946+0000 7f0068a00700 1 --2- 172.21.15.97:0/3259480536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006419df50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00580117e0 tx=0x7f0058011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.946+0000 7f0061600700 1 -- 172.21.15.97:0/3259480536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0058003ba0 con 0x7f006410a0e0 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.946+0000 7f0061600700 1 -- 172.21.15.97:0/3259480536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005801a910 con 0x7f006410a0e0 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.946+0000 7f006b400700 1 -- 172.21.15.97:0/3259480536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006419e720 con 0x7f006410a0e0 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.946+0000 7f0061600700 1 -- 172.21.15.97:0/3259480536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00580248f0 con 0x7f006410a0e0 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.946+0000 7f006b400700 1 -- 172.21.15.97:0/3259480536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006419eb80 con 0x7f006410a0e0 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.947+0000 7f0061600700 1 -- 172.21.15.97:0/3259480536 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f0058022020 con 0x7f006410a0e0 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.948+0000 7f006b400700 1 -- 172.21.15.97:0/3259480536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00640639e0 con 0x7f006410a0e0 2024-03-26T18:48:19.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.948+0000 7f0061600700 1 --2- 172.21.15.97:0/3259480536 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f004c039180 0x7f004c03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.948+0000 7f0061600700 1 -- 172.21.15.97:0/3259480536 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f005804da00 con 0x7f006410a0e0 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.948+0000 7f0063e00700 1 --2- 172.21.15.97:0/3259480536 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f004c039180 0x7f004c03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.949+0000 7f0063e00700 1 --2- 172.21.15.97:0/3259480536 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f004c039180 0x7f004c03b640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0050006fd0 tx=0x7f0050006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:17.953+0000 7f0061600700 1 -- 172.21.15.97:0/3259480536 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f005802ad60 con 0x7f006410a0e0 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.178+0000 7f006b400700 1 -- 172.21.15.97:0/3259480536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f00641a1550 con 0x7f006410a0e0 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.182+0000 7f0061600700 1 -- 172.21.15.97:0/3259480536 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f005802d3f0 con 0x7f006410a0e0 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.185+0000 7f006b400700 1 -- 172.21.15.97:0/3259480536 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f004c039180 msgr2=0x7f004c03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.185+0000 7f006b400700 1 --2- 172.21.15.97:0/3259480536 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f004c039180 0x7f004c03b640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0050006fd0 tx=0x7f0050006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.185+0000 7f006b400700 1 -- 172.21.15.97:0/3259480536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 msgr2=0x7f006419df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.185+0000 7f006b400700 1 --2- 172.21.15.97:0/3259480536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006419df50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00580117e0 tx=0x7f0058011ba0 comp rx=0 tx=0).stop 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.185+0000 7f006b400700 1 -- 172.21.15.97:0/3259480536 shutdown_connections 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.185+0000 7f006b400700 1 --2- 172.21.15.97:0/3259480536 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f004c039180 0x7f004c03b640 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.185+0000 7f006b400700 1 --2- 172.21.15.97:0/3259480536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f006410a0e0 0x7f006419df50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:19.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.185+0000 7f006b400700 1 -- 172.21.15.97:0/3259480536 >> 172.21.15.97:0/3259480536 conn(0x7f006407d6a0 msgr2=0x7f0064107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:19.055 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.185+0000 7f006b400700 1 -- 172.21.15.97:0/3259480536 shutdown_connections 2024-03-26T18:48:19.055 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:18.185+0000 7f006b400700 1 -- 172.21.15.97:0/3259480536 wait complete. 2024-03-26T18:48:19.055 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling the dashboard module... 2024-03-26T18:48:20.328 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:19 smithi097 ceph-mon[31955]: from='mgr.14120 172.21.15.97:0/3976031464' entity='mgr.x' 2024-03-26T18:48:20.329 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:19 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3863700409' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-26T18:48:20.329 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: ignoring --setuser ceph since I am not root 2024-03-26T18:48:20.329 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: ignoring --setgroup ceph since I am not root 2024-03-26T18:48:20.717 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.595+0000 7f3026e00700 1 Processor -- start 2024-03-26T18:48:20.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.595+0000 7f3026e00700 1 -- start start 2024-03-26T18:48:20.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.595+0000 7f3026e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f302010ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:20.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.595+0000 7f3026e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f302007be10 con 0x7f3020108940 2024-03-26T18:48:20.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.596+0000 7f301fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f302010ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:20.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.596+0000 7f301fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f302010ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46146/0 (socket says 172.21.15.97:46146) 2024-03-26T18:48:20.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.596+0000 7f301fe00700 1 -- 172.21.15.97:0/3202749672 learned_addr learned my addr 172.21.15.97:0/3202749672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:20.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.596+0000 7f301fe00700 1 -- 172.21.15.97:0/3202749672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302010b270 con 0x7f3020108940 2024-03-26T18:48:20.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.596+0000 7f301fe00700 1 --2- 172.21.15.97:0/3202749672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f302010ad30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30100098a0 tx=0x7f3010009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb855b057237b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:20.723 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.596+0000 7f301ea00700 1 -- 172.21.15.97:0/3202749672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3010004160 con 0x7f3020108940 2024-03-26T18:48:20.723 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.597+0000 7f301ea00700 1 -- 172.21.15.97:0/3202749672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3010004300 con 0x7f3020108940 2024-03-26T18:48:20.723 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.597+0000 7f301ea00700 1 -- 172.21.15.97:0/3202749672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3010011b20 con 0x7f3020108940 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.597+0000 7f3026e00700 1 -- 172.21.15.97:0/3202749672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 msgr2=0x7f302010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.597+0000 7f3026e00700 1 --2- 172.21.15.97:0/3202749672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f302010ad30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30100098a0 tx=0x7f3010009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.597+0000 7f3026e00700 1 -- 172.21.15.97:0/3202749672 shutdown_connections 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.597+0000 7f3026e00700 1 --2- 172.21.15.97:0/3202749672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f302010ad30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.597+0000 7f3026e00700 1 -- 172.21.15.97:0/3202749672 >> 172.21.15.97:0/3202749672 conn(0x7f3020102310 msgr2=0x7f3020104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.597+0000 7f3026e00700 1 -- 172.21.15.97:0/3202749672 shutdown_connections 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.597+0000 7f3026e00700 1 -- 172.21.15.97:0/3202749672 wait complete. 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.598+0000 7f3026e00700 1 Processor -- start 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.599+0000 7f3026e00700 1 -- start start 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.599+0000 7f3026e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f30201081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.599+0000 7f3026e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f302010c270 con 0x7f3020108940 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.599+0000 7f301fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f30201081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.599+0000 7f301fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f30201081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46156/0 (socket says 172.21.15.97:46156) 2024-03-26T18:48:20.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.599+0000 7f301fe00700 1 -- 172.21.15.97:0/3863700409 learned_addr learned my addr 172.21.15.97:0/3863700409 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.599+0000 7f301fe00700 1 -- 172.21.15.97:0/3863700409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3020108700 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.600+0000 7f301fe00700 1 --2- 172.21.15.97:0/3863700409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f30201081c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f30100117e0 tx=0x7f3010011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.600+0000 7f301cc00700 1 -- 172.21.15.97:0/3863700409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3010003ba0 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.600+0000 7f301cc00700 1 -- 172.21.15.97:0/3863700409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301001a910 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.600+0000 7f3026e00700 1 -- 172.21.15.97:0/3863700409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30201068f0 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.600+0000 7f301cc00700 1 -- 172.21.15.97:0/3863700409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30100248f0 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.600+0000 7f3026e00700 1 -- 172.21.15.97:0/3863700409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3020106db0 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.601+0000 7f301cc00700 1 -- 172.21.15.97:0/3863700409 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f3010022020 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.601+0000 7f3026e00700 1 -- 172.21.15.97:0/3863700409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30200639e0 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.602+0000 7f301cc00700 1 --2- 172.21.15.97:0/3863700409 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f30040391d0 0x7f300403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.602+0000 7f301cc00700 1 -- 172.21.15.97:0/3863700409 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f301004d5a0 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.602+0000 7f301f400700 1 --2- 172.21.15.97:0/3863700409 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f30040391d0 0x7f300403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.602+0000 7f301f400700 1 --2- 172.21.15.97:0/3863700409 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f30040391d0 0x7f300403b690 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f300c006fd0 tx=0x7f300c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.607+0000 7f301cc00700 1 -- 172.21.15.97:0/3863700409 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f301001ab70 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.889+0000 7f3026e00700 1 -- 172.21.15.97:0/3863700409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f3020107030 con 0x7f3020108940 2024-03-26T18:48:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.928+0000 7f301cc00700 1 -- 172.21.15.97:0/3863700409 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f301000f2e0 con 0x7f3020108940 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.931+0000 7f301cc00700 1 -- 172.21.15.97:0/3863700409 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 9) ==== 46307+0+0 (secure 0 0 0) 0x7f301002d3f0 con 0x7f3020108940 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.933+0000 7f3026e00700 1 -- 172.21.15.97:0/3863700409 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f30040391d0 msgr2=0x7f300403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.933+0000 7f3026e00700 1 --2- 172.21.15.97:0/3863700409 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f30040391d0 0x7f300403b690 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f300c006fd0 tx=0x7f300c006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.933+0000 7f3026e00700 1 -- 172.21.15.97:0/3863700409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 msgr2=0x7f30201081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.933+0000 7f3026e00700 1 --2- 172.21.15.97:0/3863700409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f30201081c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f30100117e0 tx=0x7f3010011ba0 comp rx=0 tx=0).stop 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.933+0000 7f3026e00700 1 -- 172.21.15.97:0/3863700409 shutdown_connections 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.933+0000 7f3026e00700 1 --2- 172.21.15.97:0/3863700409 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7f30040391d0 0x7f300403b690 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.933+0000 7f3026e00700 1 --2- 172.21.15.97:0/3863700409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3020108940 0x7f30201081c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.933+0000 7f3026e00700 1 -- 172.21.15.97:0/3863700409 >> 172.21.15.97:0/3863700409 conn(0x7f3020102310 msgr2=0x7f3020102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.933+0000 7f3026e00700 1 -- 172.21.15.97:0/3863700409 shutdown_connections 2024-03-26T18:48:20.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:19.934+0000 7f3026e00700 1 -- 172.21.15.97:0/3863700409 wait complete. 2024-03-26T18:48:20.989 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:20 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3863700409' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-26T18:48:20.989 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:20 smithi097 ceph-mon[31955]: mgrmap e9: x(active, since 24s) 2024-03-26T18:48:20.990 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:20.754+0000 7f478dcf6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T18:48:20.990 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:20.886+0000 7f478dcf6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T18:48:21.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:21 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:21.692+0000 7f478dcf6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T18:48:21.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:21 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:21.812+0000 7f478dcf6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T18:48:22.226 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:21 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2758778425' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T18:48:22.226 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:21 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:21.998+0000 7f478dcf6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T18:48:22.226 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:22 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:22.123+0000 7f478dcf6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T18:48:22.226 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:22 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:22.234+0000 7f478dcf6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T18:48:22.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-26T18:48:22.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-26T18:48:22.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T18:48:22.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T18:48:22.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T18:48:22.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-26T18:48:22.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.307+0000 7fb4cfa00700 1 Processor -- start 2024-03-26T18:48:22.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.307+0000 7fb4cfa00700 1 -- start start 2024-03-26T18:48:22.358 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.307+0000 7fb4cfa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.307+0000 7fb4cfa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c807c800 con 0x7fb4c807ddc0 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.308+0000 7fb4cd000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.308+0000 7fb4cd000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46626/0 (socket says 172.21.15.97:46626) 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.308+0000 7fb4cd000700 1 -- 172.21.15.97:0/1880423364 learned_addr learned my addr 172.21.15.97:0/1880423364 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.309+0000 7fb4cd000700 1 -- 172.21.15.97:0/1880423364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c807c980 con 0x7fb4c807ddc0 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.309+0000 7fb4cd000700 1 --2- 172.21.15.97:0/1880423364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c807c2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098a0 tx=0x7fb4bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f67f6ef876af5b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.309+0000 7fb4c7400700 1 -- 172.21.15.97:0/1880423364 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4bc004160 con 0x7fb4c807ddc0 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.309+0000 7fb4c7400700 1 -- 172.21.15.97:0/1880423364 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4bc004300 con 0x7fb4c807ddc0 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.309+0000 7fb4c7400700 1 -- 172.21.15.97:0/1880423364 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4bc011b20 con 0x7fb4c807ddc0 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.310+0000 7fb4cfa00700 1 -- 172.21.15.97:0/1880423364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 msgr2=0x7fb4c807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.310+0000 7fb4cfa00700 1 --2- 172.21.15.97:0/1880423364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c807c2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098a0 tx=0x7fb4bc009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.310+0000 7fb4cfa00700 1 -- 172.21.15.97:0/1880423364 shutdown_connections 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.310+0000 7fb4cfa00700 1 --2- 172.21.15.97:0/1880423364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c807c2c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.310+0000 7fb4cfa00700 1 -- 172.21.15.97:0/1880423364 >> 172.21.15.97:0/1880423364 conn(0x7fb4c81034a0 msgr2=0x7fb4c81058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:22.359 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.310+0000 7fb4cfa00700 1 -- 172.21.15.97:0/1880423364 shutdown_connections 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.310+0000 7fb4cfa00700 1 -- 172.21.15.97:0/1880423364 wait complete. 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.311+0000 7fb4cfa00700 1 Processor -- start 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.311+0000 7fb4cfa00700 1 -- start start 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.312+0000 7fb4cfa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.312+0000 7fb4cfa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810e400 con 0x7fb4c807ddc0 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.312+0000 7fb4cd000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.312+0000 7fb4cd000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46642/0 (socket says 172.21.15.97:46642) 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.312+0000 7fb4cd000700 1 -- 172.21.15.97:0/2758778425 learned_addr learned my addr 172.21.15.97:0/2758778425 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.312+0000 7fb4cd000700 1 -- 172.21.15.97:0/2758778425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c819e3f0 con 0x7fb4c807ddc0 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.313+0000 7fb4cd000700 1 --2- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c819deb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0117e0 tx=0x7fb4bc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.313+0000 7fb4c5600700 1 -- 172.21.15.97:0/2758778425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4bc003ba0 con 0x7fb4c807ddc0 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.313+0000 7fb4c5600700 1 -- 172.21.15.97:0/2758778425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4bc01a910 con 0x7fb4c807ddc0 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.313+0000 7fb4cfa00700 1 -- 172.21.15.97:0/2758778425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c819e680 con 0x7fb4c807ddc0 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.313+0000 7fb4c5600700 1 -- 172.21.15.97:0/2758778425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4bc0248f0 con 0x7fb4c807ddc0 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.313+0000 7fb4cfa00700 1 -- 172.21.15.97:0/2758778425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c819eae0 con 0x7fb4c807ddc0 2024-03-26T18:48:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.314+0000 7fb4cfa00700 1 -- 172.21.15.97:0/2758778425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c80639e0 con 0x7fb4c807ddc0 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.314+0000 7fb4c5600700 1 -- 172.21.15.97:0/2758778425 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) ==== 46307+0+0 (secure 0 0 0) 0x7fb4bc022020 con 0x7fb4c807ddc0 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.315+0000 7fb4c5600700 1 --2- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb4b00391d0 0x7fb4b003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.315+0000 7fb4c5600700 1 -- 172.21.15.97:0/2758778425 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb4bc04d710 con 0x7fb4c807ddc0 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.315+0000 7fb4c7e00700 1 -- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb4b00391d0 msgr2=0x7fb4b003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3462603610 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.315+0000 7fb4c7e00700 1 --2- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb4b00391d0 0x7fb4b003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.321+0000 7fb4c5600700 1 -- 172.21.15.97:0/2758778425 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb4bc04d470 con 0x7fb4c807ddc0 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.516+0000 7fb4c7e00700 1 -- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb4b00391d0 msgr2=0x7fb4b003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3462603610 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.516+0000 7fb4c7e00700 1 --2- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb4b00391d0 0x7fb4b003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.598+0000 7fb4cfa00700 1 -- 172.21.15.97:0/2758778425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb4c81a1490 con 0x7fb4c807ddc0 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.598+0000 7fb4c5600700 1 -- 172.21.15.97:0/2758778425 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb4bc02ad60 con 0x7fb4c807ddc0 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.601+0000 7fb4cfa00700 1 -- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb4b00391d0 msgr2=0x7fb4b003b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.601+0000 7fb4cfa00700 1 --2- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb4b00391d0 0x7fb4b003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.601+0000 7fb4cfa00700 1 -- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 msgr2=0x7fb4c819deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.601+0000 7fb4cfa00700 1 --2- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c819deb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0117e0 tx=0x7fb4bc011ba0 comp rx=0 tx=0).stop 2024-03-26T18:48:22.361 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.601+0000 7fb4cfa00700 1 -- 172.21.15.97:0/2758778425 shutdown_connections 2024-03-26T18:48:22.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.601+0000 7fb4cfa00700 1 --2- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7fb4b00391d0 0x7fb4b003b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:22.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.601+0000 7fb4cfa00700 1 --2- 172.21.15.97:0/2758778425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4c807ddc0 0x7fb4c819deb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:22.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.601+0000 7fb4cfa00700 1 -- 172.21.15.97:0/2758778425 >> 172.21.15.97:0/2758778425 conn(0x7fb4c81034a0 msgr2=0x7fb4c8103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:22.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.602+0000 7fb4cfa00700 1 -- 172.21.15.97:0/2758778425 shutdown_connections 2024-03-26T18:48:22.362 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:21.602+0000 7fb4cfa00700 1 -- 172.21.15.97:0/2758778425 wait complete. 2024-03-26T18:48:22.362 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2024-03-26T18:48:22.362 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 9... 2024-03-26T18:48:22.826 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:22 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:22.576+0000 7f478dcf6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T18:48:22.826 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:22 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:22.718+0000 7f478dcf6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T18:48:22.826 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:22 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:22.834+0000 7f478dcf6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T18:48:23.094 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:22 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:22.961+0000 7f478dcf6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T18:48:23.351 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:23 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:23.180+0000 7f478dcf6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T18:48:24.599 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:24.437+0000 7f478dcf6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T18:48:24.599 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:24.607+0000 7f478dcf6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T18:48:25.351 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:25 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:25.027+0000 7f478dcf6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T18:48:25.351 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:25 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:25.144+0000 7f478dcf6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T18:48:25.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:25 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:25.400+0000 7f478dcf6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:48:25.851 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:25 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:25.645+0000 7f478dcf6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T18:48:26.336 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:26.023+0000 7f478dcf6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:48:26.337 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:26.344+0000 7f478dcf6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T18:48:27.101 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:26.644+0000 7f478dcf6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T18:48:28.101 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:27.662+0000 7f478dcf6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T18:48:28.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:28.356+0000 7f478dcf6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T18:48:28.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:28.483+0000 7f478dcf6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T18:48:29.101 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:48:28.765+0000 7f478dcf6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T18:48:29.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-mon[31955]: Active manager daemon x restarted 2024-03-26T18:48:29.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-mon[31955]: Activating manager daemon x 2024-03-26T18:48:29.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-mon[31955]: osdmap e3: 0 total, 0 up, 0 in 2024-03-26T18:48:29.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-mon[31955]: mgrmap e10: x(active, starting, since 0.00487829s) 2024-03-26T18:48:29.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:48:29.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T18:48:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T18:48:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T18:48:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T18:48:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:28 smithi097 ceph-mon[31955]: Manager daemon x is now available 2024-03-26T18:48:30.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:48:30.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T18:48:30.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T18:48:30.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:29 smithi097 ceph-mon[31955]: mgrmap e11: x(active, since 1.01285s) 2024-03-26T18:48:30.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-26T18:48:30.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-26T18:48:30.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T18:48:30.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-26T18:48:30.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.893+0000 7febac200700 1 Processor -- start 2024-03-26T18:48:30.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.894+0000 7febac200700 1 -- start start 2024-03-26T18:48:30.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.894+0000 7febac200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:30.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.894+0000 7febac200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feba407c800 con 0x7feba407ddc0 2024-03-26T18:48:30.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.894+0000 7feba9800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:30.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.894+0000 7feba9800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46652/0 (socket says 172.21.15.97:46652) 2024-03-26T18:48:30.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.894+0000 7feba9800700 1 -- 172.21.15.97:0/791868021 learned_addr learned my addr 172.21.15.97:0/791868021 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:30.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.895+0000 7feba9800700 1 -- 172.21.15.97:0/791868021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba407c980 con 0x7feba407ddc0 2024-03-26T18:48:30.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.895+0000 7feba9800700 1 --2- 172.21.15.97:0/791868021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba407c2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb980098a0 tx=0x7feb98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72934eec7ff993a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:30.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.895+0000 7feba3e00700 1 -- 172.21.15.97:0/791868021 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb98004160 con 0x7feba407ddc0 2024-03-26T18:48:30.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.895+0000 7feba3e00700 1 -- 172.21.15.97:0/791868021 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb98004300 con 0x7feba407ddc0 2024-03-26T18:48:30.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.895+0000 7feba3e00700 1 -- 172.21.15.97:0/791868021 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb98011b20 con 0x7feba407ddc0 2024-03-26T18:48:30.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.896+0000 7febac200700 1 -- 172.21.15.97:0/791868021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 msgr2=0x7feba407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:30.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.896+0000 7febac200700 1 --2- 172.21.15.97:0/791868021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba407c2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb980098a0 tx=0x7feb98009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:30.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.896+0000 7febac200700 1 -- 172.21.15.97:0/791868021 shutdown_connections 2024-03-26T18:48:30.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.896+0000 7febac200700 1 --2- 172.21.15.97:0/791868021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba407c2c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:30.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.896+0000 7febac200700 1 -- 172.21.15.97:0/791868021 >> 172.21.15.97:0/791868021 conn(0x7feba41034a0 msgr2=0x7feba41058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:30.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.896+0000 7febac200700 1 -- 172.21.15.97:0/791868021 shutdown_connections 2024-03-26T18:48:30.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.897+0000 7febac200700 1 -- 172.21.15.97:0/791868021 wait complete. 2024-03-26T18:48:30.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.897+0000 7febac200700 1 Processor -- start 2024-03-26T18:48:30.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.898+0000 7febac200700 1 -- start start 2024-03-26T18:48:30.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.898+0000 7febac200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:30.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.898+0000 7febac200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feba410e400 con 0x7feba407ddc0 2024-03-26T18:48:30.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.898+0000 7feba9800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:30.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.898+0000 7feba9800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46656/0 (socket says 172.21.15.97:46656) 2024-03-26T18:48:30.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.898+0000 7feba9800700 1 -- 172.21.15.97:0/3505633255 learned_addr learned my addr 172.21.15.97:0/3505633255 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:30.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.899+0000 7feba9800700 1 -- 172.21.15.97:0/3505633255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba419e3f0 con 0x7feba407ddc0 2024-03-26T18:48:30.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.899+0000 7feba9800700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba419deb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb980117e0 tx=0x7feb98011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:30.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.899+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb98003ba0 con 0x7feba407ddc0 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.899+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9801a910 con 0x7feba407ddc0 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.899+0000 7febac200700 1 -- 172.21.15.97:0/3505633255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba419e680 con 0x7feba407ddc0 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.900+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb980248f0 con 0x7feba407ddc0 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.900+0000 7febac200700 1 -- 172.21.15.97:0/3505633255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba419eae0 con 0x7feba407ddc0 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.901+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) ==== 46307+0+0 (secure 0 0 0) 0x7feb98022020 con 0x7feba407ddc0 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.901+0000 7feba2000700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 0x7feb8c03b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.901+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feb9801a430 con 0x7feba407ddc0 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.901+0000 7feba8e00700 1 -- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 msgr2=0x7feb8c03b340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3462603610 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.901+0000 7feba8e00700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 0x7feb8c03b340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:22.901+0000 7febac200700 1 -- 172.21.15.97:0/3505633255 --> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feba40639e0 con 0x7feb8c038e80 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:23.102+0000 7feba8e00700 1 -- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 msgr2=0x7feb8c03b340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3462603610 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:23.102+0000 7feba8e00700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 0x7feb8c03b340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:23.503+0000 7feba8e00700 1 -- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 msgr2=0x7feb8c03b340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3462603610 2024-03-26T18:48:30.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:23.503+0000 7feba8e00700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 0x7feb8c03b340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:24.304+0000 7feba8e00700 1 -- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 msgr2=0x7feb8c03b340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3462603610 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:24.304+0000 7feba8e00700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 0x7feb8c03b340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:25.906+0000 7feba8e00700 1 -- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 msgr2=0x7feb8c03b340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3462603610 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:25.906+0000 7feba8e00700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 0x7feb8c03b340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:28.772+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mgrmap(e 10) ==== 46074+0+0 (secure 0 0 0) 0x7feb9800b3e0 con 0x7feba407ddc0 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:28.773+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 msgr2=0x7feb8c03b340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:28.773+0000 7feba2000700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 0x7feb8c03b340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.777+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 11) ==== 46201+0+0 (secure 0 0 0) 0x7feb98043390 con 0x7feba407ddc0 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.777+0000 7feba2000700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb8c03c7d0 0x7feb8c03ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.777+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feb98008ea0 con 0x7feb8c03c7d0 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.778+0000 7feba8e00700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb8c03c7d0 0x7feb8c03ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.780+0000 7feba8e00700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb8c03c7d0 0x7feb8c03ebc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7feb90003a10 tx=0x7feb900092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.781+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7feb98008ea0 con 0x7feb8c03c7d0 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.788+0000 7febac200700 1 -- 172.21.15.97:0/3505633255 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7feba41046a0 con 0x7feb8c03c7d0 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.789+0000 7feba2000700 1 -- 172.21.15.97:0/3505633255 <== mgr.14152 v2:172.21.15.97:6800/3680188982 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7feba41046a0 con 0x7feb8c03c7d0 2024-03-26T18:48:30.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.789+0000 7febac200700 1 -- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb8c03c7d0 msgr2=0x7feb8c03ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:30.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.789+0000 7febac200700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb8c03c7d0 0x7feb8c03ebc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7feb90003a10 tx=0x7feb900092b0 comp rx=0 tx=0).stop 2024-03-26T18:48:30.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.789+0000 7febac200700 1 -- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 msgr2=0x7feba419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:30.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.789+0000 7febac200700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba419deb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb980117e0 tx=0x7feb98011ba0 comp rx=0 tx=0).stop 2024-03-26T18:48:30.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.790+0000 7febac200700 1 -- 172.21.15.97:0/3505633255 shutdown_connections 2024-03-26T18:48:30.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.790+0000 7febac200700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb8c03c7d0 0x7feb8c03ebc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:30.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.790+0000 7febac200700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feba407ddc0 0x7feba419deb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:30.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.790+0000 7febac200700 1 --2- 172.21.15.97:0/3505633255 >> [v2:172.21.15.97:6800/3462603610,v1:172.21.15.97:6801/3462603610] conn(0x7feb8c038e80 0x7feb8c03b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:30.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.790+0000 7febac200700 1 -- 172.21.15.97:0/3505633255 >> 172.21.15.97:0/3505633255 conn(0x7feba41034a0 msgr2=0x7feba4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:30.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.790+0000 7febac200700 1 -- 172.21.15.97:0/3505633255 shutdown_connections 2024-03-26T18:48:30.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:29.790+0000 7febac200700 1 -- 172.21.15.97:0/3505633255 wait complete. 2024-03-26T18:48:30.469 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 9 is available 2024-03-26T18:48:30.469 INFO:teuthology.orchestra.run.smithi097.stdout:Generating a dashboard self-signed certificate... 2024-03-26T18:48:31.074 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:30 smithi097 ceph-mon[31955]: [26/Mar/2024:18:48:29] ENGINE Bus STARTING 2024-03-26T18:48:31.074 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:30 smithi097 ceph-mon[31955]: [26/Mar/2024:18:48:29] ENGINE Serving on https://172.21.15.97:7150 2024-03-26T18:48:31.074 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:30 smithi097 ceph-mon[31955]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T18:48:31.074 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:30 smithi097 ceph-mon[31955]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T18:48:31.074 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:30 smithi097 ceph-mon[31955]: [26/Mar/2024:18:48:29] ENGINE Serving on http://172.21.15.97:8765 2024-03-26T18:48:31.074 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:30 smithi097 ceph-mon[31955]: [26/Mar/2024:18:48:29] ENGINE Bus STARTED 2024-03-26T18:48:31.994 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:31 smithi097 ceph-mon[31955]: mgrmap e12: x(active, since 2s) 2024-03-26T18:48:31.994 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:31.994 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:32.324 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-26T18:48:32.325 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.082+0000 7fab87400700 1 Processor -- start 2024-03-26T18:48:32.325 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.082+0000 7fab87400700 1 -- start start 2024-03-26T18:48:32.325 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.082+0000 7fab87400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab80106b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:32.325 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.082+0000 7fab87400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab8007c680 con 0x7fab801066f0 2024-03-26T18:48:32.325 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.083+0000 7fab84a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab80106b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:32.326 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.083+0000 7fab84a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab80106b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34216/0 (socket says 172.21.15.97:34216) 2024-03-26T18:48:32.326 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.083+0000 7fab84a00700 1 -- 172.21.15.97:0/2247490941 learned_addr learned my addr 172.21.15.97:0/2247490941 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:32.326 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.083+0000 7fab84a00700 1 -- 172.21.15.97:0/2247490941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab80107050 con 0x7fab801066f0 2024-03-26T18:48:32.326 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.084+0000 7fab84a00700 1 --2- 172.21.15.97:0/2247490941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab80106b10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab740098a0 tx=0x7fab74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06efdcbabf837ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:32.326 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.084+0000 7fab7f400700 1 -- 172.21.15.97:0/2247490941 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab74004160 con 0x7fab801066f0 2024-03-26T18:48:32.328 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.084+0000 7fab7f400700 1 -- 172.21.15.97:0/2247490941 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab74004300 con 0x7fab801066f0 2024-03-26T18:48:32.328 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.084+0000 7fab7f400700 1 -- 172.21.15.97:0/2247490941 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab74011b20 con 0x7fab801066f0 2024-03-26T18:48:32.328 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.085+0000 7fab87400700 1 -- 172.21.15.97:0/2247490941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 msgr2=0x7fab80106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:32.328 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.085+0000 7fab87400700 1 --2- 172.21.15.97:0/2247490941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab80106b10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab740098a0 tx=0x7fab74009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.085+0000 7fab87400700 1 -- 172.21.15.97:0/2247490941 shutdown_connections 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.085+0000 7fab87400700 1 --2- 172.21.15.97:0/2247490941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab80106b10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.085+0000 7fab87400700 1 -- 172.21.15.97:0/2247490941 >> 172.21.15.97:0/2247490941 conn(0x7fab80102350 msgr2=0x7fab80104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.085+0000 7fab87400700 1 -- 172.21.15.97:0/2247490941 shutdown_connections 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.085+0000 7fab87400700 1 -- 172.21.15.97:0/2247490941 wait complete. 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.086+0000 7fab87400700 1 Processor -- start 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.086+0000 7fab87400700 1 -- start start 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.086+0000 7fab87400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab8019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.087+0000 7fab87400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab80108120 con 0x7fab801066f0 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.087+0000 7fab84a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab8019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.087+0000 7fab84a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab8019dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34226/0 (socket says 172.21.15.97:34226) 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.087+0000 7fab84a00700 1 -- 172.21.15.97:0/2873134370 learned_addr learned my addr 172.21.15.97:0/2873134370 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.087+0000 7fab84a00700 1 -- 172.21.15.97:0/2873134370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8019e400 con 0x7fab801066f0 2024-03-26T18:48:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.087+0000 7fab84a00700 1 --2- 172.21.15.97:0/2873134370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab8019dec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab740117e0 tx=0x7fab74011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:32.330 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.088+0000 7fab7d600700 1 -- 172.21.15.97:0/2873134370 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab74003ba0 con 0x7fab801066f0 2024-03-26T18:48:32.330 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.088+0000 7fab7d600700 1 -- 172.21.15.97:0/2873134370 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab7401a910 con 0x7fab801066f0 2024-03-26T18:48:32.332 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.088+0000 7fab7d600700 1 -- 172.21.15.97:0/2873134370 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab740248f0 con 0x7fab801066f0 2024-03-26T18:48:32.332 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.088+0000 7fab87400700 1 -- 172.21.15.97:0/2873134370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8019e690 con 0x7fab801066f0 2024-03-26T18:48:32.332 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.088+0000 7fab87400700 1 -- 172.21.15.97:0/2873134370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8019eaf0 con 0x7fab801066f0 2024-03-26T18:48:32.332 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.089+0000 7fab7d600700 1 -- 172.21.15.97:0/2873134370 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) ==== 46307+0+0 (secure 0 0 0) 0x7fab74022020 con 0x7fab801066f0 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.089+0000 7fab87400700 1 -- 172.21.15.97:0/2873134370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab800639e0 con 0x7fab801066f0 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.090+0000 7fab7d600700 1 --2- 172.21.15.97:0/2873134370 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fab68039230 0x7fab6803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.090+0000 7fab7d600700 1 -- 172.21.15.97:0/2873134370 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fab7404dc20 con 0x7fab801066f0 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.090+0000 7fab7fe00700 1 --2- 172.21.15.97:0/2873134370 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fab68039230 0x7fab6803b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.091+0000 7fab7fe00700 1 --2- 172.21.15.97:0/2873134370 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fab68039230 0x7fab6803b6f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab6c006fd0 tx=0x7fab6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.095+0000 7fab7d600700 1 -- 172.21.15.97:0/2873134370 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fab7400f2e0 con 0x7fab801066f0 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.326+0000 7fab87400700 1 -- 172.21.15.97:0/2873134370 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fab800627f0 con 0x7fab68039230 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.484+0000 7fab7d600700 1 -- 172.21.15.97:0/2873134370 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fab800627f0 con 0x7fab68039230 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.487+0000 7fab87400700 1 -- 172.21.15.97:0/2873134370 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fab68039230 msgr2=0x7fab6803b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.487+0000 7fab87400700 1 --2- 172.21.15.97:0/2873134370 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fab68039230 0x7fab6803b6f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab6c006fd0 tx=0x7fab6c006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.487+0000 7fab87400700 1 -- 172.21.15.97:0/2873134370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 msgr2=0x7fab8019dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.487+0000 7fab87400700 1 --2- 172.21.15.97:0/2873134370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab8019dec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab740117e0 tx=0x7fab74011ba0 comp rx=0 tx=0).stop 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.487+0000 7fab87400700 1 -- 172.21.15.97:0/2873134370 shutdown_connections 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.487+0000 7fab87400700 1 --2- 172.21.15.97:0/2873134370 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fab68039230 0x7fab6803b6f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.488+0000 7fab87400700 1 --2- 172.21.15.97:0/2873134370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab801066f0 0x7fab8019dec0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:32.334 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.488+0000 7fab87400700 1 -- 172.21.15.97:0/2873134370 >> 172.21.15.97:0/2873134370 conn(0x7fab80102350 msgr2=0x7fab80194b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:32.334 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.488+0000 7fab87400700 1 -- 172.21.15.97:0/2873134370 shutdown_connections 2024-03-26T18:48:32.334 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:31.488+0000 7fab87400700 1 -- 172.21.15.97:0/2873134370 wait complete. 2024-03-26T18:48:32.334 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial admin user... 2024-03-26T18:48:32.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:32 smithi097 ceph-mon[31955]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:32.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:32 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:34.392 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$40wy61O4QMAVAB8HkzeI1OGC.FOwltLwn1U5.RLVe8slW.FqXDz0u", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711478913, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-26T18:48:34.392 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.055+0000 7f3e1ac00700 1 Processor -- start 2024-03-26T18:48:34.392 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.055+0000 7f3e1ac00700 1 -- start start 2024-03-26T18:48:34.392 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.056+0000 7f3e1ac00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:34.392 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.056+0000 7f3e1ac00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1407c820 con 0x7f3e1407dde0 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.056+0000 7f3e13e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.056+0000 7f3e13e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34230/0 (socket says 172.21.15.97:34230) 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.056+0000 7f3e13e00700 1 -- 172.21.15.97:0/955958838 learned_addr learned my addr 172.21.15.97:0/955958838 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.056+0000 7f3e13e00700 1 -- 172.21.15.97:0/955958838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1407c9a0 con 0x7f3e1407dde0 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.057+0000 7f3e13e00700 1 --2- 172.21.15.97:0/955958838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1407c2e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e040098a0 tx=0x7f3e04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8024dd42e7e34bb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.057+0000 7f3e12a00700 1 -- 172.21.15.97:0/955958838 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e04004160 con 0x7f3e1407dde0 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.057+0000 7f3e12a00700 1 -- 172.21.15.97:0/955958838 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e04004300 con 0x7f3e1407dde0 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.057+0000 7f3e12a00700 1 -- 172.21.15.97:0/955958838 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e04011b20 con 0x7f3e1407dde0 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.058+0000 7f3e1ac00700 1 -- 172.21.15.97:0/955958838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 msgr2=0x7f3e1407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.058+0000 7f3e1ac00700 1 --2- 172.21.15.97:0/955958838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1407c2e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e040098a0 tx=0x7f3e04009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.058+0000 7f3e1ac00700 1 -- 172.21.15.97:0/955958838 shutdown_connections 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.058+0000 7f3e1ac00700 1 --2- 172.21.15.97:0/955958838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1407c2e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.058+0000 7f3e1ac00700 1 -- 172.21.15.97:0/955958838 >> 172.21.15.97:0/955958838 conn(0x7f3e141034a0 msgr2=0x7f3e141058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.058+0000 7f3e1ac00700 1 -- 172.21.15.97:0/955958838 shutdown_connections 2024-03-26T18:48:34.393 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.058+0000 7f3e1ac00700 1 -- 172.21.15.97:0/955958838 wait complete. 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.059+0000 7f3e1ac00700 1 Processor -- start 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.059+0000 7f3e1ac00700 1 -- start start 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.059+0000 7f3e1ac00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.060+0000 7f3e1ac00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1410e440 con 0x7f3e1407dde0 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.060+0000 7f3e13e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.060+0000 7f3e13e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34240/0 (socket says 172.21.15.97:34240) 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.060+0000 7f3e13e00700 1 -- 172.21.15.97:0/458928128 learned_addr learned my addr 172.21.15.97:0/458928128 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.060+0000 7f3e13e00700 1 -- 172.21.15.97:0/458928128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1419e3f0 con 0x7f3e1407dde0 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.060+0000 7f3e13e00700 1 --2- 172.21.15.97:0/458928128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1419deb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e040117e0 tx=0x7f3e04011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.061+0000 7f3e10c00700 1 -- 172.21.15.97:0/458928128 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e04003ba0 con 0x7f3e1407dde0 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.061+0000 7f3e10c00700 1 -- 172.21.15.97:0/458928128 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e0401a910 con 0x7f3e1407dde0 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.061+0000 7f3e1ac00700 1 -- 172.21.15.97:0/458928128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1419e680 con 0x7f3e1407dde0 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.061+0000 7f3e10c00700 1 -- 172.21.15.97:0/458928128 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e040248f0 con 0x7f3e1407dde0 2024-03-26T18:48:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.061+0000 7f3e1ac00700 1 -- 172.21.15.97:0/458928128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1419eae0 con 0x7f3e1407dde0 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.062+0000 7f3e10c00700 1 -- 172.21.15.97:0/458928128 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) ==== 46307+0+0 (secure 0 0 0) 0x7f3e04022020 con 0x7f3e1407dde0 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.062+0000 7f3e1ac00700 1 -- 172.21.15.97:0/458928128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e140639e0 con 0x7f3e1407dde0 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.063+0000 7f3e10c00700 1 --2- 172.21.15.97:0/458928128 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3df8039230 0x7f3df803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.063+0000 7f3e10c00700 1 -- 172.21.15.97:0/458928128 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e0404dbb0 con 0x7f3e1407dde0 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.063+0000 7f3e13400700 1 --2- 172.21.15.97:0/458928128 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3df8039230 0x7f3df803b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.064+0000 7f3e13400700 1 --2- 172.21.15.97:0/458928128 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3df8039230 0x7f3df803b6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3e00006fd0 tx=0x7f3e00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.072+0000 7f3e10c00700 1 -- 172.21.15.97:0/458928128 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3e0400fdd0 con 0x7f3e1407dde0 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.312+0000 7f3e1ac00700 1 -- 172.21.15.97:0/458928128 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3e141a1430 con 0x7f3df8039230 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.566+0000 7f3e10c00700 1 -- 172.21.15.97:0/458928128 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3e141a1430 con 0x7f3df8039230 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.569+0000 7f3e1ac00700 1 -- 172.21.15.97:0/458928128 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3df8039230 msgr2=0x7f3df803b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.569+0000 7f3e1ac00700 1 --2- 172.21.15.97:0/458928128 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3df8039230 0x7f3df803b6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3e00006fd0 tx=0x7f3e00006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.569+0000 7f3e1ac00700 1 -- 172.21.15.97:0/458928128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 msgr2=0x7f3e1419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.569+0000 7f3e1ac00700 1 --2- 172.21.15.97:0/458928128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1419deb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e040117e0 tx=0x7f3e04011ba0 comp rx=0 tx=0).stop 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.569+0000 7f3e1ac00700 1 -- 172.21.15.97:0/458928128 shutdown_connections 2024-03-26T18:48:34.395 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.569+0000 7f3e1ac00700 1 --2- 172.21.15.97:0/458928128 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3df8039230 0x7f3df803b6f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:34.396 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.569+0000 7f3e1ac00700 1 --2- 172.21.15.97:0/458928128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e1407dde0 0x7f3e1419deb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:34.396 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.569+0000 7f3e1ac00700 1 -- 172.21.15.97:0/458928128 >> 172.21.15.97:0/458928128 conn(0x7f3e141034a0 msgr2=0x7f3e14103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:34.396 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.569+0000 7f3e1ac00700 1 -- 172.21.15.97:0/458928128 shutdown_connections 2024-03-26T18:48:34.396 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:33.569+0000 7f3e1ac00700 1 -- 172.21.15.97:0/458928128 wait complete. 2024-03-26T18:48:34.396 INFO:teuthology.orchestra.run.smithi097.stdout:Fetching dashboard port number... 2024-03-26T18:48:34.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:34 smithi097 ceph-mon[31955]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:34.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:34.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:35.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:35 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3911322556' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-26T18:48:36.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 8443 2024-03-26T18:48:36.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.956+0000 7f8761a00700 1 Processor -- start 2024-03-26T18:48:36.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.956+0000 7f8761a00700 1 -- start start 2024-03-26T18:48:36.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.957+0000 7f8761a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c10ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.957+0000 7f8761a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f875c07be10 con 0x7f875c108940 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.957+0000 7f875aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c10ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.957+0000 7f875aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c10ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34250/0 (socket says 172.21.15.97:34250) 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.957+0000 7f875aa00700 1 -- 172.21.15.97:0/3322185715 learned_addr learned my addr 172.21.15.97:0/3322185715 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.957+0000 7f875aa00700 1 -- 172.21.15.97:0/3322185715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c10b270 con 0x7f875c108940 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.958+0000 7f875aa00700 1 --2- 172.21.15.97:0/3322185715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c10ad30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f874c0098a0 tx=0x7f874c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710af441482be8e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.958+0000 7f8759600700 1 -- 172.21.15.97:0/3322185715 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f874c004160 con 0x7f875c108940 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.958+0000 7f8759600700 1 -- 172.21.15.97:0/3322185715 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f874c004300 con 0x7f875c108940 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.958+0000 7f8759600700 1 -- 172.21.15.97:0/3322185715 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f874c011b20 con 0x7f875c108940 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.959+0000 7f8761a00700 1 -- 172.21.15.97:0/3322185715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 msgr2=0x7f875c10ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.959+0000 7f8761a00700 1 --2- 172.21.15.97:0/3322185715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c10ad30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f874c0098a0 tx=0x7f874c009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.959+0000 7f8761a00700 1 -- 172.21.15.97:0/3322185715 shutdown_connections 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.959+0000 7f8761a00700 1 --2- 172.21.15.97:0/3322185715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c10ad30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.959+0000 7f8761a00700 1 -- 172.21.15.97:0/3322185715 >> 172.21.15.97:0/3322185715 conn(0x7f875c102310 msgr2=0x7f875c104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.959+0000 7f8761a00700 1 -- 172.21.15.97:0/3322185715 shutdown_connections 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.959+0000 7f8761a00700 1 -- 172.21.15.97:0/3322185715 wait complete. 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.960+0000 7f8761a00700 1 Processor -- start 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.960+0000 7f8761a00700 1 -- start start 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.960+0000 7f8761a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c1081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.960+0000 7f8761a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f875c10c270 con 0x7f875c108940 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.961+0000 7f875aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c1081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.961+0000 7f875aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c1081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34254/0 (socket says 172.21.15.97:34254) 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.961+0000 7f875aa00700 1 -- 172.21.15.97:0/3911322556 learned_addr learned my addr 172.21.15.97:0/3911322556 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.961+0000 7f875aa00700 1 -- 172.21.15.97:0/3911322556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c108700 con 0x7f875c108940 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.961+0000 7f875aa00700 1 --2- 172.21.15.97:0/3911322556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c1081c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f874c0117e0 tx=0x7f874c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.961+0000 7f8753400700 1 -- 172.21.15.97:0/3911322556 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f874c003ba0 con 0x7f875c108940 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.962+0000 7f8761a00700 1 -- 172.21.15.97:0/3911322556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875c1068f0 con 0x7f875c108940 2024-03-26T18:48:36.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.962+0000 7f8753400700 1 -- 172.21.15.97:0/3911322556 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f874c01a910 con 0x7f875c108940 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.962+0000 7f8753400700 1 -- 172.21.15.97:0/3911322556 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f874c0248f0 con 0x7f875c108940 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.962+0000 7f8761a00700 1 -- 172.21.15.97:0/3911322556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875c106db0 con 0x7f875c108940 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.963+0000 7f8761a00700 1 -- 172.21.15.97:0/3911322556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f875c0639e0 con 0x7f875c108940 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.963+0000 7f8753400700 1 -- 172.21.15.97:0/3911322556 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) ==== 46307+0+0 (secure 0 0 0) 0x7f874c022020 con 0x7f875c108940 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.964+0000 7f8753400700 1 --2- 172.21.15.97:0/3911322556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f87400391d0 0x7f874003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.964+0000 7f8753400700 1 -- 172.21.15.97:0/3911322556 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f874c04dbd0 con 0x7f875c108940 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.964+0000 7f875a000700 1 --2- 172.21.15.97:0/3911322556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f87400391d0 0x7f874003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.965+0000 7f875a000700 1 --2- 172.21.15.97:0/3911322556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f87400391d0 0x7f874003b690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8748006fd0 tx=0x7f8748006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:34.970+0000 7f8753400700 1 -- 172.21.15.97:0/3911322556 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f874c01ab70 con 0x7f875c108940 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.207+0000 7f8761a00700 1 -- 172.21.15.97:0/3911322556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f875c107030 con 0x7f875c108940 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.208+0000 7f8753400700 1 -- 172.21.15.97:0/3911322556 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f874c00f2e0 con 0x7f875c108940 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.212+0000 7f8761a00700 1 -- 172.21.15.97:0/3911322556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f87400391d0 msgr2=0x7f874003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.212+0000 7f8761a00700 1 --2- 172.21.15.97:0/3911322556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f87400391d0 0x7f874003b690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8748006fd0 tx=0x7f8748006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:36.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.212+0000 7f8761a00700 1 -- 172.21.15.97:0/3911322556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 msgr2=0x7f875c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:36.048 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.212+0000 7f8761a00700 1 --2- 172.21.15.97:0/3911322556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c1081c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f874c0117e0 tx=0x7f874c011ba0 comp rx=0 tx=0).stop 2024-03-26T18:48:36.048 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.212+0000 7f8761a00700 1 -- 172.21.15.97:0/3911322556 shutdown_connections 2024-03-26T18:48:36.048 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.212+0000 7f8761a00700 1 --2- 172.21.15.97:0/3911322556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f87400391d0 0x7f874003b690 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:36.048 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.212+0000 7f8761a00700 1 --2- 172.21.15.97:0/3911322556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f875c108940 0x7f875c1081c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:36.048 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.212+0000 7f8761a00700 1 -- 172.21.15.97:0/3911322556 >> 172.21.15.97:0/3911322556 conn(0x7f875c102310 msgr2=0x7f875c102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:36.048 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.212+0000 7f8761a00700 1 -- 172.21.15.97:0/3911322556 shutdown_connections 2024-03-26T18:48:36.048 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:35.212+0000 7f8761a00700 1 -- 172.21.15.97:0/3911322556 wait complete. 2024-03-26T18:48:36.069 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T18:48:36.069 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-26T18:48:36.093 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T18:48:36.093 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-26T18:48:36.094 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-26T18:48:36.094 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-26T18:48:36.098 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph Dashboard is now available at: 2024-03-26T18:48:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: URL: https://smithi097.front.sepia.ceph.com:8443/ 2024-03-26T18:48:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: User: admin 2024-03-26T18:48:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: Password: fp0xteylyf 2024-03-26T18:48:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:36.099 INFO:teuthology.orchestra.run.smithi097.stdout:Saving cluster configuration to /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config directory 2024-03-26T18:48:36.100 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling autotune for osd_memory_target 2024-03-26T18:48:36.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:36 smithi097 ceph-mon[31955]: mgrmap e13: x(active, since 6s) 2024-03-26T18:48:38.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.043+0000 7fa9c2200700 1 Processor -- start 2024-03-26T18:48:38.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.043+0000 7fa9c2200700 1 -- start start 2024-03-26T18:48:38.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.043+0000 7fa9c2200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc10a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:38.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.043+0000 7fa9c2200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10aa60 con 0x7fa9bc10a070 2024-03-26T18:48:38.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.044+0000 7fa9bb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc10a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:38.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.044+0000 7fa9bb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc10a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34256/0 (socket says 172.21.15.97:34256) 2024-03-26T18:48:38.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.044+0000 7fa9bb400700 1 -- 172.21.15.97:0/1737300807 learned_addr learned my addr 172.21.15.97:0/1737300807 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:38.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.044+0000 7fa9bb400700 1 -- 172.21.15.97:0/1737300807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc10b220 con 0x7fa9bc10a070 2024-03-26T18:48:38.255 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.045+0000 7fa9bb400700 1 --2- 172.21.15.97:0/1737300807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc10a490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0098a0 tx=0x7fa9ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2016f1d604630e13 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:38.255 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.045+0000 7fa9ba000700 1 -- 172.21.15.97:0/1737300807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac004160 con 0x7fa9bc10a070 2024-03-26T18:48:38.255 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.045+0000 7fa9ba000700 1 -- 172.21.15.97:0/1737300807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac004300 con 0x7fa9bc10a070 2024-03-26T18:48:38.255 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.045+0000 7fa9ba000700 1 -- 172.21.15.97:0/1737300807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac011b20 con 0x7fa9bc10a070 2024-03-26T18:48:38.255 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.046+0000 7fa9c2200700 1 -- 172.21.15.97:0/1737300807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 msgr2=0x7fa9bc10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:38.255 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.046+0000 7fa9c2200700 1 --2- 172.21.15.97:0/1737300807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc10a490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0098a0 tx=0x7fa9ac009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:38.255 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.046+0000 7fa9c2200700 1 -- 172.21.15.97:0/1737300807 shutdown_connections 2024-03-26T18:48:38.255 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.046+0000 7fa9c2200700 1 --2- 172.21.15.97:0/1737300807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc10a490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.046+0000 7fa9c2200700 1 -- 172.21.15.97:0/1737300807 >> 172.21.15.97:0/1737300807 conn(0x7fa9bc07d6a0 msgr2=0x7fa9bc07dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.046+0000 7fa9c2200700 1 -- 172.21.15.97:0/1737300807 shutdown_connections 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.046+0000 7fa9c2200700 1 -- 172.21.15.97:0/1737300807 wait complete. 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.047+0000 7fa9c2200700 1 Processor -- start 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.047+0000 7fa9c2200700 1 -- start start 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.048+0000 7fa9c2200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.048+0000 7fa9c2200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10e5d0 con 0x7fa9bc10a070 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.048+0000 7fa9bb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.048+0000 7fa9bb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc19dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34262/0 (socket says 172.21.15.97:34262) 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.048+0000 7fa9bb400700 1 -- 172.21.15.97:0/3150085819 learned_addr learned my addr 172.21.15.97:0/3150085819 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.048+0000 7fa9bb400700 1 -- 172.21.15.97:0/3150085819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc19e420 con 0x7fa9bc10a070 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.048+0000 7fa9bb400700 1 --2- 172.21.15.97:0/3150085819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc19dee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0117e0 tx=0x7fa9ac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.049+0000 7fa9b3e00700 1 -- 172.21.15.97:0/3150085819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac003bf0 con 0x7fa9bc10a070 2024-03-26T18:48:38.256 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.049+0000 7fa9b3e00700 1 -- 172.21.15.97:0/3150085819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac01a910 con 0x7fa9bc10a070 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.049+0000 7fa9c2200700 1 -- 172.21.15.97:0/3150085819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9bc19e6b0 con 0x7fa9bc10a070 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.049+0000 7fa9b3e00700 1 -- 172.21.15.97:0/3150085819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac024970 con 0x7fa9bc10a070 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.049+0000 7fa9c2200700 1 -- 172.21.15.97:0/3150085819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9bc19eb10 con 0x7fa9bc10a070 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.050+0000 7fa9c2200700 1 -- 172.21.15.97:0/3150085819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9bc0639e0 con 0x7fa9bc10a070 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.050+0000 7fa9b3e00700 1 -- 172.21.15.97:0/3150085819 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fa9ac022020 con 0x7fa9bc10a070 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.051+0000 7fa9b3e00700 1 --2- 172.21.15.97:0/3150085819 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9a00392d0 0x7fa9a003b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.051+0000 7fa9b3e00700 1 -- 172.21.15.97:0/3150085819 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa9ac04d760 con 0x7fa9bc10a070 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.051+0000 7fa9baa00700 1 --2- 172.21.15.97:0/3150085819 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9a00392d0 0x7fa9a003b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.052+0000 7fa9baa00700 1 --2- 172.21.15.97:0/3150085819 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9a00392d0 0x7fa9a003b790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8006fd0 tx=0x7fa9a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.056+0000 7fa9b3e00700 1 -- 172.21.15.97:0/3150085819 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa9ac010380 con 0x7fa9bc10a070 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.295+0000 7fa9c2200700 1 -- 172.21.15.97:0/3150085819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fa9bc1a14e0 con 0x7fa9bc10a070 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.296+0000 7fa9b3e00700 1 -- 172.21.15.97:0/3150085819 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fa9ac00f2e0 con 0x7fa9bc10a070 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.299+0000 7fa9c2200700 1 -- 172.21.15.97:0/3150085819 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9a00392d0 msgr2=0x7fa9a003b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.299+0000 7fa9c2200700 1 --2- 172.21.15.97:0/3150085819 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9a00392d0 0x7fa9a003b790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8006fd0 tx=0x7fa9a8006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:38.257 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.299+0000 7fa9c2200700 1 -- 172.21.15.97:0/3150085819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 msgr2=0x7fa9bc19dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:38.258 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.299+0000 7fa9c2200700 1 --2- 172.21.15.97:0/3150085819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc19dee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0117e0 tx=0x7fa9ac011ba0 comp rx=0 tx=0).stop 2024-03-26T18:48:38.258 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.299+0000 7fa9c2200700 1 -- 172.21.15.97:0/3150085819 shutdown_connections 2024-03-26T18:48:38.258 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.299+0000 7fa9c2200700 1 --2- 172.21.15.97:0/3150085819 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9a00392d0 0x7fa9a003b790 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:38.258 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.299+0000 7fa9c2200700 1 --2- 172.21.15.97:0/3150085819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc10a070 0x7fa9bc19dee0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:38.258 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.299+0000 7fa9c2200700 1 -- 172.21.15.97:0/3150085819 >> 172.21.15.97:0/3150085819 conn(0x7fa9bc07d6a0 msgr2=0x7fa9bc107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:38.258 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.299+0000 7fa9c2200700 1 -- 172.21.15.97:0/3150085819 shutdown_connections 2024-03-26T18:48:38.258 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:37.299+0000 7fa9c2200700 1 -- 172.21.15.97:0/3150085819 wait complete. 2024-03-26T18:48:39.920 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.866+0000 7fe817800700 1 Processor -- start 2024-03-26T18:48:39.920 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.866+0000 7fe817800700 1 -- start start 2024-03-26T18:48:39.920 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.866+0000 7fe817800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81010a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:39.920 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.866+0000 7fe817800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe81010aad0 con 0x7fe81010a0e0 2024-03-26T18:48:39.921 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.867+0000 7fe814e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81010a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:39.921 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.867+0000 7fe814e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81010a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34278/0 (socket says 172.21.15.97:34278) 2024-03-26T18:48:39.921 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.867+0000 7fe814e00700 1 -- 172.21.15.97:0/3485666338 learned_addr learned my addr 172.21.15.97:0/3485666338 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:39.921 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.867+0000 7fe814e00700 1 -- 172.21.15.97:0/3485666338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81010b1f0 con 0x7fe81010a0e0 2024-03-26T18:48:39.921 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.868+0000 7fe814e00700 1 --2- 172.21.15.97:0/3485666338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81010a500 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8040098a0 tx=0x7fe804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9040b0a6cfb45b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:39.921 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.868+0000 7fe80f400700 1 -- 172.21.15.97:0/3485666338 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe804004160 con 0x7fe81010a0e0 2024-03-26T18:48:39.921 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.868+0000 7fe80f400700 1 -- 172.21.15.97:0/3485666338 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe804004300 con 0x7fe81010a0e0 2024-03-26T18:48:39.921 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.868+0000 7fe80f400700 1 -- 172.21.15.97:0/3485666338 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe804011b20 con 0x7fe81010a0e0 2024-03-26T18:48:39.925 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.868+0000 7fe817800700 1 -- 172.21.15.97:0/3485666338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 msgr2=0x7fe81010a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:39.925 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.868+0000 7fe817800700 1 --2- 172.21.15.97:0/3485666338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81010a500 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8040098a0 tx=0x7fe804009bb0 comp rx=0 tx=0).stop 2024-03-26T18:48:39.925 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.869+0000 7fe817800700 1 -- 172.21.15.97:0/3485666338 shutdown_connections 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.869+0000 7fe817800700 1 --2- 172.21.15.97:0/3485666338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81010a500 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.869+0000 7fe817800700 1 -- 172.21.15.97:0/3485666338 >> 172.21.15.97:0/3485666338 conn(0x7fe81007d6a0 msgr2=0x7fe81007dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.869+0000 7fe817800700 1 -- 172.21.15.97:0/3485666338 shutdown_connections 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.869+0000 7fe817800700 1 -- 172.21.15.97:0/3485666338 wait complete. 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.870+0000 7fe817800700 1 Processor -- start 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.870+0000 7fe817800700 1 -- start start 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.870+0000 7fe817800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.870+0000 7fe817800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe81010e5a0 con 0x7fe81010a0e0 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.870+0000 7fe814e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.870+0000 7fe814e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81019df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34288/0 (socket says 172.21.15.97:34288) 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.870+0000 7fe814e00700 1 -- 172.21.15.97:0/738796687 learned_addr learned my addr 172.21.15.97:0/738796687 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.871+0000 7fe814e00700 1 -- 172.21.15.97:0/738796687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81019e490 con 0x7fe81010a0e0 2024-03-26T18:48:39.926 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.871+0000 7fe814e00700 1 --2- 172.21.15.97:0/738796687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81019df50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe8040117e0 tx=0x7fe804011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.871+0000 7fe80d600700 1 -- 172.21.15.97:0/738796687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe804003ba0 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.871+0000 7fe80d600700 1 -- 172.21.15.97:0/738796687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe80401a910 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.872+0000 7fe817800700 1 -- 172.21.15.97:0/738796687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81019e720 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.872+0000 7fe80d600700 1 -- 172.21.15.97:0/738796687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8040248f0 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.872+0000 7fe817800700 1 -- 172.21.15.97:0/738796687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81019eb80 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.873+0000 7fe80d600700 1 -- 172.21.15.97:0/738796687 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fe804022020 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.873+0000 7fe817800700 1 -- 172.21.15.97:0/738796687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8100639e0 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.873+0000 7fe80d600700 1 --2- 172.21.15.97:0/738796687 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe7f80392d0 0x7fe7f803b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.873+0000 7fe80d600700 1 -- 172.21.15.97:0/738796687 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe80404d9f0 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.874+0000 7fe80fe00700 1 --2- 172.21.15.97:0/738796687 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe7f80392d0 0x7fe7f803b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.874+0000 7fe80fe00700 1 --2- 172.21.15.97:0/738796687 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe7f80392d0 0x7fe7f803b790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc006fd0 tx=0x7fe7fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:38.879+0000 7fe80d600700 1 -- 172.21.15.97:0/738796687 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe80401aba0 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.213+0000 7fe817800700 1 -- 172.21.15.97:0/738796687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe8101a1550 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.219+0000 7fe80d600700 1 -- 172.21.15.97:0/738796687 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fe80400fdd0 con 0x7fe81010a0e0 2024-03-26T18:48:39.927 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.221+0000 7fe817800700 1 -- 172.21.15.97:0/738796687 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe7f80392d0 msgr2=0x7fe7f803b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.221+0000 7fe817800700 1 --2- 172.21.15.97:0/738796687 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe7f80392d0 0x7fe7f803b790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc006fd0 tx=0x7fe7fc006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.221+0000 7fe817800700 1 -- 172.21.15.97:0/738796687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 msgr2=0x7fe81019df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.221+0000 7fe817800700 1 --2- 172.21.15.97:0/738796687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81019df50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe8040117e0 tx=0x7fe804011ba0 comp rx=0 tx=0).stop 2024-03-26T18:48:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.221+0000 7fe817800700 1 -- 172.21.15.97:0/738796687 shutdown_connections 2024-03-26T18:48:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.221+0000 7fe817800700 1 --2- 172.21.15.97:0/738796687 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe7f80392d0 0x7fe7f803b790 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.221+0000 7fe817800700 1 --2- 172.21.15.97:0/738796687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe81010a0e0 0x7fe81019df50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.221+0000 7fe817800700 1 -- 172.21.15.97:0/738796687 >> 172.21.15.97:0/738796687 conn(0x7fe81007d6a0 msgr2=0x7fe810107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.222+0000 7fe817800700 1 -- 172.21.15.97:0/738796687 shutdown_connections 2024-03-26T18:48:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-26T18:48:39.222+0000 7fe817800700 1 -- 172.21.15.97:0/738796687 wait complete. 2024-03-26T18:48:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-26T18:48:39.929 INFO:teuthology.orchestra.run.smithi097.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-26T18:48:39.929 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:39.929 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:48:39.929 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:39.929 INFO:teuthology.orchestra.run.smithi097.stdout:Or, if you are only running a single cluster on this host: 2024-03-26T18:48:39.929 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:39.929 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-26T18:48:39.929 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:39.929 INFO:teuthology.orchestra.run.smithi097.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-26T18:48:39.929 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:39.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph telemetry on 2024-03-26T18:48:39.930 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:39.930 INFO:teuthology.orchestra.run.smithi097.stdout:For more information see: 2024-03-26T18:48:39.930 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:39.930 INFO:teuthology.orchestra.run.smithi097.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-26T18:48:39.930 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:39.930 INFO:teuthology.orchestra.run.smithi097.stdout:Bootstrap complete. 2024-03-26T18:48:40.045 INFO:tasks.cephadm:Fetching config... 2024-03-26T18:48:40.045 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:48:40.045 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-26T18:48:40.130 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-26T18:48:40.130 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:48:40.130 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-26T18:48:40.215 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-26T18:48:40.215 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:48:40.215 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-26T18:48:40.340 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-26T18:48:40.341 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:48:40.341 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-26T18:48:40.431 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-26T18:48:40.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD6wr2B0ZNNqNIANPkfHHAQuw4/mKkq4IBbSCA5LG7xLkunFKKTw3LqrakJOlCJs7XxSY49PB0Idv57MuXrHrwLh1jJRDjvbJz0+5i+ldX4gZ1/RS9YhoXsict9Do/xDLTkfgmb3ihCprxArg/8E6N0Zn8Zs6VQvOoWsqFDY3pDgqkb0GkQzmDyL/JbceSoUYfxPd/6tl5F0MU9UhSTJi306gL3ppEm1pU0SnuuMFPUwc4aDIF9+cZYHKwZRVpmm++kG9Xst00gII+UqDMMLQqnn5nD82v2h8rQASYRPRWbIzUWphzSr+3S3JSVj4ZQjSMLJSjsqubEkr0FWwreQ996nvAXvhBYYhwQSq9Y5HUkmpUwyvPYmXxeNVrz3BNU/oMuIpB2DmJhPkxUqKvPjNQks08AjP87wYffIXOgKqUb08YHJ9EzsrtUDg/v4Hum1giB9trI4jm9ve1G4dZP1a3bjIVdCVsqrvLE6oh/oTcgnRGUg55uf6BFO26bx3C94U= ceph-2172d50a-eba1-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T18:48:40.550 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:40 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/738796687' entity='client.admin' 2024-03-26T18:48:40.602 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD6wr2B0ZNNqNIANPkfHHAQuw4/mKkq4IBbSCA5LG7xLkunFKKTw3LqrakJOlCJs7XxSY49PB0Idv57MuXrHrwLh1jJRDjvbJz0+5i+ldX4gZ1/RS9YhoXsict9Do/xDLTkfgmb3ihCprxArg/8E6N0Zn8Zs6VQvOoWsqFDY3pDgqkb0GkQzmDyL/JbceSoUYfxPd/6tl5F0MU9UhSTJi306gL3ppEm1pU0SnuuMFPUwc4aDIF9+cZYHKwZRVpmm++kG9Xst00gII+UqDMMLQqnn5nD82v2h8rQASYRPRWbIzUWphzSr+3S3JSVj4ZQjSMLJSjsqubEkr0FWwreQ996nvAXvhBYYhwQSq9Y5HUkmpUwyvPYmXxeNVrz3BNU/oMuIpB2DmJhPkxUqKvPjNQks08AjP87wYffIXOgKqUb08YHJ9EzsrtUDg/v4Hum1giB9trI4jm9ve1G4dZP1a3bjIVdCVsqrvLE6oh/oTcgnRGUg55uf6BFO26bx3C94U= ceph-2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:48:40.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD6wr2B0ZNNqNIANPkfHHAQuw4/mKkq4IBbSCA5LG7xLkunFKKTw3LqrakJOlCJs7XxSY49PB0Idv57MuXrHrwLh1jJRDjvbJz0+5i+ldX4gZ1/RS9YhoXsict9Do/xDLTkfgmb3ihCprxArg/8E6N0Zn8Zs6VQvOoWsqFDY3pDgqkb0GkQzmDyL/JbceSoUYfxPd/6tl5F0MU9UhSTJi306gL3ppEm1pU0SnuuMFPUwc4aDIF9+cZYHKwZRVpmm++kG9Xst00gII+UqDMMLQqnn5nD82v2h8rQASYRPRWbIzUWphzSr+3S3JSVj4ZQjSMLJSjsqubEkr0FWwreQ996nvAXvhBYYhwQSq9Y5HUkmpUwyvPYmXxeNVrz3BNU/oMuIpB2DmJhPkxUqKvPjNQks08AjP87wYffIXOgKqUb08YHJ9EzsrtUDg/v4Hum1giB9trI4jm9ve1G4dZP1a3bjIVdCVsqrvLE6oh/oTcgnRGUg55uf6BFO26bx3C94U= ceph-2172d50a-eba1-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T18:48:40.700 INFO:teuthology.orchestra.run.smithi105.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD6wr2B0ZNNqNIANPkfHHAQuw4/mKkq4IBbSCA5LG7xLkunFKKTw3LqrakJOlCJs7XxSY49PB0Idv57MuXrHrwLh1jJRDjvbJz0+5i+ldX4gZ1/RS9YhoXsict9Do/xDLTkfgmb3ihCprxArg/8E6N0Zn8Zs6VQvOoWsqFDY3pDgqkb0GkQzmDyL/JbceSoUYfxPd/6tl5F0MU9UhSTJi306gL3ppEm1pU0SnuuMFPUwc4aDIF9+cZYHKwZRVpmm++kG9Xst00gII+UqDMMLQqnn5nD82v2h8rQASYRPRWbIzUWphzSr+3S3JSVj4ZQjSMLJSjsqubEkr0FWwreQ996nvAXvhBYYhwQSq9Y5HUkmpUwyvPYmXxeNVrz3BNU/oMuIpB2DmJhPkxUqKvPjNQks08AjP87wYffIXOgKqUb08YHJ9EzsrtUDg/v4Hum1giB9trI4jm9ve1G4dZP1a3bjIVdCVsqrvLE6oh/oTcgnRGUg55uf6BFO26bx3C94U= ceph-2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:48:40.715 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD6wr2B0ZNNqNIANPkfHHAQuw4/mKkq4IBbSCA5LG7xLkunFKKTw3LqrakJOlCJs7XxSY49PB0Idv57MuXrHrwLh1jJRDjvbJz0+5i+ldX4gZ1/RS9YhoXsict9Do/xDLTkfgmb3ihCprxArg/8E6N0Zn8Zs6VQvOoWsqFDY3pDgqkb0GkQzmDyL/JbceSoUYfxPd/6tl5F0MU9UhSTJi306gL3ppEm1pU0SnuuMFPUwc4aDIF9+cZYHKwZRVpmm++kG9Xst00gII+UqDMMLQqnn5nD82v2h8rQASYRPRWbIzUWphzSr+3S3JSVj4ZQjSMLJSjsqubEkr0FWwreQ996nvAXvhBYYhwQSq9Y5HUkmpUwyvPYmXxeNVrz3BNU/oMuIpB2DmJhPkxUqKvPjNQks08AjP87wYffIXOgKqUb08YHJ9EzsrtUDg/v4Hum1giB9trI4jm9ve1G4dZP1a3bjIVdCVsqrvLE6oh/oTcgnRGUg55uf6BFO26bx3C94U= ceph-2172d50a-eba1-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T18:48:40.764 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD6wr2B0ZNNqNIANPkfHHAQuw4/mKkq4IBbSCA5LG7xLkunFKKTw3LqrakJOlCJs7XxSY49PB0Idv57MuXrHrwLh1jJRDjvbJz0+5i+ldX4gZ1/RS9YhoXsict9Do/xDLTkfgmb3ihCprxArg/8E6N0Zn8Zs6VQvOoWsqFDY3pDgqkb0GkQzmDyL/JbceSoUYfxPd/6tl5F0MU9UhSTJi306gL3ppEm1pU0SnuuMFPUwc4aDIF9+cZYHKwZRVpmm++kG9Xst00gII+UqDMMLQqnn5nD82v2h8rQASYRPRWbIzUWphzSr+3S3JSVj4ZQjSMLJSjsqubEkr0FWwreQ996nvAXvhBYYhwQSq9Y5HUkmpUwyvPYmXxeNVrz3BNU/oMuIpB2DmJhPkxUqKvPjNQks08AjP87wYffIXOgKqUb08YHJ9EzsrtUDg/v4Hum1giB9trI4jm9ve1G4dZP1a3bjIVdCVsqrvLE6oh/oTcgnRGUg55uf6BFO26bx3C94U= ceph-2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:48:40.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-26T18:48:41.185 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:48:43.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.951+0000 7f8960200700 1 -- 172.21.15.97:0/3954010618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89580763e0 msgr2=0x7f8958074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:43.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.951+0000 7f8960200700 1 --2- 172.21.15.97:0/3954010618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89580763e0 0x7f8958074840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f894c009910 tx=0x7f894c009c20 comp rx=0 tx=0).stop 2024-03-26T18:48:43.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.951+0000 7f8960200700 1 -- 172.21.15.97:0/3954010618 shutdown_connections 2024-03-26T18:48:43.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.951+0000 7f8960200700 1 --2- 172.21.15.97:0/3954010618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89580763e0 0x7f8958074840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:43.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.951+0000 7f8960200700 1 -- 172.21.15.97:0/3954010618 >> 172.21.15.97:0/3954010618 conn(0x7f89580fd3a0 msgr2=0x7f89580ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:43.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.951+0000 7f8960200700 1 -- 172.21.15.97:0/3954010618 shutdown_connections 2024-03-26T18:48:43.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.952+0000 7f8960200700 1 -- 172.21.15.97:0/3954010618 wait complete. 2024-03-26T18:48:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.953+0000 7f8960200700 1 Processor -- start 2024-03-26T18:48:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.953+0000 7f8960200700 1 -- start start 2024-03-26T18:48:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.953+0000 7f8960200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89580763e0 0x7f8958199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.953+0000 7f8960200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f895810a090 con 0x7f89580763e0 2024-03-26T18:48:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.953+0000 7f895d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89580763e0 0x7f8958199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.953+0000 7f895d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89580763e0 0x7f8958199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58768/0 (socket says 172.21.15.97:58768) 2024-03-26T18:48:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.953+0000 7f895d800700 1 -- 172.21.15.97:0/3050327064 learned_addr learned my addr 172.21.15.97:0/3050327064 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.954+0000 7f895d800700 1 -- 172.21.15.97:0/3050327064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8958199c50 con 0x7f89580763e0 2024-03-26T18:48:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.954+0000 7f895d800700 1 --2- 172.21.15.97:0/3050327064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89580763e0 0x7f8958199710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f894c005850 tx=0x7f894c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.954+0000 7f8956000700 1 -- 172.21.15.97:0/3050327064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f894c026070 con 0x7f89580763e0 2024-03-26T18:48:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.954+0000 7f8960200700 1 -- 172.21.15.97:0/3050327064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8958199ee0 con 0x7f89580763e0 2024-03-26T18:48:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.954+0000 7f8956000700 1 -- 172.21.15.97:0/3050327064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f894c0044a0 con 0x7f89580763e0 2024-03-26T18:48:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.954+0000 7f8960200700 1 -- 172.21.15.97:0/3050327064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895819cbd0 con 0x7f89580763e0 2024-03-26T18:48:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.955+0000 7f8956000700 1 -- 172.21.15.97:0/3050327064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f894c003ac0 con 0x7f89580763e0 2024-03-26T18:48:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.956+0000 7f8956000700 1 -- 172.21.15.97:0/3050327064 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f894c029070 con 0x7f89580763e0 2024-03-26T18:48:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.956+0000 7f8960200700 1 -- 172.21.15.97:0/3050327064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f895810a810 con 0x7f89580763e0 2024-03-26T18:48:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.956+0000 7f8956000700 1 --2- 172.21.15.97:0/3050327064 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8940039220 0x7f894003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.956+0000 7f8956000700 1 -- 172.21.15.97:0/3050327064 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f894c01e070 con 0x7f89580763e0 2024-03-26T18:48:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.956+0000 7f895ce00700 1 --2- 172.21.15.97:0/3050327064 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8940039220 0x7f894003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.957+0000 7f895ce00700 1 --2- 172.21.15.97:0/3050327064 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8940039220 0x7f894003b6e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8944006fd0 tx=0x7f8944006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:43.962+0000 7f8956000700 1 -- 172.21.15.97:0/3050327064 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f894c013070 con 0x7f89580763e0 2024-03-26T18:48:44.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:48:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:48:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:44.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.183+0000 7f8960200700 1 -- 172.21.15.97:0/3050327064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f895802dd10 con 0x7f89580763e0 2024-03-26T18:48:44.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.188+0000 7f8956000700 1 -- 172.21.15.97:0/3050327064 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f894c028d80 con 0x7f89580763e0 2024-03-26T18:48:44.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.191+0000 7f8960200700 1 -- 172.21.15.97:0/3050327064 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8940039220 msgr2=0x7f894003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:44.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.191+0000 7f8960200700 1 --2- 172.21.15.97:0/3050327064 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8940039220 0x7f894003b6e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8944006fd0 tx=0x7f8944006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:44.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.191+0000 7f8960200700 1 -- 172.21.15.97:0/3050327064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89580763e0 msgr2=0x7f8958199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:44.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.191+0000 7f8960200700 1 --2- 172.21.15.97:0/3050327064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89580763e0 0x7f8958199710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f894c005850 tx=0x7f894c005030 comp rx=0 tx=0).stop 2024-03-26T18:48:44.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.191+0000 7f8960200700 1 -- 172.21.15.97:0/3050327064 shutdown_connections 2024-03-26T18:48:44.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.191+0000 7f8960200700 1 --2- 172.21.15.97:0/3050327064 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8940039220 0x7f894003b6e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:44.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.191+0000 7f8960200700 1 --2- 172.21.15.97:0/3050327064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89580763e0 0x7f8958199710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:44.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.192+0000 7f8960200700 1 -- 172.21.15.97:0/3050327064 >> 172.21.15.97:0/3050327064 conn(0x7f89580fd3a0 msgr2=0x7f89580fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:44.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.192+0000 7f8960200700 1 -- 172.21.15.97:0/3050327064 shutdown_connections 2024-03-26T18:48:44.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:44.192+0000 7f8960200700 1 -- 172.21.15.97:0/3050327064 wait complete. 2024-03-26T18:48:44.913 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-26T18:48:44.914 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-26T18:48:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:48:45.491 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:45 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3050327064' entity='client.admin' 2024-03-26T18:48:45.492 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:48:45.492 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.349+0000 7f5921400700 1 -- 172.21.15.97:0/4292240925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c1085c0 msgr2=0x7f591c1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.349+0000 7f5921400700 1 --2- 172.21.15.97:0/4292240925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c1085c0 0x7f591c1089e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f590c009910 tx=0x7f590c009c20 comp rx=0 tx=0).stop 2024-03-26T18:48:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.349+0000 7f5921400700 1 -- 172.21.15.97:0/4292240925 shutdown_connections 2024-03-26T18:48:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.349+0000 7f5921400700 1 --2- 172.21.15.97:0/4292240925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c1085c0 0x7f591c1089e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.349+0000 7f5921400700 1 -- 172.21.15.97:0/4292240925 >> 172.21.15.97:0/4292240925 conn(0x7f591c0774a0 msgr2=0x7f591c079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.349+0000 7f5921400700 1 -- 172.21.15.97:0/4292240925 shutdown_connections 2024-03-26T18:48:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.349+0000 7f5921400700 1 -- 172.21.15.97:0/4292240925 wait complete. 2024-03-26T18:48:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.351+0000 7f5921400700 1 Processor -- start 2024-03-26T18:48:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.351+0000 7f5921400700 1 -- start start 2024-03-26T18:48:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.352+0000 7f5921400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c1085c0 0x7f591c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.352+0000 7f5921400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f591c10e430 con 0x7f591c1085c0 2024-03-26T18:48:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.352+0000 7f591a800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c1085c0 0x7f591c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.352+0000 7f591a800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c1085c0 0x7f591c19dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58800/0 (socket says 172.21.15.97:58800) 2024-03-26T18:48:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.352+0000 7f591a800700 1 -- 172.21.15.97:0/3879464615 learned_addr learned my addr 172.21.15.97:0/3879464615 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.353+0000 7f591a800700 1 -- 172.21.15.97:0/3879464615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c19e000 con 0x7f591c1085c0 2024-03-26T18:48:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.353+0000 7f591a800700 1 --2- 172.21.15.97:0/3879464615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c1085c0 0x7f591c19dac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f590c005950 tx=0x7f590c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.353+0000 7f5913e00700 1 -- 172.21.15.97:0/3879464615 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f590c027070 con 0x7f591c1085c0 2024-03-26T18:48:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.353+0000 7f5913e00700 1 -- 172.21.15.97:0/3879464615 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f590c022470 con 0x7f591c1085c0 2024-03-26T18:48:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.353+0000 7f5913e00700 1 -- 172.21.15.97:0/3879464615 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f590c00f5d0 con 0x7f591c1085c0 2024-03-26T18:48:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.353+0000 7f5921400700 1 -- 172.21.15.97:0/3879464615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c19e290 con 0x7f591c1085c0 2024-03-26T18:48:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.353+0000 7f5921400700 1 -- 172.21.15.97:0/3879464615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c1a0f80 con 0x7f591c1085c0 2024-03-26T18:48:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.355+0000 7f5913e00700 1 -- 172.21.15.97:0/3879464615 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f590c02a070 con 0x7f591c1085c0 2024-03-26T18:48:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.355+0000 7f5921400700 1 -- 172.21.15.97:0/3879464615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591c10ebb0 con 0x7f591c1085c0 2024-03-26T18:48:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.355+0000 7f5913e00700 1 --2- 172.21.15.97:0/3879464615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5900039280 0x7f590003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.355+0000 7f5913e00700 1 -- 172.21.15.97:0/3879464615 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f590c01e030 con 0x7f591c1085c0 2024-03-26T18:48:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.355+0000 7f5911e00700 1 --2- 172.21.15.97:0/3879464615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5900039280 0x7f590003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.356+0000 7f5911e00700 1 --2- 172.21.15.97:0/3879464615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5900039280 0x7f590003b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5904006fd0 tx=0x7f5904006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.361+0000 7f5913e00700 1 -- 172.21.15.97:0/3879464615 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f590c013070 con 0x7f591c1085c0 2024-03-26T18:48:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.586+0000 7f5921400700 1 -- 172.21.15.97:0/3879464615 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f591c02dda0 con 0x7f5900039280 2024-03-26T18:48:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.593+0000 7f5913e00700 1 -- 172.21.15.97:0/3879464615 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f591c02dda0 con 0x7f5900039280 2024-03-26T18:48:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.595+0000 7f5921400700 1 -- 172.21.15.97:0/3879464615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5900039280 msgr2=0x7f590003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.595+0000 7f5921400700 1 --2- 172.21.15.97:0/3879464615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5900039280 0x7f590003b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5904006fd0 tx=0x7f5904006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.595+0000 7f5921400700 1 -- 172.21.15.97:0/3879464615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c1085c0 msgr2=0x7f591c19dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.595+0000 7f5921400700 1 --2- 172.21.15.97:0/3879464615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c1085c0 0x7f591c19dac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f590c005950 tx=0x7f590c005030 comp rx=0 tx=0).stop 2024-03-26T18:48:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.596+0000 7f5921400700 1 -- 172.21.15.97:0/3879464615 shutdown_connections 2024-03-26T18:48:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.596+0000 7f5921400700 1 --2- 172.21.15.97:0/3879464615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5900039280 0x7f590003b740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.596+0000 7f5921400700 1 --2- 172.21.15.97:0/3879464615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c1085c0 0x7f591c19dac0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.596+0000 7f5921400700 1 -- 172.21.15.97:0/3879464615 >> 172.21.15.97:0/3879464615 conn(0x7f591c0774a0 msgr2=0x7f591c077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.596+0000 7f5921400700 1 -- 172.21.15.97:0/3879464615 shutdown_connections 2024-03-26T18:48:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:47.596+0000 7f5921400700 1 -- 172.21.15.97:0/3879464615 wait complete. 2024-03-26T18:48:48.351 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi105 2024-03-26T18:48:48.351 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:48:48.351 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/etc/ceph/ceph.conf 2024-03-26T18:48:48.369 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:48:48.369 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:48:48.426 INFO:tasks.cephadm:Adding host smithi105 to orchestrator... 2024-03-26T18:48:48.426 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch host add smithi105 2024-03-26T18:48:48.831 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:48:48.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:48 smithi097 ceph-mon[31955]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:48.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:48 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:48.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:48 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:48:48.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:48 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:48:48.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:48 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:48:48.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:48 smithi097 ceph-mon[31955]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-26T18:48:49.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:49 smithi097 ceph-mon[31955]: Updating smithi097:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:48:49.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:49 smithi097 ceph-mon[31955]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:48:49.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:49 smithi097 ceph-mon[31955]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:48:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.947+0000 7f178d200700 1 -- 172.21.15.97:0/538614608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788108610 msgr2=0x7f17881089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.947+0000 7f178d200700 1 --2- 172.21.15.97:0/538614608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788108610 0x7f17881089f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1778009910 tx=0x7f1778009c20 comp rx=0 tx=0).stop 2024-03-26T18:48:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.947+0000 7f178d200700 1 -- 172.21.15.97:0/538614608 shutdown_connections 2024-03-26T18:48:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.947+0000 7f178d200700 1 --2- 172.21.15.97:0/538614608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788108610 0x7f17881089f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.947+0000 7f178d200700 1 -- 172.21.15.97:0/538614608 >> 172.21.15.97:0/538614608 conn(0x7f1788077b50 msgr2=0x7f1788079f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.947+0000 7f178d200700 1 -- 172.21.15.97:0/538614608 shutdown_connections 2024-03-26T18:48:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.947+0000 7f178d200700 1 -- 172.21.15.97:0/538614608 wait complete. 2024-03-26T18:48:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.949+0000 7f178d200700 1 Processor -- start 2024-03-26T18:48:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.949+0000 7f178d200700 1 -- start start 2024-03-26T18:48:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.949+0000 7f178d200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788108610 0x7f178819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.949+0000 7f178d200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17881129e0 con 0x7f1788108610 2024-03-26T18:48:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.950+0000 7f1785e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788108610 0x7f178819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.950+0000 7f1785e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788108610 0x7f178819dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51902/0 (socket says 172.21.15.97:51902) 2024-03-26T18:48:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.950+0000 7f1785e00700 1 -- 172.21.15.97:0/1800272086 learned_addr learned my addr 172.21.15.97:0/1800272086 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.950+0000 7f1785e00700 1 -- 172.21.15.97:0/1800272086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178819e4f0 con 0x7f1788108610 2024-03-26T18:48:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.950+0000 7f1785e00700 1 --2- 172.21.15.97:0/1800272086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788108610 0x7f178819dfb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1778005850 tx=0x7f1778005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.951+0000 7f177ea00700 1 -- 172.21.15.97:0/1800272086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1778026070 con 0x7f1788108610 2024-03-26T18:48:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.951+0000 7f178d200700 1 -- 172.21.15.97:0/1800272086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178819e7e0 con 0x7f1788108610 2024-03-26T18:48:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.951+0000 7f177ea00700 1 -- 172.21.15.97:0/1800272086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f177800bbd0 con 0x7f1788108610 2024-03-26T18:48:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.951+0000 7f177ea00700 1 -- 172.21.15.97:0/1800272086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1778003970 con 0x7f1788108610 2024-03-26T18:48:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.951+0000 7f178d200700 1 -- 172.21.15.97:0/1800272086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17881a27e0 con 0x7f1788108610 2024-03-26T18:48:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.952+0000 7f177ea00700 1 -- 172.21.15.97:0/1800272086 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f1778029070 con 0x7f1788108610 2024-03-26T18:48:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.952+0000 7f178d200700 1 -- 172.21.15.97:0/1800272086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1788109af0 con 0x7f1788108610 2024-03-26T18:48:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.952+0000 7f177ea00700 1 --2- 172.21.15.97:0/1800272086 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f176c039220 0x7f176c03b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.952+0000 7f177ea00700 1 -- 172.21.15.97:0/1800272086 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f177801e070 con 0x7f1788108610 2024-03-26T18:48:50.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.953+0000 7f1785400700 1 --2- 172.21.15.97:0/1800272086 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f176c039220 0x7f176c03b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:50.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.953+0000 7f1785400700 1 --2- 172.21.15.97:0/1800272086 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f176c039220 0x7f176c03b6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1774006fd0 tx=0x7f1774006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:50.958+0000 7f177ea00700 1 -- 172.21.15.97:0/1800272086 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1778013070 con 0x7f1788108610 2024-03-26T18:48:51.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:51.189+0000 7f178d200700 1 -- 172.21.15.97:0/1800272086 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi105", "target": ["mon-mgr", ""]}) -- 0x7f178819f270 con 0x7f176c039220 2024-03-26T18:48:51.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:51 smithi097 ceph-mon[31955]: Updating smithi097:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:48:51.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:51 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:51.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:51 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:51.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:51 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:52.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:52 smithi097 ceph-mon[31955]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:48:53.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:53 smithi097 ceph-mon[31955]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi105", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:48:53.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:53 smithi097 ceph-mon[31955]: Deploying cephadm binary to smithi105 2024-03-26T18:48:54.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:54 smithi097 ceph-mon[31955]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:48:55.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.912+0000 7f177ea00700 1 -- 172.21.15.97:0/1800272086 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f178819f270 con 0x7f176c039220 2024-03-26T18:48:55.905 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi105' with addr '172.21.15.105' 2024-03-26T18:48:55.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.916+0000 7f178d200700 1 -- 172.21.15.97:0/1800272086 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f176c039220 msgr2=0x7f176c03b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:55.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.916+0000 7f178d200700 1 --2- 172.21.15.97:0/1800272086 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f176c039220 0x7f176c03b6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1774006fd0 tx=0x7f1774006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:55.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.916+0000 7f178d200700 1 -- 172.21.15.97:0/1800272086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788108610 msgr2=0x7f178819dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:55.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.916+0000 7f178d200700 1 --2- 172.21.15.97:0/1800272086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788108610 0x7f178819dfb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1778005850 tx=0x7f1778005030 comp rx=0 tx=0).stop 2024-03-26T18:48:55.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.917+0000 7f178d200700 1 -- 172.21.15.97:0/1800272086 shutdown_connections 2024-03-26T18:48:55.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.917+0000 7f178d200700 1 --2- 172.21.15.97:0/1800272086 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f176c039220 0x7f176c03b6e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:55.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.917+0000 7f178d200700 1 --2- 172.21.15.97:0/1800272086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788108610 0x7f178819dfb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:55.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.917+0000 7f178d200700 1 -- 172.21.15.97:0/1800272086 >> 172.21.15.97:0/1800272086 conn(0x7f1788077b50 msgr2=0x7f1788109fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:55.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.917+0000 7f178d200700 1 -- 172.21.15.97:0/1800272086 shutdown_connections 2024-03-26T18:48:55.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:55.917+0000 7f178d200700 1 -- 172.21.15.97:0/1800272086 wait complete. 2024-03-26T18:48:56.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:56 smithi097 ceph-mon[31955]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:48:56.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:56 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:56.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:56 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:48:56.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T18:48:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:48:57.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:57 smithi097 ceph-mon[31955]: Added host smithi105 2024-03-26T18:48:57.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:57 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:57.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:57 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:58.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:58 smithi097 ceph-mon[31955]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:48:59.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.203+0000 7fc457600700 1 -- 172.21.15.97:0/3505834613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc450104210 msgr2=0x7fc4501045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:59.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.203+0000 7fc457600700 1 --2- 172.21.15.97:0/3505834613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc450104210 0x7fc4501045f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc444009910 tx=0x7fc444009c20 comp rx=0 tx=0).stop 2024-03-26T18:48:59.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.203+0000 7fc457600700 1 -- 172.21.15.97:0/3505834613 shutdown_connections 2024-03-26T18:48:59.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.203+0000 7fc457600700 1 --2- 172.21.15.97:0/3505834613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc450104210 0x7fc4501045f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:59.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.203+0000 7fc457600700 1 -- 172.21.15.97:0/3505834613 >> 172.21.15.97:0/3505834613 conn(0x7fc4500ffaa0 msgr2=0x7fc450101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:59.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.203+0000 7fc457600700 1 -- 172.21.15.97:0/3505834613 shutdown_connections 2024-03-26T18:48:59.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.203+0000 7fc457600700 1 -- 172.21.15.97:0/3505834613 wait complete. 2024-03-26T18:48:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.204+0000 7fc457600700 1 Processor -- start 2024-03-26T18:48:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.205+0000 7fc457600700 1 -- start start 2024-03-26T18:48:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.205+0000 7fc457600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc450104210 0x7fc45019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:59.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.205+0000 7fc457600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc45010e5e0 con 0x7fc450104210 2024-03-26T18:48:59.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.205+0000 7fc454c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc450104210 0x7fc45019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:59.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.205+0000 7fc454c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc450104210 0x7fc45019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51924/0 (socket says 172.21.15.97:51924) 2024-03-26T18:48:59.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.205+0000 7fc454c00700 1 -- 172.21.15.97:0/2638569071 learned_addr learned my addr 172.21.15.97:0/2638569071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:48:59.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.206+0000 7fc454c00700 1 -- 172.21.15.97:0/2638569071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45019fc50 con 0x7fc450104210 2024-03-26T18:48:59.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.206+0000 7fc454c00700 1 --2- 172.21.15.97:0/2638569071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc450104210 0x7fc45019f710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc444006010 tx=0x7fc444005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:59.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.206+0000 7fc44d600700 1 -- 172.21.15.97:0/2638569071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc444026070 con 0x7fc450104210 2024-03-26T18:48:59.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.206+0000 7fc457600700 1 -- 172.21.15.97:0/2638569071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45019ff40 con 0x7fc450104210 2024-03-26T18:48:59.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.206+0000 7fc44d600700 1 -- 172.21.15.97:0/2638569071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44400bbd0 con 0x7fc450104210 2024-03-26T18:48:59.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.206+0000 7fc44d600700 1 -- 172.21.15.97:0/2638569071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc444003910 con 0x7fc450104210 2024-03-26T18:48:59.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.207+0000 7fc457600700 1 -- 172.21.15.97:0/2638569071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4501998d0 con 0x7fc450104210 2024-03-26T18:48:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.208+0000 7fc457600700 1 -- 172.21.15.97:0/2638569071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4501056f0 con 0x7fc450104210 2024-03-26T18:48:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.208+0000 7fc44d600700 1 -- 172.21.15.97:0/2638569071 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fc444029070 con 0x7fc450104210 2024-03-26T18:48:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.208+0000 7fc44d600700 1 --2- 172.21.15.97:0/2638569071 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc438039220 0x7fc43803b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:48:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.208+0000 7fc44d600700 1 -- 172.21.15.97:0/2638569071 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc44401e070 con 0x7fc450104210 2024-03-26T18:48:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.209+0000 7fc44fe00700 1 --2- 172.21.15.97:0/2638569071 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc438039220 0x7fc43803b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:48:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.209+0000 7fc44fe00700 1 --2- 172.21.15.97:0/2638569071 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc438039220 0x7fc43803b6e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc43c006fd0 tx=0x7fc43c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:48:59.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.214+0000 7fc44d600700 1 -- 172.21.15.97:0/2638569071 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc444013070 con 0x7fc450104210 2024-03-26T18:48:59.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.445+0000 7fc457600700 1 -- 172.21.15.97:0/2638569071 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc45019ae70 con 0x7fc438039220 2024-03-26T18:48:59.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.448+0000 7fc44d600700 1 -- 172.21.15.97:0/2638569071 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fc45019ae70 con 0x7fc438039220 2024-03-26T18:48:59.440 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:48:59.441 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}] 2024-03-26T18:48:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.451+0000 7fc457600700 1 -- 172.21.15.97:0/2638569071 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc438039220 msgr2=0x7fc43803b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.451+0000 7fc457600700 1 --2- 172.21.15.97:0/2638569071 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc438039220 0x7fc43803b6e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc43c006fd0 tx=0x7fc43c006e40 comp rx=0 tx=0).stop 2024-03-26T18:48:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.451+0000 7fc457600700 1 -- 172.21.15.97:0/2638569071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc450104210 msgr2=0x7fc45019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:48:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.451+0000 7fc457600700 1 --2- 172.21.15.97:0/2638569071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc450104210 0x7fc45019f710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc444006010 tx=0x7fc444005030 comp rx=0 tx=0).stop 2024-03-26T18:48:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.451+0000 7fc457600700 1 -- 172.21.15.97:0/2638569071 shutdown_connections 2024-03-26T18:48:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.451+0000 7fc457600700 1 --2- 172.21.15.97:0/2638569071 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc438039220 0x7fc43803b6e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.451+0000 7fc457600700 1 --2- 172.21.15.97:0/2638569071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc450104210 0x7fc45019f710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:48:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.451+0000 7fc457600700 1 -- 172.21.15.97:0/2638569071 >> 172.21.15.97:0/2638569071 conn(0x7fc4500ffaa0 msgr2=0x7fc450105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:48:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.451+0000 7fc457600700 1 -- 172.21.15.97:0/2638569071 shutdown_connections 2024-03-26T18:48:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:48:59.451+0000 7fc457600700 1 -- 172.21.15.97:0/2638569071 wait complete. 2024-03-26T18:48:59.802 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:59 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:48:59.802 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:48:59 smithi097 ceph-mon[31955]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:00.148 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi144 2024-03-26T18:49:00.148 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:49:00.148 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.conf 2024-03-26T18:49:00.167 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:49:00.167 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:49:00.223 INFO:tasks.cephadm:Adding host smithi144 to orchestrator... 2024-03-26T18:49:00.223 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch host add smithi144 2024-03-26T18:49:00.626 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:49:00.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:00 smithi097 ceph-mon[31955]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:49:01.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:01 smithi097 ceph-mon[31955]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.697+0000 7ff936200700 1 -- 172.21.15.97:0/3215058392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff930101fe0 msgr2=0x7ff9301023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.697+0000 7ff936200700 1 --2- 172.21.15.97:0/3215058392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff930101fe0 0x7ff9301023c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff920009910 tx=0x7ff920009c20 comp rx=0 tx=0).stop 2024-03-26T18:49:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.697+0000 7ff936200700 1 -- 172.21.15.97:0/3215058392 shutdown_connections 2024-03-26T18:49:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.697+0000 7ff936200700 1 --2- 172.21.15.97:0/3215058392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff930101fe0 0x7ff9301023c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.697+0000 7ff936200700 1 -- 172.21.15.97:0/3215058392 >> 172.21.15.97:0/3215058392 conn(0x7ff9300fd890 msgr2=0x7ff9300ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.697+0000 7ff936200700 1 -- 172.21.15.97:0/3215058392 shutdown_connections 2024-03-26T18:49:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.697+0000 7ff936200700 1 -- 172.21.15.97:0/3215058392 wait complete. 2024-03-26T18:49:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.698+0000 7ff936200700 1 Processor -- start 2024-03-26T18:49:02.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.698+0000 7ff936200700 1 -- start start 2024-03-26T18:49:02.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.699+0000 7ff936200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff930101fe0 0x7ff93019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:02.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.699+0000 7ff936200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff93010c3b0 con 0x7ff930101fe0 2024-03-26T18:49:02.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.699+0000 7ff92f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff930101fe0 0x7ff93019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:02.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.699+0000 7ff92f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff930101fe0 0x7ff93019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36670/0 (socket says 172.21.15.97:36670) 2024-03-26T18:49:02.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.699+0000 7ff92f400700 1 -- 172.21.15.97:0/1780104192 learned_addr learned my addr 172.21.15.97:0/1780104192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:49:02.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.699+0000 7ff92f400700 1 -- 172.21.15.97:0/1780104192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93019fc60 con 0x7ff930101fe0 2024-03-26T18:49:02.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.700+0000 7ff92f400700 1 --2- 172.21.15.97:0/1780104192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff930101fe0 0x7ff93019f720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff920006010 tx=0x7ff920005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:02.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.700+0000 7ff92cc00700 1 -- 172.21.15.97:0/1780104192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff920026070 con 0x7ff930101fe0 2024-03-26T18:49:02.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.700+0000 7ff936200700 1 -- 172.21.15.97:0/1780104192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93019ff50 con 0x7ff930101fe0 2024-03-26T18:49:02.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.700+0000 7ff92cc00700 1 -- 172.21.15.97:0/1780104192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff92000bbd0 con 0x7ff930101fe0 2024-03-26T18:49:02.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.700+0000 7ff92cc00700 1 -- 172.21.15.97:0/1780104192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff920003910 con 0x7ff930101fe0 2024-03-26T18:49:02.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.700+0000 7ff936200700 1 -- 172.21.15.97:0/1780104192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9301998e0 con 0x7ff930101fe0 2024-03-26T18:49:02.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.701+0000 7ff92cc00700 1 -- 172.21.15.97:0/1780104192 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7ff920029070 con 0x7ff930101fe0 2024-03-26T18:49:02.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.701+0000 7ff936200700 1 -- 172.21.15.97:0/1780104192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9301034c0 con 0x7ff930101fe0 2024-03-26T18:49:02.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.702+0000 7ff92cc00700 1 --2- 172.21.15.97:0/1780104192 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff914039280 0x7ff91403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:02.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.702+0000 7ff92cc00700 1 -- 172.21.15.97:0/1780104192 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff92001e070 con 0x7ff930101fe0 2024-03-26T18:49:02.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.702+0000 7ff927e00700 1 --2- 172.21.15.97:0/1780104192 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff914039280 0x7ff91403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:02.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.703+0000 7ff927e00700 1 --2- 172.21.15.97:0/1780104192 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff914039280 0x7ff91403b740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff918006fd0 tx=0x7ff918006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:02.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.708+0000 7ff92cc00700 1 -- 172.21.15.97:0/1780104192 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff920013070 con 0x7ff930101fe0 2024-03-26T18:49:02.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:02.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:02.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:02.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:02.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:49:02.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:02.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:02.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:02 smithi097 ceph-mon[31955]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-26T18:49:02.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:02 smithi097 ceph-mon[31955]: Updating smithi105:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:02.938+0000 7ff936200700 1 -- 172.21.15.97:0/1780104192 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}) -- 0x7ff93019ae80 con 0x7ff914039280 2024-03-26T18:49:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:03 smithi097 ceph-mon[31955]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:49:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:03 smithi097 ceph-mon[31955]: Updating smithi105:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:49:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:03 smithi097 ceph-mon[31955]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:03 smithi097 ceph-mon[31955]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:49:05.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:04 smithi097 ceph-mon[31955]: Deploying cephadm binary to smithi144 2024-03-26T18:49:06.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:05 smithi097 ceph-mon[31955]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:07.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.708+0000 7ff92cc00700 1 -- 172.21.15.97:0/1780104192 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff93019ae80 con 0x7ff914039280 2024-03-26T18:49:07.700 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi144' with addr '172.21.15.144' 2024-03-26T18:49:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.711+0000 7ff936200700 1 -- 172.21.15.97:0/1780104192 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff914039280 msgr2=0x7ff91403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.711+0000 7ff936200700 1 --2- 172.21.15.97:0/1780104192 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff914039280 0x7ff91403b740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff918006fd0 tx=0x7ff918006e40 comp rx=0 tx=0).stop 2024-03-26T18:49:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.712+0000 7ff936200700 1 -- 172.21.15.97:0/1780104192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff930101fe0 msgr2=0x7ff93019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.712+0000 7ff936200700 1 --2- 172.21.15.97:0/1780104192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff930101fe0 0x7ff93019f720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff920006010 tx=0x7ff920005030 comp rx=0 tx=0).stop 2024-03-26T18:49:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.712+0000 7ff936200700 1 -- 172.21.15.97:0/1780104192 shutdown_connections 2024-03-26T18:49:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.712+0000 7ff936200700 1 --2- 172.21.15.97:0/1780104192 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff914039280 0x7ff91403b740 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.712+0000 7ff936200700 1 --2- 172.21.15.97:0/1780104192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff930101fe0 0x7ff93019f720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.712+0000 7ff936200700 1 -- 172.21.15.97:0/1780104192 >> 172.21.15.97:0/1780104192 conn(0x7ff9300fd890 msgr2=0x7ff9300760e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.712+0000 7ff936200700 1 -- 172.21.15.97:0/1780104192 shutdown_connections 2024-03-26T18:49:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:07.712+0000 7ff936200700 1 -- 172.21.15.97:0/1780104192 wait complete. 2024-03-26T18:49:08.075 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:07 smithi097 ceph-mon[31955]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:08.075 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:08.075 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:08.503 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T18:49:08.931 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:49:09.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:08 smithi097 ceph-mon[31955]: Added host smithi144 2024-03-26T18:49:09.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:08 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:09.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:08 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:10.089 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:09 smithi097 ceph-mon[31955]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:10.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.891+0000 7f1227800700 1 -- 172.21.15.97:0/1393265431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1220103000 msgr2=0x7f12201033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.891+0000 7f1227800700 1 --2- 172.21.15.97:0/1393265431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1220103000 0x7f12201033e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1210009910 tx=0x7f1210009c20 comp rx=0 tx=0).stop 2024-03-26T18:49:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.892+0000 7f1227800700 1 -- 172.21.15.97:0/1393265431 shutdown_connections 2024-03-26T18:49:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.892+0000 7f1227800700 1 --2- 172.21.15.97:0/1393265431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1220103000 0x7f12201033e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.892+0000 7f1227800700 1 -- 172.21.15.97:0/1393265431 >> 172.21.15.97:0/1393265431 conn(0x7f12200778f0 msgr2=0x7f12200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.892+0000 7f1227800700 1 -- 172.21.15.97:0/1393265431 shutdown_connections 2024-03-26T18:49:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.892+0000 7f1227800700 1 -- 172.21.15.97:0/1393265431 wait complete. 2024-03-26T18:49:10.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.893+0000 7f1227800700 1 Processor -- start 2024-03-26T18:49:10.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.893+0000 7f1227800700 1 -- start start 2024-03-26T18:49:10.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.893+0000 7f1227800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1220103000 0x7f122019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:10.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.893+0000 7f1227800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f122010c420 con 0x7f1220103000 2024-03-26T18:49:10.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.894+0000 7f1224e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1220103000 0x7f122019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:10.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.894+0000 7f1224e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1220103000 0x7f122019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54662/0 (socket says 172.21.15.97:54662) 2024-03-26T18:49:10.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.894+0000 7f1224e00700 1 -- 172.21.15.97:0/2444591269 learned_addr learned my addr 172.21.15.97:0/2444591269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:49:10.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.894+0000 7f1224e00700 1 -- 172.21.15.97:0/2444591269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122019db40 con 0x7f1220103000 2024-03-26T18:49:10.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.894+0000 7f1224e00700 1 --2- 172.21.15.97:0/2444591269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1220103000 0x7f122019d600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1210006010 tx=0x7f1210005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:10.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.895+0000 7f121e000700 1 -- 172.21.15.97:0/2444591269 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1210026070 con 0x7f1220103000 2024-03-26T18:49:10.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.895+0000 7f121e000700 1 -- 172.21.15.97:0/2444591269 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f121000bc30 con 0x7f1220103000 2024-03-26T18:49:10.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.895+0000 7f121e000700 1 -- 172.21.15.97:0/2444591269 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1210003ab0 con 0x7f1220103000 2024-03-26T18:49:10.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.895+0000 7f1227800700 1 -- 172.21.15.97:0/2444591269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122019ddd0 con 0x7f1220103000 2024-03-26T18:49:10.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.895+0000 7f1227800700 1 -- 172.21.15.97:0/2444591269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1220197780 con 0x7f1220103000 2024-03-26T18:49:10.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.896+0000 7f1227800700 1 -- 172.21.15.97:0/2444591269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1220069fc0 con 0x7f1220103000 2024-03-26T18:49:10.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.896+0000 7f121e000700 1 -- 172.21.15.97:0/2444591269 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f1210029070 con 0x7f1220103000 2024-03-26T18:49:10.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.896+0000 7f121e000700 1 --2- 172.21.15.97:0/2444591269 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1204039280 0x7f120403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:10.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.897+0000 7f121e000700 1 -- 172.21.15.97:0/2444591269 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f121001e070 con 0x7f1220103000 2024-03-26T18:49:10.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.897+0000 7f1217e00700 1 --2- 172.21.15.97:0/2444591269 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1204039280 0x7f120403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:10.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.897+0000 7f1217e00700 1 --2- 172.21.15.97:0/2444591269 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1204039280 0x7f120403b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1208006fd0 tx=0x7f1208006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:10.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:10.902+0000 7f121e000700 1 -- 172.21.15.97:0/2444591269 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1210013070 con 0x7f1220103000 2024-03-26T18:49:11.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.130+0000 7f1227800700 1 -- 172.21.15.97:0/2444591269 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1220198cb0 con 0x7f1204039280 2024-03-26T18:49:11.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.132+0000 7f121e000700 1 -- 172.21.15.97:0/2444591269 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f1220198cb0 con 0x7f1204039280 2024-03-26T18:49:11.124 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:49:11.124 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}] 2024-03-26T18:49:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.135+0000 7f1227800700 1 -- 172.21.15.97:0/2444591269 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1204039280 msgr2=0x7f120403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.135+0000 7f1227800700 1 --2- 172.21.15.97:0/2444591269 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1204039280 0x7f120403b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1208006fd0 tx=0x7f1208006e40 comp rx=0 tx=0).stop 2024-03-26T18:49:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.135+0000 7f1227800700 1 -- 172.21.15.97:0/2444591269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1220103000 msgr2=0x7f122019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.135+0000 7f1227800700 1 --2- 172.21.15.97:0/2444591269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1220103000 0x7f122019d600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1210006010 tx=0x7f1210005050 comp rx=0 tx=0).stop 2024-03-26T18:49:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.135+0000 7f1227800700 1 -- 172.21.15.97:0/2444591269 shutdown_connections 2024-03-26T18:49:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.135+0000 7f1227800700 1 --2- 172.21.15.97:0/2444591269 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1204039280 0x7f120403b740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.135+0000 7f1227800700 1 --2- 172.21.15.97:0/2444591269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1220103000 0x7f122019d600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:11.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.135+0000 7f1227800700 1 -- 172.21.15.97:0/2444591269 >> 172.21.15.97:0/2444591269 conn(0x7f12200778f0 msgr2=0x7f1220103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:11.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.136+0000 7f1227800700 1 -- 172.21.15.97:0/2444591269 shutdown_connections 2024-03-26T18:49:11.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:11.136+0000 7f1227800700 1 -- 172.21.15.97:0/2444591269 wait complete. 2024-03-26T18:49:11.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:11.816 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:11 smithi097 ceph-mon[31955]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:11.890 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-26T18:49:11.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-26T18:49:12.318 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:49:12.735 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:12 smithi097 ceph-mon[31955]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:49:13.734 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:13 smithi097 ceph-mon[31955]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.273+0000 7f3ea4a00700 1 -- 172.21.15.97:0/2716675275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea0102f40 msgr2=0x7f3ea0103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.273+0000 7f3ea4a00700 1 --2- 172.21.15.97:0/2716675275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea0102f40 0x7f3ea0103320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000b0a0 tx=0x7f3e9000b3b0 comp rx=0 tx=0).stop 2024-03-26T18:49:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.273+0000 7f3ea4a00700 1 -- 172.21.15.97:0/2716675275 shutdown_connections 2024-03-26T18:49:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.273+0000 7f3ea4a00700 1 --2- 172.21.15.97:0/2716675275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea0102f40 0x7f3ea0103320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.273+0000 7f3ea4a00700 1 -- 172.21.15.97:0/2716675275 >> 172.21.15.97:0/2716675275 conn(0x7f3ea00778f0 msgr2=0x7f3ea00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.273+0000 7f3ea4a00700 1 -- 172.21.15.97:0/2716675275 shutdown_connections 2024-03-26T18:49:14.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.273+0000 7f3ea4a00700 1 -- 172.21.15.97:0/2716675275 wait complete. 2024-03-26T18:49:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.274+0000 7f3ea4a00700 1 Processor -- start 2024-03-26T18:49:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.275+0000 7f3ea4a00700 1 -- start start 2024-03-26T18:49:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.275+0000 7f3ea4a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea0102f40 0x7f3ea019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.275+0000 7f3ea4a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea010c430 con 0x7f3ea0102f40 2024-03-26T18:49:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.275+0000 7f3e9de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea0102f40 0x7f3ea019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.275+0000 7f3e9de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea0102f40 0x7f3ea019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54694/0 (socket says 172.21.15.97:54694) 2024-03-26T18:49:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.275+0000 7f3e9de00700 1 -- 172.21.15.97:0/378855297 learned_addr learned my addr 172.21.15.97:0/378855297 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:49:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.276+0000 7f3e9de00700 1 -- 172.21.15.97:0/378855297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea019fcd0 con 0x7f3ea0102f40 2024-03-26T18:49:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.276+0000 7f3e9de00700 1 --2- 172.21.15.97:0/378855297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea0102f40 0x7f3ea019f790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e900062a0 tx=0x7f3e9000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.276+0000 7f3e96a00700 1 -- 172.21.15.97:0/378855297 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e90017050 con 0x7f3ea0102f40 2024-03-26T18:49:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.276+0000 7f3ea4a00700 1 -- 172.21.15.97:0/378855297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea019ff60 con 0x7f3ea0102f40 2024-03-26T18:49:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.276+0000 7f3e96a00700 1 -- 172.21.15.97:0/378855297 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e90023070 con 0x7f3ea0102f40 2024-03-26T18:49:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.276+0000 7f3e96a00700 1 -- 172.21.15.97:0/378855297 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e90004000 con 0x7f3ea0102f40 2024-03-26T18:49:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.276+0000 7f3ea4a00700 1 -- 172.21.15.97:0/378855297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea0199950 con 0x7f3ea0102f40 2024-03-26T18:49:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.278+0000 7f3ea4a00700 1 -- 172.21.15.97:0/378855297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea006a080 con 0x7f3ea0102f40 2024-03-26T18:49:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.278+0000 7f3e96a00700 1 -- 172.21.15.97:0/378855297 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f3e9001e450 con 0x7f3ea0102f40 2024-03-26T18:49:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.278+0000 7f3e96a00700 1 --2- 172.21.15.97:0/378855297 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3e84039280 0x7f3e8403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.278+0000 7f3e96a00700 1 -- 172.21.15.97:0/378855297 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e9000f050 con 0x7f3ea0102f40 2024-03-26T18:49:14.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.279+0000 7f3e97e00700 1 --2- 172.21.15.97:0/378855297 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3e84039280 0x7f3e8403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:14.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.279+0000 7f3e97e00700 1 --2- 172.21.15.97:0/378855297 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3e84039280 0x7f3e8403b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e88006fd0 tx=0x7f3e88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:14.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.284+0000 7f3e96a00700 1 -- 172.21.15.97:0/378855297 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3e90026bd0 con 0x7f3ea0102f40 2024-03-26T18:49:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:14.517+0000 7f3ea4a00700 1 -- 172.21.15.97:0/378855297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3ea019ae80 con 0x7f3ea0102f40 2024-03-26T18:49:14.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:14.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:14.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:14.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:14.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:49:14.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:14.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:14.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:14 smithi097 ceph-mon[31955]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-26T18:49:14.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:14 smithi097 ceph-mon[31955]: Updating smithi144:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:15.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.508+0000 7f3e96a00700 1 -- 172.21.15.97:0/378855297 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3e90008cb0 con 0x7f3ea0102f40 2024-03-26T18:49:15.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.513+0000 7f3ea4a00700 1 -- 172.21.15.97:0/378855297 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3e84039280 msgr2=0x7f3e8403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:15.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.513+0000 7f3ea4a00700 1 --2- 172.21.15.97:0/378855297 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3e84039280 0x7f3e8403b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e88006fd0 tx=0x7f3e88006e40 comp rx=0 tx=0).stop 2024-03-26T18:49:15.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.513+0000 7f3ea4a00700 1 -- 172.21.15.97:0/378855297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea0102f40 msgr2=0x7f3ea019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:15.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.513+0000 7f3ea4a00700 1 --2- 172.21.15.97:0/378855297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea0102f40 0x7f3ea019f790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e900062a0 tx=0x7f3e9000bdd0 comp rx=0 tx=0).stop 2024-03-26T18:49:15.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.513+0000 7f3ea4a00700 1 -- 172.21.15.97:0/378855297 shutdown_connections 2024-03-26T18:49:15.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.513+0000 7f3ea4a00700 1 --2- 172.21.15.97:0/378855297 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3e84039280 0x7f3e8403b740 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:15.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.513+0000 7f3ea4a00700 1 --2- 172.21.15.97:0/378855297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea0102f40 0x7f3ea019f790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:15.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.513+0000 7f3ea4a00700 1 -- 172.21.15.97:0/378855297 >> 172.21.15.97:0/378855297 conn(0x7f3ea00778f0 msgr2=0x7f3ea0108100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:15.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.513+0000 7f3ea4a00700 1 -- 172.21.15.97:0/378855297 shutdown_connections 2024-03-26T18:49:15.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:15.513+0000 7f3ea4a00700 1 -- 172.21.15.97:0/378855297 wait complete. 2024-03-26T18:49:15.506 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-03-26T18:49:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:15 smithi097 ceph-mon[31955]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:49:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:15 smithi097 ceph-mon[31955]: Updating smithi144:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:49:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:15 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/378855297' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T18:49:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:15 smithi097 ceph-mon[31955]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:16.266 INFO:tasks.cephadm:Adding mon.a on smithi097 2024-03-26T18:49:16.267 INFO:tasks.cephadm:Adding mon.b on smithi105 2024-03-26T18:49:16.267 INFO:tasks.cephadm:Adding mon.c on smithi144 2024-03-26T18:49:16.267 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi097:172.21.15.97=a;smithi105:172.21.15.105=b;smithi144:172.21.15.144=c' 2024-03-26T18:49:16.504 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:16.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:16 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/378855297' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T18:49:16.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:16 smithi097 ceph-mon[31955]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T18:49:17.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:17 smithi097 ceph-mon[31955]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:17.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.955+0000 7fe9091a9700 1 -- 172.21.15.144:0/362936525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904101ff0 msgr2=0x7fe9041023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:17.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.955+0000 7fe9091a9700 1 --2- 172.21.15.144:0/362936525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904101ff0 0x7fe9041023d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009910 tx=0x7fe8f4009c20 comp rx=0 tx=0).stop 2024-03-26T18:49:17.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.955+0000 7fe9091a9700 1 -- 172.21.15.144:0/362936525 shutdown_connections 2024-03-26T18:49:17.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.955+0000 7fe9091a9700 1 --2- 172.21.15.144:0/362936525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904101ff0 0x7fe9041023d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:17.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.955+0000 7fe9091a9700 1 -- 172.21.15.144:0/362936525 >> 172.21.15.144:0/362936525 conn(0x7fe9040fd8c0 msgr2=0x7fe9040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:17.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.955+0000 7fe9091a9700 1 -- 172.21.15.144:0/362936525 shutdown_connections 2024-03-26T18:49:17.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.955+0000 7fe9091a9700 1 -- 172.21.15.144:0/362936525 wait complete. 2024-03-26T18:49:17.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.956+0000 7fe9091a9700 1 Processor -- start 2024-03-26T18:49:17.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.956+0000 7fe9091a9700 1 -- start start 2024-03-26T18:49:17.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.956+0000 7fe9091a9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904101ff0 0x7fe9040746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:17.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.956+0000 7fe9091a9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe90410c3c0 con 0x7fe904101ff0 2024-03-26T18:49:17.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.956+0000 7fe902d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904101ff0 0x7fe9040746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:17.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.956+0000 7fe902d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904101ff0 0x7fe9040746b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.144:48178/0 (socket says 172.21.15.144:48178) 2024-03-26T18:49:17.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.957+0000 7fe902d9d700 1 -- 172.21.15.144:0/2837859637 learned_addr learned my addr 172.21.15.144:0/2837859637 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T18:49:17.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.957+0000 7fe902d9d700 1 -- 172.21.15.144:0/2837859637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9040765a0 con 0x7fe904101ff0 2024-03-26T18:49:17.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.957+0000 7fe902d9d700 1 --2- 172.21.15.144:0/2837859637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904101ff0 0x7fe9040746b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005850 tx=0x7fe8f4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:17.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.958+0000 7fe8ebfff700 1 -- 172.21.15.144:0/2837859637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4026070 con 0x7fe904101ff0 2024-03-26T18:49:17.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.958+0000 7fe9091a9700 1 -- 172.21.15.144:0/2837859637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe904074c50 con 0x7fe904101ff0 2024-03-26T18:49:17.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.958+0000 7fe9091a9700 1 -- 172.21.15.144:0/2837859637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe904075110 con 0x7fe904101ff0 2024-03-26T18:49:17.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.958+0000 7fe8ebfff700 1 -- 172.21.15.144:0/2837859637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f400b750 con 0x7fe904101ff0 2024-03-26T18:49:17.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.958+0000 7fe8ebfff700 1 -- 172.21.15.144:0/2837859637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4003680 con 0x7fe904101ff0 2024-03-26T18:49:17.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.959+0000 7fe9091a9700 1 -- 172.21.15.144:0/2837859637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f0005330 con 0x7fe904101ff0 2024-03-26T18:49:17.952 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.960+0000 7fe8ebfff700 1 -- 172.21.15.144:0/2837859637 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fe8f4029070 con 0x7fe904101ff0 2024-03-26T18:49:17.952 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.960+0000 7fe8ebfff700 1 --2- 172.21.15.144:0/2837859637 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8ec039220 0x7fe8ec03b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:17.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.960+0000 7fe8ebfff700 1 -- 172.21.15.144:0/2837859637 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe8f401e070 con 0x7fe904101ff0 2024-03-26T18:49:17.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.960+0000 7fe90259c700 1 --2- 172.21.15.144:0/2837859637 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8ec039220 0x7fe8ec03b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:17.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.961+0000 7fe90259c700 1 --2- 172.21.15.144:0/2837859637 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8ec039220 0x7fe8ec03b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8006fd0 tx=0x7fe8f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:17.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:17.975+0000 7fe8ebfff700 1 -- 172.21.15.144:0/2837859637 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe8f4030020 con 0x7fe904101ff0 2024-03-26T18:49:18.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.183+0000 7fe9091a9700 1 -- 172.21.15.144:0/2837859637 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi105:172.21.15.105=b;smithi144:172.21.15.144=c", "target": ["mon-mgr", ""]}) -- 0x7fe8f0000c80 con 0x7fe8ec039220 2024-03-26T18:49:18.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.199+0000 7fe8ebfff700 1 -- 172.21.15.144:0/2837859637 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe8f0000c80 con 0x7fe8ec039220 2024-03-26T18:49:18.190 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mon update... 2024-03-26T18:49:18.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.200+0000 7fe9091a9700 1 -- 172.21.15.144:0/2837859637 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8ec039220 msgr2=0x7fe8ec03b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:18.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.200+0000 7fe9091a9700 1 --2- 172.21.15.144:0/2837859637 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8ec039220 0x7fe8ec03b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8006fd0 tx=0x7fe8f8006e40 comp rx=0 tx=0).stop 2024-03-26T18:49:18.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.200+0000 7fe9091a9700 1 -- 172.21.15.144:0/2837859637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904101ff0 msgr2=0x7fe9040746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:18.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.200+0000 7fe9091a9700 1 --2- 172.21.15.144:0/2837859637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904101ff0 0x7fe9040746b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005850 tx=0x7fe8f4005030 comp rx=0 tx=0).stop 2024-03-26T18:49:18.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.200+0000 7fe9091a9700 1 -- 172.21.15.144:0/2837859637 shutdown_connections 2024-03-26T18:49:18.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.200+0000 7fe9091a9700 1 --2- 172.21.15.144:0/2837859637 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8ec039220 0x7fe8ec03b6e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:18.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.200+0000 7fe9091a9700 1 --2- 172.21.15.144:0/2837859637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904101ff0 0x7fe9040746b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:18.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.200+0000 7fe9091a9700 1 -- 172.21.15.144:0/2837859637 >> 172.21.15.144:0/2837859637 conn(0x7fe9040fd8c0 msgr2=0x7fe904104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:18.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.200+0000 7fe9091a9700 1 -- 172.21.15.144:0/2837859637 shutdown_connections 2024-03-26T18:49:18.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:18.200+0000 7fe9091a9700 1 -- 172.21.15.144:0/2837859637 wait complete. 2024-03-26T18:49:18.711 DEBUG:teuthology.orchestra.run.smithi105:mon.b> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.b.service 2024-03-26T18:49:18.713 DEBUG:teuthology.orchestra.run.smithi144:mon.c> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.c.service 2024-03-26T18:49:18.716 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T18:49:18.716 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T18:49:19.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi105:172.21.15.105=b;smithi144:172.21.15.144=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:49:19.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi105:172.21.15.105=b;smithi144:172.21.15.144=c;count:3 2024-03-26T18:49:19.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:19.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:19.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:19.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:19.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:19.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:19.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:19.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: Deploying daemon mon.c on smithi144 2024-03-26T18:49:19.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:19 smithi097 ceph-mon[31955]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:20.545 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:49:21.834 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:21 smithi144 systemd[1]: Starting Ceph mon.c for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:49:22.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:21 smithi097 ceph-mon[31955]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.297+0000 7f1206926700 1 -- 172.21.15.144:0/1462567016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1200100fd0 msgr2=0x7f12001013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.297+0000 7f1206926700 1 --2- 172.21.15.144:0/1462567016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1200100fd0 0x7f12001013b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009910 tx=0x7f11f4009c20 comp rx=0 tx=0).stop 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.297+0000 7f1206926700 1 -- 172.21.15.144:0/1462567016 shutdown_connections 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.297+0000 7f1206926700 1 --2- 172.21.15.144:0/1462567016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1200100fd0 0x7f12001013b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.297+0000 7f1206926700 1 -- 172.21.15.144:0/1462567016 >> 172.21.15.144:0/1462567016 conn(0x7f12000fc860 msgr2=0x7f12000fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.297+0000 7f1206926700 1 -- 172.21.15.144:0/1462567016 shutdown_connections 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.297+0000 7f1206926700 1 -- 172.21.15.144:0/1462567016 wait complete. 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.298+0000 7f1206926700 1 Processor -- start 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.298+0000 7f1206926700 1 -- start start 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.298+0000 7f1206926700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1200100fd0 0x7f120019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.298+0000 7f1206926700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f120010a290 con 0x7f1200100fd0 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.299+0000 7f11fffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1200100fd0 0x7f120019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:22.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.299+0000 7f11fffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1200100fd0 0x7f120019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.144:46606/0 (socket says 172.21.15.144:46606) 2024-03-26T18:49:22.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.299+0000 7f11fffff700 1 -- 172.21.15.144:0/1767113591 learned_addr learned my addr 172.21.15.144:0/1767113591 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T18:49:22.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.299+0000 7f11fffff700 1 -- 172.21.15.144:0/1767113591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120019fd60 con 0x7f1200100fd0 2024-03-26T18:49:22.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.300+0000 7f11fffff700 1 --2- 172.21.15.144:0/1767113591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1200100fd0 0x7f120019f820 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f11f4005850 tx=0x7f11f4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:22.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.300+0000 7f11fd7fa700 1 -- 172.21.15.144:0/1767113591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11f4026070 con 0x7f1200100fd0 2024-03-26T18:49:22.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.300+0000 7f1206926700 1 -- 172.21.15.144:0/1767113591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12001a0050 con 0x7f1200100fd0 2024-03-26T18:49:22.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.300+0000 7f1206926700 1 -- 172.21.15.144:0/1767113591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12001999e0 con 0x7f1200100fd0 2024-03-26T18:49:22.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.300+0000 7f11fd7fa700 1 -- 172.21.15.144:0/1767113591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f400b6f0 con 0x7f1200100fd0 2024-03-26T18:49:22.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.300+0000 7f11fd7fa700 1 -- 172.21.15.144:0/1767113591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11f4030060 con 0x7f1200100fd0 2024-03-26T18:49:22.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.301+0000 7f1206926700 1 -- 172.21.15.144:0/1767113591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ec005330 con 0x7f1200100fd0 2024-03-26T18:49:22.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.302+0000 7f11fd7fa700 1 -- 172.21.15.144:0/1767113591 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f11f4029070 con 0x7f1200100fd0 2024-03-26T18:49:22.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.302+0000 7f11fd7fa700 1 --2- 172.21.15.144:0/1767113591 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11e8039220 0x7f11e803b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:22.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.302+0000 7f11fd7fa700 1 -- 172.21.15.144:0/1767113591 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f11f401e070 con 0x7f1200100fd0 2024-03-26T18:49:22.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.302+0000 7f11ff7fe700 1 --2- 172.21.15.144:0/1767113591 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11e8039220 0x7f11e803b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:22.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.303+0000 7f11ff7fe700 1 --2- 172.21.15.144:0/1767113591 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11e8039220 0x7f11e803b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11f0006fd0 tx=0x7f11f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:22.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.317+0000 7f11fd7fa700 1 -- 172.21.15.144:0/1767113591 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f11f4013070 con 0x7f1200100fd0 2024-03-26T18:49:22.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.611+0000 7f1206926700 1 -- 172.21.15.144:0/1767113591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f11ec0058f0 con 0x7f1200100fd0 2024-03-26T18:49:22.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.612+0000 7f11fd7fa700 1 -- 172.21.15.144:0/1767113591 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f11f4020e30 con 0x7f1200100fd0 2024-03-26T18:49:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:49:22.604 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","modified":"2024-03-26T18:47:19.215409Z","created":"2024-03-26T18:47:19.215409Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-26T18:49:22.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.614+0000 7f1206926700 1 -- 172.21.15.144:0/1767113591 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11e8039220 msgr2=0x7f11e803b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:22.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.614+0000 7f1206926700 1 --2- 172.21.15.144:0/1767113591 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11e8039220 0x7f11e803b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11f0006fd0 tx=0x7f11f0006e40 comp rx=0 tx=0).stop 2024-03-26T18:49:22.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.614+0000 7f1206926700 1 -- 172.21.15.144:0/1767113591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1200100fd0 msgr2=0x7f120019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:22.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.614+0000 7f1206926700 1 --2- 172.21.15.144:0/1767113591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1200100fd0 0x7f120019f820 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f11f4005850 tx=0x7f11f4005030 comp rx=0 tx=0).stop 2024-03-26T18:49:22.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.614+0000 7f1206926700 1 -- 172.21.15.144:0/1767113591 shutdown_connections 2024-03-26T18:49:22.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.614+0000 7f1206926700 1 --2- 172.21.15.144:0/1767113591 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11e8039220 0x7f11e803b6e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:22.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.614+0000 7f1206926700 1 --2- 172.21.15.144:0/1767113591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1200100fd0 0x7f120019f820 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:22.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.614+0000 7f1206926700 1 -- 172.21.15.144:0/1767113591 >> 172.21.15.144:0/1767113591 conn(0x7f12000fc860 msgr2=0x7f12001029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:22.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.614+0000 7f1206926700 1 -- 172.21.15.144:0/1767113591 shutdown_connections 2024-03-26T18:49:22.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:22.614+0000 7f1206926700 1 -- 172.21.15.144:0/1767113591 wait complete. 2024-03-26T18:49:22.607 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-03-26T18:49:22.657 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 podman[36034]: 2024-03-26 18:49:22.413501598 +0000 UTC m=+0.097801936 container create a632f2217aa92887c9a1d1075afdc47830df9369eb293ca614f35146b90644ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-26T18:49:22.657 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 podman[36034]: 2024-03-26 18:49:22.333987696 +0000 UTC m=+0.018288039 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:49:22.657 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 podman[36034]: 2024-03-26 18:49:22.599459155 +0000 UTC m=+0.283759532 container init a632f2217aa92887c9a1d1075afdc47830df9369eb293ca614f35146b90644ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-26T18:49:22.657 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 podman[36034]: 2024-03-26 18:49:22.602545095 +0000 UTC m=+0.286845441 container start a632f2217aa92887c9a1d1075afdc47830df9369eb293ca614f35146b90644ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-03-26T18:49:22.657 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T18:49:22.657 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev), process ceph-mon, pid 2 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pidfile_write: ignore empty --pid-file 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: load: jerasure load: lrc 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: RocksDB version: 7.9.2 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Git sha 0 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Compile date 2024-03-20 16:36:06 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: DB SUMMARY 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: DB Session ID: TWSN999Q4U0ZIVKPH34B 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: CURRENT file: CURRENT 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-26T18:49:22.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.error_if_exists: 0 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.create_if_missing: 0 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.paranoid_checks: 1 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.env: 0x5651debd9c60 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.info_log: 0x5651df60fae0 2024-03-26T18:49:22.659 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.statistics: (nil) 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.use_fsync: 0 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_log_file_size: 0 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.allow_fallocate: 1 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.use_direct_reads: 0 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T18:49:22.660 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.db_log_dir: 2024-03-26T18:49:22.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.wal_dir: 2024-03-26T18:49:22.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T18:49:22.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T18:49:22.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T18:49:22.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T18:49:22.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T18:49:22.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T18:49:22.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T18:49:22.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T18:49:22.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.write_buffer_manager: 0x5651df5e21e0 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.unordered_write: 0 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T18:49:22.662 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.row_cache: None 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.wal_filter: None 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.two_write_queues: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.wal_compression: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.atomic_flush: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.log_readahead_size: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T18:49:22.663 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_background_jobs: 2 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_background_compactions: -1 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_subcompactions: 1 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T18:49:22.664 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_open_files: -1 2024-03-26T18:49:22.666 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T18:49:22.666 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T18:49:22.666 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T18:49:22.666 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T18:49:22.666 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_background_flushes: -1 2024-03-26T18:49:22.666 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Compression algorithms supported: 2024-03-26T18:49:22.666 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T18:49:22.666 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: kZSTD supported: 0 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: kXpressCompression supported: 0 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: kZlibCompression supported: 1 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: kSnappyCompression supported: 1 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: kLZ4Compression supported: 1 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: kBZip2Compression supported: 0 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.merge_operator: 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_filter: None 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T18:49:22.667 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5651df60fbe0) 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: cache_index_and_filter_blocks: 1 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: pin_top_level_index_and_filter: 1 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: index_type: 0 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: data_block_index_type: 0 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: index_shortening: 1 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: checksum: 4 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: no_block_cache: 0 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_cache: 0x5651df615090 2024-03-26T18:49:22.668 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_cache_name: BinnedLRUCache 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_cache_options: 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: capacity : 536870912 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: num_shard_bits : 4 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: strict_capacity_limit : 0 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: high_pri_pool_ratio: 0.000 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_cache_compressed: (nil) 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: persistent_cache: (nil) 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_size: 4096 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_size_deviation: 10 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_restart_interval: 16 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: index_block_restart_interval: 1 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: metadata_block_size: 4096 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: partition_filters: 0 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: use_delta_encoding: 1 2024-03-26T18:49:22.669 INFO:journalctl@ceph.mon.c.smithi144.stdout: filter_policy: bloomfilter 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout: whole_key_filtering: 1 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout: verify_compression: 0 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout: read_amp_bytes_per_bit: 0 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout: format_version: 5 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout: enable_index_compression: 1 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_align: 0 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout: max_auto_readahead_size: 262144 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout: prepopulate_block_cache: 0 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout: initial_auto_readahead_size: 8192 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compression: NoCompression 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T18:49:22.670 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.num_levels: 7 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T18:49:22.671 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T18:49:22.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T18:49:22.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.table_properties_collectors: 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.inplace_update_support: 0 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.bloom_locality: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.max_successive_merges: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.ttl: 2592000 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.enable_blob_files: false 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.min_blob_size: 0 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3b9810a7-7000-40fa-9c1a-541a890572f7 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711478962637515, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711478962638148, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711478962, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3b9810a7-7000-40fa-9c1a-541a890572f7", "db_session_id": "TWSN999Q4U0ZIVKPH34B", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711478962638257, "job": 1, "event": "recovery_finished"} 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-26T18:49:22.676 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5651df708000 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: DB pointer 0x5651df6f2000 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: ** DB Stats ** 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: ** Compaction Stats [default] ** 2024-03-26T18:49:22.677 INFO:journalctl@ceph.mon.c.smithi144.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: ** Compaction Stats [default] ** 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T18:49:22.678 INFO:journalctl@ceph.mon.c.smithi144.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: Block cache BinnedLRUCache@0x5651df615090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T18:49:22.679 INFO:journalctl@ceph.mon.c.smithi144.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: using public_addr v2:172.21.15.144:0/0 -> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: starting mon.c rank -1 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(???) e0 preinit fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).mds e1 new map 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout: e1 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout: legacy client fscid: -1 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout: No filesystems configured 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-26T18:49:22.680 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Added host smithi105 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.681 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:22.682 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Updating smithi105:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Updating smithi105:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Deploying cephadm binary to smithi144 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Added host smithi144 2024-03-26T18:49:22.683 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Updating smithi144:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:22.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Updating smithi144:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/378855297' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/378855297' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi105:172.21.15.105=b;smithi144:172.21.15.144=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi105:172.21.15.105=b;smithi144:172.21.15.144=c;count:3 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:22.685 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:22.686 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:22.686 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:22.686 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: Deploying daemon mon.c on smithi144 2024-03-26T18:49:22.686 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.686 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:22.686 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 ceph-mon[36053]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T18:49:23.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 bash[36034]: a632f2217aa92887c9a1d1075afdc47830df9369eb293ca614f35146b90644ea 2024-03-26T18:49:23.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:22 smithi144 systemd[1]: Started Ceph mon.c for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:49:24.330 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T18:49:24.330 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T18:49:24.575 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:49:28.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:49:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: mon.a calling monitor election 2024-03-26T18:49:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: mon.c calling monitor election 2024-03-26T18:49:28.157 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:28.157 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.157 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.157 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:28.157 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.157 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: monmap epoch 2 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: last_changed 2024-03-26T18:49:22.674308+0000 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: created 2024-03-26T18:47:19.215409+0000 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: min_mon_release 19 (squid) 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: election_strategy: 1 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: fsmap 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: mgrmap e13: x(active, since 58s) 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: overall HEALTH_OK 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:28.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:28.159 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:28.159 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:28.159 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:49:28.159 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: mon.a calling monitor election 2024-03-26T18:49:28.159 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.159 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:28.159 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.159 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: mon.c calling monitor election 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: monmap epoch 2 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: last_changed 2024-03-26T18:49:22.674308+0000 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: created 2024-03-26T18:47:19.215409+0000 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: min_mon_release 19 (squid) 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: election_strategy: 1 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-26T18:49:28.160 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T18:49:28.161 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: fsmap 2024-03-26T18:49:28.161 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T18:49:28.161 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: mgrmap e13: x(active, since 58s) 2024-03-26T18:49:28.161 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: overall HEALTH_OK 2024-03-26T18:49:28.161 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:28.161 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:28.161 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:28.161 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:28.161 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:28.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.003+0000 7fc7985ee700 1 -- 172.21.15.144:0/2542709519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77c0035c0 msgr2=0x7fc77c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:28.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.003+0000 7fc7985ee700 1 --2- 172.21.15.144:0/2542709519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77c0035c0 0x7fc77c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc790101dc0 tx=0x7fc788009e80 comp rx=0 tx=0).stop 2024-03-26T18:49:28.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.003+0000 7fc7985ee700 1 -- 172.21.15.144:0/2542709519 shutdown_connections 2024-03-26T18:49:28.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.003+0000 7fc7985ee700 1 --2- 172.21.15.144:0/2542709519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77c0035c0 0x7fc77c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:28.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.003+0000 7fc7985ee700 1 -- 172.21.15.144:0/2542709519 >> 172.21.15.144:0/2542709519 conn(0x7fc7900fd9a0 msgr2=0x7fc7900ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:28.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.003+0000 7fc7985ee700 1 -- 172.21.15.144:0/2542709519 shutdown_connections 2024-03-26T18:49:28.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.003+0000 7fc7985ee700 1 -- 172.21.15.144:0/2542709519 wait complete. 2024-03-26T18:49:28.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.004+0000 7fc7985ee700 1 Processor -- start 2024-03-26T18:49:28.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.004+0000 7fc7985ee700 1 -- start start 2024-03-26T18:49:28.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.004+0000 7fc7985ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc77c0035c0 0x7fc7900746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:28.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.004+0000 7fc7985ee700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc790074bf0 0x7fc790075070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:28.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.004+0000 7fc7985ee700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc790109f20 con 0x7fc790074bf0 2024-03-26T18:49:28.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.004+0000 7fc7985ee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc790109da0 con 0x7fc77c0035c0 2024-03-26T18:49:28.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.004+0000 7fc79638a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc77c0035c0 0x7fc7900746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:28.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.005+0000 7fc79638a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc77c0035c0 0x7fc7900746b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43084/0 (socket says 172.21.15.144:43084) 2024-03-26T18:49:28.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.005+0000 7fc79638a700 1 -- 172.21.15.144:0/183732250 learned_addr learned my addr 172.21.15.144:0/183732250 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T18:49:28.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.005+0000 7fc79638a700 1 -- 172.21.15.144:0/183732250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc77c0035c0 msgr2=0x7fc7900746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-26T18:49:28.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.005+0000 7fc79638a700 1 -- 172.21.15.144:0/183732250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc77c0035c0 msgr2=0x7fc7900746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T18:49:28.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.005+0000 7fc79638a700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc77c0035c0 0x7fc7900746b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T18:49:28.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.005+0000 7fc79638a700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc77c0035c0 0x7fc7900746b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:49:28.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.005+0000 7fc795b89700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc790074bf0 0x7fc790075070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:28.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.005+0000 7fc795b89700 1 -- 172.21.15.144:0/183732250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc77c0035c0 msgr2=0x7fc7900746b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:49:28.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.005+0000 7fc795b89700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc77c0035c0 0x7fc7900746b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:28.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.005+0000 7fc795b89700 1 -- 172.21.15.144:0/183732250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7900765d0 con 0x7fc790074bf0 2024-03-26T18:49:28.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.006+0000 7fc795b89700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc790074bf0 0x7fc790075070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc78000b4d0 tx=0x7fc78000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:28.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.006+0000 7fc7877fe700 1 -- 172.21.15.144:0/183732250 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc780011630 con 0x7fc790074bf0 2024-03-26T18:49:29.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.006+0000 7fc7985ee700 1 -- 172.21.15.144:0/183732250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7901a3d10 con 0x7fc790074bf0 2024-03-26T18:49:29.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.006+0000 7fc7985ee700 1 -- 172.21.15.144:0/183732250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7901a4250 con 0x7fc790074bf0 2024-03-26T18:49:29.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.006+0000 7fc7877fe700 1 -- 172.21.15.144:0/183732250 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc780011cb0 con 0x7fc790074bf0 2024-03-26T18:49:29.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.006+0000 7fc7877fe700 1 -- 172.21.15.144:0/183732250 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc7800193b0 con 0x7fc790074bf0 2024-03-26T18:49:29.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.008+0000 7fc7877fe700 1 -- 172.21.15.144:0/183732250 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fc780018020 con 0x7fc790074bf0 2024-03-26T18:49:29.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.008+0000 7fc7985ee700 1 -- 172.21.15.144:0/183732250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7901035d0 con 0x7fc790074bf0 2024-03-26T18:49:29.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.008+0000 7fc7877fe700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc774038fc0 0x7fc77403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:29.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.008+0000 7fc7877fe700 1 -- 172.21.15.144:0/183732250 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc78004bc90 con 0x7fc790074bf0 2024-03-26T18:49:29.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.009+0000 7fc79638a700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc774038fc0 0x7fc77403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:29.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.009+0000 7fc79638a700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc774038fc0 0x7fc77403b480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc78801c040 tx=0x7fc788005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:29.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.025+0000 7fc7877fe700 1 -- 172.21.15.144:0/183732250 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc78001f020 con 0x7fc790074bf0 2024-03-26T18:49:29.046 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:28 smithi097 ceph-mon[31955]: Deploying daemon mon.b on smithi105 2024-03-26T18:49:29.046 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:29.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:28 smithi144 ceph-mon[36053]: Deploying daemon mon.b on smithi105 2024-03-26T18:49:29.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:29.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.310+0000 7fc7985ee700 1 -- 172.21.15.144:0/183732250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc79002dc00 con 0x7fc790074bf0 2024-03-26T18:49:29.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.311+0000 7fc7877fe700 1 -- 172.21.15.144:0/183732250 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fc780018570 con 0x7fc790074bf0 2024-03-26T18:49:29.303 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:49:29.303 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":2,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","modified":"2024-03-26T18:49:22.674308Z","created":"2024-03-26T18:47:19.215409Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T18:49:29.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 -- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc774038fc0 msgr2=0x7fc77403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:29.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc774038fc0 0x7fc77403b480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc78801c040 tx=0x7fc788005040 comp rx=0 tx=0).stop 2024-03-26T18:49:29.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 -- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc790074bf0 msgr2=0x7fc790075070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:29.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc790074bf0 0x7fc790075070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc78000b4d0 tx=0x7fc78000b890 comp rx=0 tx=0).stop 2024-03-26T18:49:29.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 -- 172.21.15.144:0/183732250 shutdown_connections 2024-03-26T18:49:29.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc774038fc0 0x7fc77403b480 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:29.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc77c0035c0 0x7fc7900746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:29.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 --2- 172.21.15.144:0/183732250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc790074bf0 0x7fc790075070 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:29.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 -- 172.21.15.144:0/183732250 >> 172.21.15.144:0/183732250 conn(0x7fc7900fd9a0 msgr2=0x7fc790100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:29.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 -- 172.21.15.144:0/183732250 shutdown_connections 2024-03-26T18:49:29.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:29.313+0000 7fc7985ee700 1 -- 172.21.15.144:0/183732250 wait complete. 2024-03-26T18:49:29.306 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 2 2024-03-26T18:49:29.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:29 smithi144 ceph-mon[36053]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:29.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:29 smithi144 ceph-mon[36053]: from='client.? 172.21.15.144:0/183732250' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T18:49:30.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:29 smithi097 ceph-mon[31955]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:30.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:29 smithi097 ceph-mon[31955]: from='client.? 172.21.15.144:0/183732250' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T18:49:30.102 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:49:29 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:49:29.675+0000 7f4743000700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-26T18:49:30.918 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T18:49:30.918 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T18:49:31.161 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:49:31.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:31 smithi105 systemd[1]: Started Ceph mon.b for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:49:37.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.008+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3097577797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe32c0057d0 msgr2=0x7fe32c007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:37.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.008+0000 7fe34a7c0700 1 --2- 172.21.15.144:0/3097577797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe32c0057d0 0x7fe32c007c60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe344100cb0 tx=0x7fe33400fe60 comp rx=0 tx=0).stop 2024-03-26T18:49:37.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.008+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3097577797 shutdown_connections 2024-03-26T18:49:37.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.008+0000 7fe34a7c0700 1 --2- 172.21.15.144:0/3097577797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe32c0057d0 0x7fe32c007c60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:37.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.008+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3097577797 >> 172.21.15.144:0/3097577797 conn(0x7fe3440fc850 msgr2=0x7fe3440fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:37.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.009+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3097577797 shutdown_connections 2024-03-26T18:49:37.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.009+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3097577797 wait complete. 2024-03-26T18:49:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.009+0000 7fe34a7c0700 1 Processor -- start 2024-03-26T18:49:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe34a7c0700 1 -- start start 2024-03-26T18:49:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe34a7c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c0057d0 0x7fe344199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe34a7c0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34419a270 0x7fe34419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe34a7c0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34419eb40 0x7fe34419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe34a7c0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe344107de0 con 0x7fe34419eb40 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe34a7c0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe344107c60 con 0x7fe34419a270 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe34a7c0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe344107f60 con 0x7fe32c0057d0 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe343fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c0057d0 0x7fe344199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe343fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c0057d0 0x7fe344199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44820/0 (socket says 172.21.15.144:44820) 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe343fff700 1 -- 172.21.15.144:0/3801718246 learned_addr learned my addr 172.21.15.144:0/3801718246 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.010+0000 7fe3437fe700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34419a270 0x7fe34419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.011+0000 7fe343fff700 1 -- 172.21.15.144:0/3801718246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c0057d0 msgr2=0x7fe344199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.011+0000 7fe343fff700 1 -- 172.21.15.144:0/3801718246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c0057d0 msgr2=0x7fe344199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.011+0000 7fe343fff700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c0057d0 0x7fe344199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.011+0000 7fe3437fe700 1 -- 172.21.15.144:0/3801718246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34419a270 msgr2=0x7fe34419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.011+0000 7fe3437fe700 1 -- 172.21.15.144:0/3801718246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34419a270 msgr2=0x7fe34419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T18:49:37.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.011+0000 7fe3437fe700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34419a270 0x7fe34419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T18:49:37.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.011+0000 7fe343fff700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c0057d0 0x7fe344199d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:49:37.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.011+0000 7fe3437fe700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34419a270 0x7fe34419e600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:49:37.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.011+0000 7fe348d5d700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34419eb40 0x7fe34419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:37.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.012+0000 7fe348d5d700 1 -- 172.21.15.144:0/3801718246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c0057d0 msgr2=0x7fe344199d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:49:37.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.012+0000 7fe348d5d700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c0057d0 0x7fe344199d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:37.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.012+0000 7fe348d5d700 1 -- 172.21.15.144:0/3801718246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34419a270 msgr2=0x7fe34419e600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:49:37.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.012+0000 7fe348d5d700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34419a270 0x7fe34419e600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:37.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.012+0000 7fe348d5d700 1 -- 172.21.15.144:0/3801718246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3441a68b0 con 0x7fe34419eb40 2024-03-26T18:49:37.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.012+0000 7fe348d5d700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34419eb40 0x7fe34419eff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00bd50 tx=0x7fe33c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:37.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.012+0000 7fe3417fa700 1 -- 172.21.15.144:0/3801718246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c019070 con 0x7fe34419eb40 2024-03-26T18:49:37.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.012+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3801718246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3441a6ae0 con 0x7fe34419eb40 2024-03-26T18:49:37.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.012+0000 7fe3417fa700 1 -- 172.21.15.144:0/3801718246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe33c00cd10 con 0x7fe34419eb40 2024-03-26T18:49:37.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.012+0000 7fe3417fa700 1 -- 172.21.15.144:0/3801718246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0127e0 con 0x7fe34419eb40 2024-03-26T18:49:37.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.013+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3801718246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3441a7020 con 0x7fe34419eb40 2024-03-26T18:49:37.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.014+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3801718246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3440639e0 con 0x7fe34419eb40 2024-03-26T18:49:37.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.014+0000 7fe3417fa700 1 -- 172.21.15.144:0/3801718246 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fe33c01c070 con 0x7fe34419eb40 2024-03-26T18:49:37.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.014+0000 7fe3417fa700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe3300396b0 0x7fe33003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:37.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.014+0000 7fe3417fa700 1 -- 172.21.15.144:0/3801718246 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe33c04bbf0 con 0x7fe34419eb40 2024-03-26T18:49:37.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.015+0000 7fe343fff700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe3300396b0 0x7fe33003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:37.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.016+0000 7fe343fff700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe3300396b0 0x7fe33003bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe334005b40 tx=0x7fe334005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:37.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.030+0000 7fe3417fa700 1 -- 172.21.15.144:0/3801718246 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe33c01d3b0 con 0x7fe34419eb40 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: mon.c calling monitor election 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: mon.a calling monitor election 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:37.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: monmap epoch 3 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: last_changed 2024-03-26T18:49:31.748792+0000 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: created 2024-03-26T18:47:19.215409+0000 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: min_mon_release 19 (squid) 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: election_strategy: 1 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: fsmap 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: mgrmap e13: x(active, since 67s) 2024-03-26T18:49:37.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: overall HEALTH_OK 2024-03-26T18:49:37.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:37.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:37.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:37.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:37.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: mon.c calling monitor election 2024-03-26T18:49:37.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:49:37.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: mon.a calling monitor election 2024-03-26T18:49:37.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:37.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: monmap epoch 3 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: last_changed 2024-03-26T18:49:31.748792+0000 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: created 2024-03-26T18:47:19.215409+0000 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: min_mon_release 19 (squid) 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: election_strategy: 1 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-26T18:49:37.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T18:49:37.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-26T18:49:37.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: fsmap 2024-03-26T18:49:37.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T18:49:37.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: mgrmap e13: x(active, since 67s) 2024-03-26T18:49:37.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: overall HEALTH_OK 2024-03-26T18:49:37.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:37.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:37.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:37.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:37.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.323+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3801718246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe34402dc00 con 0x7fe34419eb40 2024-03-26T18:49:37.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.324+0000 7fe3417fa700 1 -- 172.21.15.144:0/3801718246 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fe33c010200 con 0x7fe34419eb40 2024-03-26T18:49:37.315 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T18:49:37.316 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":3,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","modified":"2024-03-26T18:49:31.748792Z","created":"2024-03-26T18:47:19.215409Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe3300396b0 msgr2=0x7fe33003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe3300396b0 0x7fe33003bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe334005b40 tx=0x7fe334005a90 comp rx=0 tx=0).stop 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34419eb40 msgr2=0x7fe34419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34419eb40 0x7fe34419eff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00bd50 tx=0x7fe33c00be30 comp rx=0 tx=0).stop 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3801718246 shutdown_connections 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34419eb40 0x7fe34419eff0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe3300396b0 0x7fe33003bb70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c0057d0 0x7fe344199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 --2- 172.21.15.144:0/3801718246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe34419a270 0x7fe34419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3801718246 >> 172.21.15.144:0/3801718246 conn(0x7fe3440fc850 msgr2=0x7fe344107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3801718246 shutdown_connections 2024-03-26T18:49:37.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:37.326+0000 7fe34a7c0700 1 -- 172.21.15.144:0/3801718246 wait complete. 2024-03-26T18:49:37.319 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 3 2024-03-26T18:49:37.926 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:37 smithi144 ceph-mon[36053]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:37.926 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:37.926 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:37 smithi144 ceph-mon[36053]: from='client.? 172.21.15.144:0/3801718246' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T18:49:37.926 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:37.953 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-26T18:49:37.953 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-26T18:49:38.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:37 smithi097 ceph-mon[31955]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:38.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:38.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:37 smithi097 ceph-mon[31955]: from='client.? 172.21.15.144:0/3801718246' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T18:49:38.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:38.373 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:49:39.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: mon.c calling monitor election 2024-03-26T18:49:39.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:49:39.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: mon.a calling monitor election 2024-03-26T18:49:39.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: monmap epoch 3 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: last_changed 2024-03-26T18:49:31.748792+0000 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: created 2024-03-26T18:47:19.215409+0000 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: min_mon_release 19 (squid) 2024-03-26T18:49:39.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: election_strategy: 1 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: fsmap 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: mgrmap e13: x(active, since 67s) 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: overall HEALTH_OK 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='client.? 172.21.15.144:0/3801718246' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T18:49:39.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:39.964 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: mon.b calling monitor election 2024-03-26T18:49:39.964 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: mon.b calling monitor election 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: mon.c calling monitor election 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: mon.a calling monitor election 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: monmap epoch 3 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: last_changed 2024-03-26T18:49:31.748792+0000 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: created 2024-03-26T18:47:19.215409+0000 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: min_mon_release 19 (squid) 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: election_strategy: 1 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-26T18:49:39.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: fsmap 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: mgrmap e13: x(active, since 70s) 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: overall HEALTH_OK 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: Updating smithi105:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: Updating smithi144:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:39.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:40.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: mon.b calling monitor election 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: mon.b calling monitor election 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: mon.c calling monitor election 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: mon.a calling monitor election 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: monmap epoch 3 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: last_changed 2024-03-26T18:49:31.748792+0000 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: created 2024-03-26T18:47:19.215409+0000 2024-03-26T18:49:40.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: min_mon_release 19 (squid) 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: election_strategy: 1 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: fsmap 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: mgrmap e13: x(active, since 70s) 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: overall HEALTH_OK 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: Updating smithi105:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: Updating smithi144:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:40.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:40.128 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:40.128 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:40.128 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:40.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: mon.b calling monitor election 2024-03-26T18:49:40.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: mon.b calling monitor election 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: mon.c calling monitor election 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: mon.a calling monitor election 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: monmap epoch 3 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: last_changed 2024-03-26T18:49:31.748792+0000 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: created 2024-03-26T18:47:19.215409+0000 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: min_mon_release 19 (squid) 2024-03-26T18:49:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: election_strategy: 1 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: fsmap 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: mgrmap e13: x(active, since 70s) 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: overall HEALTH_OK 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: Updating smithi105:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: Updating smithi144:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:49:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.463+0000 7fa16ec00700 1 -- 172.21.15.97:0/599476717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa168074750 msgr2=0x7fa168074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.463+0000 7fa16ec00700 1 --2- 172.21.15.97:0/599476717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa168074750 0x7fa168074b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa158009910 tx=0x7fa158009c20 comp rx=0 tx=0).stop 2024-03-26T18:49:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.463+0000 7fa16ec00700 1 -- 172.21.15.97:0/599476717 shutdown_connections 2024-03-26T18:49:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.463+0000 7fa16ec00700 1 --2- 172.21.15.97:0/599476717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa168074750 0x7fa168074b30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.463+0000 7fa16ec00700 1 -- 172.21.15.97:0/599476717 >> 172.21.15.97:0/599476717 conn(0x7fa1680fd890 msgr2=0x7fa1680ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.463+0000 7fa16ec00700 1 -- 172.21.15.97:0/599476717 shutdown_connections 2024-03-26T18:49:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.463+0000 7fa16ec00700 1 -- 172.21.15.97:0/599476717 wait complete. 2024-03-26T18:49:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.465+0000 7fa16ec00700 1 Processor -- start 2024-03-26T18:49:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.465+0000 7fa16ec00700 1 -- start start 2024-03-26T18:49:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.465+0000 7fa16ec00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168074750 0x7fa16819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:40.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa16ec00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16819e690 0x7fa1681a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:40.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa16ec00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1681a2f60 0x7fa1681a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:40.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa16ec00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa1681129d0 con 0x7fa16819e690 2024-03-26T18:49:40.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa16ec00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa168112850 con 0x7fa168074750 2024-03-26T18:49:40.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa16ec00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa168112b50 con 0x7fa1681a2f60 2024-03-26T18:49:40.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa167400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16819e690 0x7fa1681a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:40.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa167400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16819e690 0x7fa1681a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56854/0 (socket says 172.21.15.97:56854) 2024-03-26T18:49:40.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa167400700 1 -- 172.21.15.97:0/684470135 learned_addr learned my addr 172.21.15.97:0/684470135 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:49:40.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa167e00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168074750 0x7fa16819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:40.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa16cc00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1681a2f60 0x7fa1681a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:40.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa167e00700 1 -- 172.21.15.97:0/684470135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1681a2f60 msgr2=0x7fa1681a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:40.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.466+0000 7fa167e00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1681a2f60 0x7fa1681a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:40.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.467+0000 7fa167e00700 1 -- 172.21.15.97:0/684470135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16819e690 msgr2=0x7fa1681a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.467+0000 7fa167e00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16819e690 0x7fa1681a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.467+0000 7fa167e00700 1 -- 172.21.15.97:0/684470135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1681aacd0 con 0x7fa168074750 2024-03-26T18:49:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.467+0000 7fa167400700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16819e690 0x7fa1681a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:49:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.467+0000 7fa167e00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168074750 0x7fa16819e150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa158009910 tx=0x7fa15800bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.467+0000 7fa164c00700 1 -- 172.21.15.97:0/684470135 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158026070 con 0x7fa168074750 2024-03-26T18:49:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.467+0000 7fa164c00700 1 -- 172.21.15.97:0/684470135 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1580049e0 con 0x7fa168074750 2024-03-26T18:49:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.467+0000 7fa164c00700 1 -- 172.21.15.97:0/684470135 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1580038e0 con 0x7fa168074750 2024-03-26T18:49:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.467+0000 7fa16ec00700 1 -- 172.21.15.97:0/684470135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1681aaea0 con 0x7fa168074750 2024-03-26T18:49:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.468+0000 7fa16ec00700 1 -- 172.21.15.97:0/684470135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1681ab240 con 0x7fa168074750 2024-03-26T18:49:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.469+0000 7fa164c00700 1 -- 172.21.15.97:0/684470135 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fa158029070 con 0x7fa168074750 2024-03-26T18:49:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.469+0000 7fa16ec00700 1 -- 172.21.15.97:0/684470135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1680639e0 con 0x7fa168074750 2024-03-26T18:49:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.470+0000 7fa164c00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa14c0396b0 0x7fa14c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.470+0000 7fa164c00700 1 -- 172.21.15.97:0/684470135 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa15804cf00 con 0x7fa168074750 2024-03-26T18:49:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.470+0000 7fa167400700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa14c0396b0 0x7fa14c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.470+0000 7fa167400700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa14c0396b0 0x7fa14c03bb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa154009f70 tx=0x7fa1540092c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:40.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.475+0000 7fa164c00700 1 -- 172.21.15.97:0/684470135 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa158013070 con 0x7fa168074750 2024-03-26T18:49:40.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.697+0000 7fa16ec00700 1 -- 172.21.15.97:0/684470135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa1681ab6d0 con 0x7fa168074750 2024-03-26T18:49:40.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.698+0000 7fa164c00700 1 -- 172.21.15.97:0/684470135 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fa158030020 con 0x7fa168074750 2024-03-26T18:49:40.691 INFO:teuthology.orchestra.run.smithi097.stdout:# minimal ceph.conf for 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:40.691 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-03-26T18:49:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:49:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: mon_host = [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] 2024-03-26T18:49:40.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 -- 172.21.15.97:0/684470135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa14c0396b0 msgr2=0x7fa14c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa14c0396b0 0x7fa14c03bb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa154009f70 tx=0x7fa1540092c0 comp rx=0 tx=0).stop 2024-03-26T18:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 -- 172.21.15.97:0/684470135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168074750 msgr2=0x7fa16819e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168074750 0x7fa16819e150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa158009910 tx=0x7fa15800bde0 comp rx=0 tx=0).stop 2024-03-26T18:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 -- 172.21.15.97:0/684470135 shutdown_connections 2024-03-26T18:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa14c0396b0 0x7fa14c03bb70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1681a2f60 0x7fa1681a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:40.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168074750 0x7fa16819e150 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:40.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 --2- 172.21.15.97:0/684470135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa16819e690 0x7fa1681a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:40.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 -- 172.21.15.97:0/684470135 >> 172.21.15.97:0/684470135 conn(0x7fa1680fd890 msgr2=0x7fa1680fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:40.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 -- 172.21.15.97:0/684470135 shutdown_connections 2024-03-26T18:49:40.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:40.701+0000 7fa16ec00700 1 -- 172.21.15.97:0/684470135 wait complete. 2024-03-26T18:49:41.321 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: Updating smithi097:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:41.321 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.321 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.321 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.321 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.321 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.321 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.321 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.322 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:41.322 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T18:49:41.322 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:41.322 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:41 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/684470135' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:41.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: Updating smithi097:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:41.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:41.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T18:49:41.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:41.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:41 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/684470135' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:41.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: Updating smithi097:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:49:41.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T18:49:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:41 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/684470135' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:41.469 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-26T18:49:41.469 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:49:41.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T18:49:41.565 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:49:41.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:49:41.686 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:49:41.687 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T18:49:41.721 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:49:41.722 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:49:41.791 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:49:41.791 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T18:49:41.822 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:49:41.822 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:49:41.890 INFO:tasks.cephadm:Adding mgr.x on smithi097 2024-03-26T18:49:41.890 INFO:tasks.cephadm:Adding mgr.y on smithi105 2024-03-26T18:49:41.890 INFO:tasks.cephadm:Adding mgr.z on smithi144 2024-03-26T18:49:41.890 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi097=x;smithi105=y;smithi144=z' 2024-03-26T18:49:42.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:42 smithi144 ceph-mon[36053]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T18:49:42.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:42 smithi144 ceph-mon[36053]: Reconfiguring daemon mon.a on smithi097 2024-03-26T18:49:42.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:42 smithi144 ceph-mon[36053]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:42.160 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:49:42.301 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:42 smithi097 ceph-mon[31955]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T18:49:42.301 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:42 smithi097 ceph-mon[31955]: Reconfiguring daemon mon.a on smithi097 2024-03-26T18:49:42.301 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:42 smithi097 ceph-mon[31955]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:42.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:42 smithi105 ceph-mon[36270]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T18:49:42.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:42 smithi105 ceph-mon[36270]: Reconfiguring daemon mon.a on smithi097 2024-03-26T18:49:42.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:42 smithi105 ceph-mon[36270]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:43.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.825+0000 7f663e618700 1 -- 172.21.15.144:0/196871550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6638100e10 msgr2=0x7f6638101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:43.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.825+0000 7f663e618700 1 --2- 172.21.15.144:0/196871550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6638100e10 0x7f6638101230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6628009910 tx=0x7f6628009c20 comp rx=0 tx=0).stop 2024-03-26T18:49:43.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.826+0000 7f663e618700 1 -- 172.21.15.144:0/196871550 shutdown_connections 2024-03-26T18:49:43.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.826+0000 7f663e618700 1 --2- 172.21.15.144:0/196871550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6638100e10 0x7f6638101230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:43.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.826+0000 7f663e618700 1 -- 172.21.15.144:0/196871550 >> 172.21.15.144:0/196871550 conn(0x7f66380fc3b0 msgr2=0x7f66380fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:43.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.826+0000 7f663e618700 1 -- 172.21.15.144:0/196871550 shutdown_connections 2024-03-26T18:49:43.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.826+0000 7f663e618700 1 -- 172.21.15.144:0/196871550 wait complete. 2024-03-26T18:49:43.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.826+0000 7f663e618700 1 Processor -- start 2024-03-26T18:49:43.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.826+0000 7f663e618700 1 -- start start 2024-03-26T18:49:43.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f663e618700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6638100e10 0x7f663806b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f663e618700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f663819a4d0 0x7f663819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f663e618700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663819ce00 0x7f663819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f663e618700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6638106e20 con 0x7f663819a4d0 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f663e618700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6638106ca0 con 0x7f663819ce00 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f663e618700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6638106fa0 con 0x7f6638100e10 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f6637fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6638100e10 0x7f663806b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f6637fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6638100e10 0x7f663806b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56270/0 (socket says 172.21.15.144:56270) 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f6637fff700 1 -- 172.21.15.144:0/2605284804 learned_addr learned my addr 172.21.15.144:0/2605284804 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f663cbb5700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663819ce00 0x7f663819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f6637fff700 1 -- 172.21.15.144:0/2605284804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663819ce00 msgr2=0x7f663819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f6637fff700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663819ce00 0x7f663819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f6637fff700 1 -- 172.21.15.144:0/2605284804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f663819a4d0 msgr2=0x7f663819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f6637fff700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f663819a4d0 0x7f663819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f6637fff700 1 -- 172.21.15.144:0/2605284804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663819fb40 con 0x7f6638100e10 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f6637fff700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6638100e10 0x7f663806b0a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6628005950 tx=0x7f6628005030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:43.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f66357fa700 1 -- 172.21.15.144:0/2605284804 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628027070 con 0x7f6638100e10 2024-03-26T18:49:43.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f66357fa700 1 -- 172.21.15.144:0/2605284804 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662800be40 con 0x7f6638100e10 2024-03-26T18:49:43.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f66357fa700 1 -- 172.21.15.144:0/2605284804 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662800f460 con 0x7f6638100e10 2024-03-26T18:49:43.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.827+0000 7f663e618700 1 -- 172.21.15.144:0/2605284804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663819fdd0 con 0x7f6638100e10 2024-03-26T18:49:43.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.828+0000 7f663e618700 1 -- 172.21.15.144:0/2605284804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66381a02d0 con 0x7f6638100e10 2024-03-26T18:49:43.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.828+0000 7f66357fa700 1 -- 172.21.15.144:0/2605284804 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f662802a070 con 0x7f6638100e10 2024-03-26T18:49:43.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.828+0000 7f66357fa700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f66200396b0 0x7f662003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:43.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.829+0000 7f66357fa700 1 -- 172.21.15.144:0/2605284804 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f662804d3b0 con 0x7f6638100e10 2024-03-26T18:49:43.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.829+0000 7f663e618700 1 -- 172.21.15.144:0/2605284804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f663802dc00 con 0x7f6638100e10 2024-03-26T18:49:43.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.829+0000 7f66377fe700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f66200396b0 0x7f662003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:43.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.830+0000 7f66377fe700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f66200396b0 0x7f662003bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f662c0098e0 tx=0x7f662c006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:43.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:43.834+0000 7f66357fa700 1 -- 172.21.15.144:0/2605284804 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6628013070 con 0x7f6638100e10 2024-03-26T18:49:43.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:43.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:43.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:43 smithi144 ceph-mon[36053]: Reconfiguring mon.b (monmap changed)... 2024-03-26T18:49:43.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:43.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T18:49:43.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:43.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:43 smithi144 ceph-mon[36053]: Reconfiguring daemon mon.b on smithi105 2024-03-26T18:49:43.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:43 smithi144 ceph-mon[36053]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:43.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:43.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:43.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:43 smithi105 ceph-mon[36270]: Reconfiguring mon.b (monmap changed)... 2024-03-26T18:49:43.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:43.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T18:49:43.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:43.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:43 smithi105 ceph-mon[36270]: Reconfiguring daemon mon.b on smithi105 2024-03-26T18:49:43.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:43 smithi105 ceph-mon[36270]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:44.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.041+0000 7f663e618700 1 -- 172.21.15.144:0/2605284804 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi105=y;smithi144=z", "target": ["mon-mgr", ""]}) -- 0x7f66380627f0 con 0x7f66200396b0 2024-03-26T18:49:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.056+0000 7f66357fa700 1 -- 172.21.15.144:0/2605284804 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f66380627f0 con 0x7f66200396b0 2024-03-26T18:49:44.048 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mgr update... 2024-03-26T18:49:44.050 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 -- 172.21.15.144:0/2605284804 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f66200396b0 msgr2=0x7f662003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:44.050 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f66200396b0 0x7f662003bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f662c0098e0 tx=0x7f662c006dd0 comp rx=0 tx=0).stop 2024-03-26T18:49:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 -- 172.21.15.144:0/2605284804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6638100e10 msgr2=0x7f663806b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6638100e10 0x7f663806b0a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6628005950 tx=0x7f6628005030 comp rx=0 tx=0).stop 2024-03-26T18:49:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 -- 172.21.15.144:0/2605284804 shutdown_connections 2024-03-26T18:49:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663819ce00 0x7f663819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f66200396b0 0x7f662003bb70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6638100e10 0x7f663806b0a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 --2- 172.21.15.144:0/2605284804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f663819a4d0 0x7f663819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 -- 172.21.15.144:0/2605284804 >> 172.21.15.144:0/2605284804 conn(0x7f66380fc3b0 msgr2=0x7f66380fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 -- 172.21.15.144:0/2605284804 shutdown_connections 2024-03-26T18:49:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:49:44.058+0000 7f663e618700 1 -- 172.21.15.144:0/2605284804 wait complete. 2024-03-26T18:49:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:43 smithi097 ceph-mon[31955]: Reconfiguring mon.b (monmap changed)... 2024-03-26T18:49:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T18:49:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:43 smithi097 ceph-mon[31955]: Reconfiguring daemon mon.b on smithi105 2024-03-26T18:49:44.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:43 smithi097 ceph-mon[31955]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:44.664 DEBUG:teuthology.orchestra.run.smithi105:mgr.y> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.y.service 2024-03-26T18:49:44.666 DEBUG:teuthology.orchestra.run.smithi144:mgr.z> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.z.service 2024-03-26T18:49:44.669 INFO:tasks.cephadm:Deploying OSDs... 2024-03-26T18:49:44.669 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:49:44.669 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T18:49:44.724 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T18:49:44.725 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-03-26T18:49:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T18:49:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:44.776 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-26T18:49:44.776 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:44.776 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:44.776 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-26 18:48:41.521899099 +0000 2024-03-26T18:49:44.776 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-26 18:42:53.762764433 +0000 2024-03-26T18:49:44.776 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-26 18:42:53.762764433 +0000 2024-03-26T18:49:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-26 18:42:53.762764433 +0000 2024-03-26T18:49:44.776 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T18:49:44.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-26T18:49:44.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-26T18:49:44.865 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000346646 s, 1.5 MB/s 2024-03-26T18:49:44.870 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T18:49:44.924 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-03-26T18:49:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T18:49:44.972 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:44.972 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-26T18:49:44.972 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:44.972 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:44.972 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-26 18:48:41.604897681 +0000 2024-03-26T18:49:44.972 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-26 18:42:54.334754518 +0000 2024-03-26T18:49:44.972 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-26 18:42:54.334754518 +0000 2024-03-26T18:49:44.973 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-26 18:42:54.334754518 +0000 2024-03-26T18:49:44.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T18:49:45.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-26T18:49:45.051 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-26T18:49:45.051 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000278898 s, 1.8 MB/s 2024-03-26T18:49:45.054 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T18:49:45.107 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-03-26T18:49:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T18:49:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:45.152 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-26T18:49:45.152 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:45.152 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:45.152 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-26 18:48:41.671896538 +0000 2024-03-26T18:49:45.152 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-26 18:42:54.906744604 +0000 2024-03-26T18:49:45.152 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-26 18:42:54.906744604 +0000 2024-03-26T18:49:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-26 18:42:54.906744604 +0000 2024-03-26T18:49:45.152 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T18:49:45.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:45 smithi144 ceph-mon[36053]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi105=y;smithi144=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:49:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:45 smithi144 ceph-mon[36053]: Saving service mgr spec with placement smithi097=x;smithi105=y;smithi144=z;count:3 2024-03-26T18:49:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T18:49:45.245 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:45.275 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:45 smithi097 ceph-mon[31955]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi105=y;smithi144=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:49:45.276 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:45 smithi097 ceph-mon[31955]: Saving service mgr spec with placement smithi097=x;smithi105=y;smithi144=z;count:3 2024-03-26T18:49:45.276 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:45.276 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:45.276 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:45.276 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:45.276 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T18:49:45.276 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:45.285 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-26T18:49:45.285 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-26T18:49:45.285 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000328743 s, 1.6 MB/s 2024-03-26T18:49:45.290 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T18:49:45.345 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-03-26T18:49:45.395 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T18:49:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:45.396 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-26T18:49:45.396 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:45.396 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:45.396 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-26 18:48:41.731895513 +0000 2024-03-26T18:49:45.396 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-26 18:42:55.475734742 +0000 2024-03-26T18:49:45.396 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-26 18:42:55.475734742 +0000 2024-03-26T18:49:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-26 18:42:55.475734742 +0000 2024-03-26T18:49:45.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T18:49:45.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:45 smithi105 ceph-mon[36270]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi105=y;smithi144=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:49:45.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:45 smithi105 ceph-mon[36270]: Saving service mgr spec with placement smithi097=x;smithi105=y;smithi144=z;count:3 2024-03-26T18:49:45.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:45.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:45.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:45.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T18:49:45.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T18:49:45.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-26T18:49:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-26T18:49:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00033404 s, 1.5 MB/s 2024-03-26T18:49:45.487 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T18:49:45.542 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:49:45.542 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T18:49:45.561 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T18:49:45.561 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-03-26T18:49:45.619 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T18:49:45.619 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:45.619 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-26T18:49:45.619 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:45.619 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:45.619 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-26 18:49:31.193062911 +0000 2024-03-26T18:49:45.619 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-26 18:41:39.087951146 +0000 2024-03-26T18:49:45.619 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-26 18:41:39.087951146 +0000 2024-03-26T18:49:45.619 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-26 18:41:39.087951146 +0000 2024-03-26T18:49:45.619 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T18:49:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-26T18:49:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-26T18:49:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000216801 s, 2.4 MB/s 2024-03-26T18:49:45.690 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T18:49:45.747 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-03-26T18:49:45.803 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T18:49:45.803 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:45.803 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-26T18:49:45.803 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:45.803 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:45.803 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-26 18:49:31.193062911 +0000 2024-03-26T18:49:45.803 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-26 18:41:39.422944540 +0000 2024-03-26T18:49:45.803 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-26 18:41:39.422944540 +0000 2024-03-26T18:49:45.803 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-26 18:41:39.422944540 +0000 2024-03-26T18:49:45.803 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T18:49:45.870 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-26T18:49:45.870 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-26T18:49:45.870 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000207134 s, 2.5 MB/s 2024-03-26T18:49:45.871 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T18:49:45.928 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-03-26T18:49:45.984 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T18:49:45.984 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:45.984 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-26T18:49:45.984 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:45.984 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:45.984 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-26 18:49:31.193062911 +0000 2024-03-26T18:49:45.985 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-26 18:41:39.753938014 +0000 2024-03-26T18:49:45.985 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-26 18:41:39.753938014 +0000 2024-03-26T18:49:45.985 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-26 18:41:39.753938014 +0000 2024-03-26T18:49:45.985 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T18:49:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-26T18:49:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-26T18:49:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000173073 s, 3.0 MB/s 2024-03-26T18:49:46.052 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T18:49:46.108 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-03-26T18:49:46.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: Reconfiguring mon.c (monmap changed)... 2024-03-26T18:49:46.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: Reconfiguring daemon mon.c on smithi144 2024-03-26T18:49:46.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:46.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:46.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:46.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:46.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:46.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:46.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:46.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T18:49:46.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T18:49:46.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T18:49:46.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:46.163 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T18:49:46.163 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:46.163 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-03-26T18:49:46.163 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:46.164 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:46.164 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-26 18:49:31.194062892 +0000 2024-03-26T18:49:46.164 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-26 18:41:40.100931171 +0000 2024-03-26T18:49:46.164 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-26 18:41:40.100931171 +0000 2024-03-26T18:49:46.164 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-26 18:41:40.100931171 +0000 2024-03-26T18:49:46.164 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T18:49:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-26T18:49:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-26T18:49:46.229 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000233372 s, 2.2 MB/s 2024-03-26T18:49:46.230 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T18:49:46.286 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T18:49:46.286 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T18:49:46.301 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T18:49:46.301 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-03-26T18:49:46.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: Reconfiguring mon.c (monmap changed)... 2024-03-26T18:49:46.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: Reconfiguring daemon mon.c on smithi144 2024-03-26T18:49:46.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:46.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:46.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:46.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:46.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:46.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:46.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:46.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T18:49:46.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T18:49:46.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T18:49:46.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:46.356 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T18:49:46.357 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:46.357 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-26T18:49:46.357 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:46.357 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:46.357 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-26 18:49:21.601205529 +0000 2024-03-26T18:49:46.357 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-26 18:42:13.515427965 +0000 2024-03-26T18:49:46.357 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-26 18:42:13.515427965 +0000 2024-03-26T18:49:46.357 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-26 18:42:13.515427965 +0000 2024-03-26T18:49:46.357 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T18:49:46.423 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-26T18:49:46.423 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-26T18:49:46.423 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000253165 s, 2.0 MB/s 2024-03-26T18:49:46.424 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T18:49:46.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: Reconfiguring mon.c (monmap changed)... 2024-03-26T18:49:46.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: Reconfiguring daemon mon.c on smithi144 2024-03-26T18:49:46.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:46.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:46.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:46.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:46.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:46.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T18:49:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T18:49:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T18:49:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:46.481 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-03-26T18:49:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T18:49:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:46.537 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-26T18:49:46.537 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:46.537 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:46.537 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-26 18:49:21.602205514 +0000 2024-03-26T18:49:46.537 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-26 18:42:13.852422522 +0000 2024-03-26T18:49:46.537 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-26 18:42:13.852422522 +0000 2024-03-26T18:49:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-26 18:42:13.852422522 +0000 2024-03-26T18:49:46.537 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T18:49:46.602 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-26T18:49:46.602 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-26T18:49:46.602 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000168966 s, 3.0 MB/s 2024-03-26T18:49:46.603 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T18:49:46.660 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-03-26T18:49:46.715 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T18:49:46.715 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:46.715 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-26T18:49:46.715 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:46.715 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:46.716 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-26 18:49:21.602205514 +0000 2024-03-26T18:49:46.716 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-26 18:42:14.203416852 +0000 2024-03-26T18:49:46.716 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-26 18:42:14.203416852 +0000 2024-03-26T18:49:46.716 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-26 18:42:14.203416852 +0000 2024-03-26T18:49:46.716 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T18:49:46.782 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-26T18:49:46.783 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-26T18:49:46.783 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000211766 s, 2.4 MB/s 2024-03-26T18:49:46.784 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T18:49:46.839 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-03-26T18:49:46.893 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T18:49:46.893 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T18:49:46.893 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-03-26T18:49:46.893 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T18:49:46.893 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T18:49:46.893 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-26 18:49:21.602205514 +0000 2024-03-26T18:49:46.893 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-26 18:42:14.523411683 +0000 2024-03-26T18:49:46.893 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-26 18:42:14.523411683 +0000 2024-03-26T18:49:46.894 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-26 18:42:14.523411683 +0000 2024-03-26T18:49:46.894 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T18:49:46.959 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-26T18:49:46.959 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-26T18:49:46.959 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000189586 s, 2.7 MB/s 2024-03-26T18:49:46.961 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T18:49:47.016 INFO:tasks.cephadm:Deploying osd.0 on smithi097 with /dev/vg_nvme/lv_4... 2024-03-26T18:49:47.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T18:49:47.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:47 smithi097 ceph-mon[31955]: Deploying daemon mgr.z on smithi144 2024-03-26T18:49:47.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:47 smithi144 ceph-mon[36053]: Deploying daemon mgr.z on smithi144 2024-03-26T18:49:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:49:47.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:47 smithi105 ceph-mon[36270]: Deploying daemon mgr.z on smithi144 2024-03-26T18:49:48.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:48 smithi144 ceph-mon[36053]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:48.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:48 smithi097 ceph-mon[31955]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:48.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:48 smithi105 ceph-mon[36270]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:48.525 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:48 smithi144 ceph-mgr[37934]: -- 172.21.15.144:0/2470016722 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55701f63c5a0 con 0x55701f611000 2024-03-26T18:49:48.875 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:48 smithi144 bash[37915]: 88fa8a76e5eb8ffa3e602f508f30b32b8bf5b4fb361f9d9d29fb09c9416b80d6 2024-03-26T18:49:48.875 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:48 smithi144 systemd[1]: Started Ceph mgr.z for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:49:49.375 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:49.002+0000 7fb167634200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T18:49:49.375 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:49.096+0000 7fb167634200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T18:49:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T18:49:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T18:49:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T18:49:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:49 smithi097 ceph-mon[31955]: Deploying daemon mgr.y on smithi105 2024-03-26T18:49:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:49 smithi097 ceph-mon[31955]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:49.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:49.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:49.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:49.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T18:49:49.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T18:49:49.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T18:49:49.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:49.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-mon[36053]: Deploying daemon mgr.y on smithi105 2024-03-26T18:49:49.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-mon[36053]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:49.876 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:49.732+0000 7fb167634200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T18:49:49.876 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:49.823+0000 7fb167634200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T18:49:49.880 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:49.880 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:49.880 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:49.880 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T18:49:49.880 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T18:49:49.880 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T18:49:49.880 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:49.880 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:49 smithi105 ceph-mon[36270]: Deploying daemon mgr.y on smithi105 2024-03-26T18:49:49.880 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:49 smithi105 ceph-mon[36270]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:50.324 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T18:49:50.324 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T18:49:50.324 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-26T18:49:50.324 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-26T18:49:50.325 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0349728 s, 300 MB/s 2024-03-26T18:49:50.325 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-26T18:49:50.375 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:49.964+0000 7fb167634200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T18:49:50.375 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:50.056+0000 7fb167634200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T18:49:50.375 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:50.139+0000 7fb167634200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T18:49:50.677 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:50.399+0000 7fb167634200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T18:49:50.677 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:50.506+0000 7fb167634200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T18:49:50.677 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:50.589+0000 7fb167634200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T18:49:50.677 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:50.685+0000 7fb167634200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T18:49:50.709 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:50 smithi105 systemd[1]: Starting Ceph mgr.y for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:49:51.087 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:50 smithi105 podman[37643]: 2024-03-26 18:49:50.736052577 +0000 UTC m=+0.017805737 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:49:51.088 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:50 smithi105 podman[37643]: 2024-03-26 18:49:50.850179712 +0000 UTC m=+0.131932851 container create 201c4d33d4dbf42f9ca88e2960713facb67690f6fb117acf452409a2aeca77ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2) 2024-03-26T18:49:51.088 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:51 smithi105 podman[37643]: 2024-03-26 18:49:51.069188228 +0000 UTC m=+0.350941382 container init 201c4d33d4dbf42f9ca88e2960713facb67690f6fb117acf452409a2aeca77ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-26T18:49:51.088 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:51 smithi105 podman[37643]: 2024-03-26 18:49:51.071944726 +0000 UTC m=+0.353697858 container start 201c4d33d4dbf42f9ca88e2960713facb67690f6fb117acf452409a2aeca77ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-26T18:49:51.125 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:50.862+0000 7fb167634200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T18:49:51.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-03-26T18:49:51.459 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:51 smithi105 ceph-mgr[37662]: -- 172.21.15.105:0/3809244322 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631105fa5a0 con 0x5631105cf000 2024-03-26T18:49:51.459 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:51 smithi105 bash[37643]: 201c4d33d4dbf42f9ca88e2960713facb67690f6fb117acf452409a2aeca77ad 2024-03-26T18:49:51.459 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:51 smithi105 systemd[1]: Started Ceph mgr.y for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:49:51.593 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:49:51.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:51 smithi105 ceph-mon[36270]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:51.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:51 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:51.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:51 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:51.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:51 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:51.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:51 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:51.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:51 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:51.960 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:51 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:51.627+0000 7f32bba99200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T18:49:51.960 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:51 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:51.725+0000 7f32bba99200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T18:49:52.023 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:51 smithi097 ceph-mon[31955]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:52.023 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:51 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:52.023 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:51 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:52.023 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:51 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:52.023 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:51 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:52.024 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:51 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:52.125 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:51.842+0000 7fb167634200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T18:49:52.125 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:51.982+0000 7fb167634200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T18:49:52.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:51 smithi144 ceph-mon[36053]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:52.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:51 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:52.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:51 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:52.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:51 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:52.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:51 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:52.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:51 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:52.620 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:52.339+0000 7fb167634200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T18:49:52.620 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:52.432+0000 7fb167634200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T18:49:52.620 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:52.627+0000 7fb167634200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:49:52.647 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:52.421+0000 7f32bba99200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T18:49:52.647 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:52.512+0000 7f32bba99200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T18:49:52.875 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:52.815+0000 7fb167634200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T18:49:52.959 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:52.655+0000 7f32bba99200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T18:49:52.959 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:52.754+0000 7f32bba99200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T18:49:52.959 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:52.835+0000 7f32bba99200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T18:49:53.361 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:53.105+0000 7fb167634200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:49:53.361 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:53.369+0000 7fb167634200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T18:49:53.373 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:53.100+0000 7f32bba99200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T18:49:53.373 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:53.206+0000 7f32bba99200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T18:49:53.373 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:53.289+0000 7f32bba99200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T18:49:53.373 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:53.381+0000 7f32bba99200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T18:49:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.544+0000 7fb1a8400700 1 -- 172.21.15.97:0/1956402226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a0074e10 msgr2=0x7fb1a0075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.544+0000 7fb1a8400700 1 --2- 172.21.15.97:0/1956402226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a0074e10 0x7fb1a0075290 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009990 tx=0x7fb18c009ca0 comp rx=0 tx=0).stop 2024-03-26T18:49:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.544+0000 7fb1a8400700 1 -- 172.21.15.97:0/1956402226 shutdown_connections 2024-03-26T18:49:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.544+0000 7fb1a8400700 1 --2- 172.21.15.97:0/1956402226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1a010a2c0 0x7fb1a010c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.544+0000 7fb1a8400700 1 --2- 172.21.15.97:0/1956402226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a00763e0 0x7fb1a0074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.544+0000 7fb1a8400700 1 --2- 172.21.15.97:0/1956402226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a0074e10 0x7fb1a0075290 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.544+0000 7fb1a8400700 1 -- 172.21.15.97:0/1956402226 >> 172.21.15.97:0/1956402226 conn(0x7fb1a00fd950 msgr2=0x7fb1a00ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:49:53.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.544+0000 7fb1a8400700 1 -- 172.21.15.97:0/1956402226 shutdown_connections 2024-03-26T18:49:53.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.544+0000 7fb1a8400700 1 -- 172.21.15.97:0/1956402226 wait complete. 2024-03-26T18:49:53.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.546+0000 7fb1a8400700 1 Processor -- start 2024-03-26T18:49:53.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.546+0000 7fb1a8400700 1 -- start start 2024-03-26T18:49:53.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.546+0000 7fb1a8400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a0074e10 0x7fb1a019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:53.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.546+0000 7fb1a8400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a00763e0 0x7fb1a019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:53.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.546+0000 7fb1a8400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1a010a2c0 0x7fb1a01a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:53.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.546+0000 7fb1a8400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a0110240 con 0x7fb1a0074e10 2024-03-26T18:49:53.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.546+0000 7fb1a8400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01100c0 con 0x7fb1a00763e0 2024-03-26T18:49:53.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.546+0000 7fb1a8400700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01103c0 con 0x7fb1a010a2c0 2024-03-26T18:49:53.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a5000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a00763e0 0x7fb1a019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:53.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a6400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1a010a2c0 0x7fb1a01a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:53.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a5a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a0074e10 0x7fb1a019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:53.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a5000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a00763e0 0x7fb1a019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:47950/0 (socket says 172.21.15.97:47950) 2024-03-26T18:49:53.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a6400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1a010a2c0 0x7fb1a01a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52256/0 (socket says 172.21.15.97:52256) 2024-03-26T18:49:53.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a5000700 1 -- 172.21.15.97:0/4048792597 learned_addr learned my addr 172.21.15.97:0/4048792597 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:49:53.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a5000700 1 -- 172.21.15.97:0/4048792597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1a010a2c0 msgr2=0x7fb1a01a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:53.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a5000700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1a010a2c0 0x7fb1a01a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:53.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a5000700 1 -- 172.21.15.97:0/4048792597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a0074e10 msgr2=0x7fb1a019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:49:53.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a5000700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a0074e10 0x7fb1a019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:49:53.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a5000700 1 -- 172.21.15.97:0/4048792597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a01a24c0 con 0x7fb1a00763e0 2024-03-26T18:49:53.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a6400700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1a010a2c0 0x7fb1a01a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:49:53.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.547+0000 7fb1a5000700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a00763e0 0x7fb1a019ada0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b7b0 tx=0x7fb18c00bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:53.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.548+0000 7fb19e000700 1 -- 172.21.15.97:0/4048792597 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c026070 con 0x7fb1a00763e0 2024-03-26T18:49:53.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.548+0000 7fb19e000700 1 -- 172.21.15.97:0/4048792597 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c00f460 con 0x7fb1a00763e0 2024-03-26T18:49:53.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.548+0000 7fb1a8400700 1 -- 172.21.15.97:0/4048792597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a01a2750 con 0x7fb1a00763e0 2024-03-26T18:49:53.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.548+0000 7fb19e000700 1 -- 172.21.15.97:0/4048792597 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c021670 con 0x7fb1a00763e0 2024-03-26T18:49:53.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.548+0000 7fb1a8400700 1 -- 172.21.15.97:0/4048792597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a01a2c00 con 0x7fb1a00763e0 2024-03-26T18:49:53.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.549+0000 7fb19e000700 1 -- 172.21.15.97:0/4048792597 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fb18c029070 con 0x7fb1a00763e0 2024-03-26T18:49:53.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.549+0000 7fb19e000700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb188039460 0x7fb18803b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:49:53.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.549+0000 7fb1a8400700 1 -- 172.21.15.97:0/4048792597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a0069070 con 0x7fb1a00763e0 2024-03-26T18:49:53.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.550+0000 7fb19e000700 1 -- 172.21.15.97:0/4048792597 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb18c04d310 con 0x7fb1a00763e0 2024-03-26T18:49:53.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.550+0000 7fb1a5a00700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb188039460 0x7fb18803b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:49:53.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.550+0000 7fb1a5a00700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb188039460 0x7fb18803b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00767c0 tx=0x7fb194009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:49:53.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.559+0000 7fb19e000700 1 -- 172.21.15.97:0/4048792597 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb18c013070 con 0x7fb1a00763e0 2024-03-26T18:49:53.625 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:53.603+0000 7fb167634200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T18:49:53.697 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:53.554+0000 7f32bba99200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T18:49:53.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:53.794+0000 7fb1a8400700 1 -- 172.21.15.97:0/4048792597 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb1a00628b0 con 0x7fb188039460 2024-03-26T18:49:53.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:53.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:53.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:53.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:53.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:53.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:53.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:53.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:53.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T18:49:53.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T18:49:53.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:53.981 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:53.981 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:53.981 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:53.981 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:53.981 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:53.982 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:53.982 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:53.982 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:53.982 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T18:49:53.982 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T18:49:53.982 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:54.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:54.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:54.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:54.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:54.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:54.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:54.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:54.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:54.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T18:49:54.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T18:49:54.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:54.708 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:54 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:54.397+0000 7fb167634200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T18:49:54.958 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:54 smithi144 ceph-mon[36053]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T18:49:54.959 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:54 smithi144 ceph-mon[36053]: Reconfiguring daemon mgr.x on smithi097 2024-03-26T18:49:54.959 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:54 smithi144 ceph-mon[36053]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:49:54.959 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:54 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:49:54.959 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:54 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:49:54.959 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:54 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:54.959 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:54 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:54.959 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:54 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:54.515+0000 7f32bba99200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T18:49:54.960 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:54 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:54.641+0000 7f32bba99200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T18:49:54.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:54 smithi105 ceph-mon[36270]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T18:49:54.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:54 smithi105 ceph-mon[36270]: Reconfiguring daemon mgr.x on smithi097 2024-03-26T18:49:54.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:54 smithi105 ceph-mon[36270]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:49:54.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:54 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:49:54.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:54 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:49:54.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:54 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:54.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:54 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:55.046 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:54 smithi097 ceph-mon[31955]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T18:49:55.046 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:54 smithi097 ceph-mon[31955]: Reconfiguring daemon mgr.x on smithi097 2024-03-26T18:49:55.046 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:54 smithi097 ceph-mon[31955]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:49:55.046 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:54 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:49:55.047 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:54 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:49:55.047 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:54 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:55.047 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:54 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:55.236 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:54 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:54.969+0000 7f32bba99200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T18:49:55.236 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:55.055+0000 7f32bba99200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T18:49:55.236 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:55.244+0000 7f32bba99200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:49:55.269 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:54 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:54.966+0000 7fb167634200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T18:49:55.269 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:55.063+0000 7fb167634200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T18:49:55.269 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z[37930]: 2024-03-26T18:49:55.277+0000 7fb167634200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T18:49:55.695 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:55.426+0000 7f32bba99200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T18:49:55.950 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:55.703+0000 7f32bba99200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:55.958+0000 7f32bba99200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: Standby manager daemon z started 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:55.951 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:55.952 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:55.952 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:56.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:56.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T18:49:56.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:56.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:49:56.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: Standby manager daemon z started 2024-03-26T18:49:56.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T18:49:56.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:49:56.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:56.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:56.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:56.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:56.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T18:49:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:49:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: Standby manager daemon z started 2024-03-26T18:49:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T18:49:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: from='mgr.? 172.21.15.144:0/3312964196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:49:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:49:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:49:56.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:49:56.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:49:56.209 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:56 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:56.188+0000 7f32bba99200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T18:49:56.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:56.300+0000 7fb19e000700 1 -- 172.21.15.97:0/4048792597 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 14) ==== 92074+0+0 (secure 0 0 0) 0x7fb18c009d10 con 0x7fb1a00763e0 2024-03-26T18:49:57.298 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:56 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:56.972+0000 7f32bba99200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T18:49:57.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:57 smithi097 ceph-mon[31955]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-26T18:49:57.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:57 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T18:49:57.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:57 smithi097 ceph-mon[31955]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:57.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:57 smithi105 ceph-mon[36270]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-26T18:49:57.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:57 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T18:49:57.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:57 smithi105 ceph-mon[36270]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:57.616 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:57 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:57.531+0000 7f32bba99200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T18:49:57.616 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:57 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:57.624+0000 7f32bba99200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T18:49:57.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:57 smithi144 ceph-mon[36053]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-26T18:49:57.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:57 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T18:49:57.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:57 smithi144 ceph-mon[36053]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:57.959 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 18:49:57 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y[37658]: 2024-03-26T18:49:57.834+0000 7f32bba99200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T18:49:58.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:49:58.329+0000 7fb19e000700 1 -- 172.21.15.97:0/4048792597 <== mon.2 v2:172.21.15.105:3300/0 8 ==== mgrmap(e 15) ==== 137798+0+0 (secure 0 0 0) 0x7fb18c0163a0 con 0x7fb1a00763e0 2024-03-26T18:49:58.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:58 smithi097 ceph-mon[31955]: Standby manager daemon y started 2024-03-26T18:49:58.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:58 smithi097 ceph-mon[31955]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T18:49:58.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:58 smithi097 ceph-mon[31955]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:49:58.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:58 smithi097 ceph-mon[31955]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T18:49:58.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:58 smithi097 ceph-mon[31955]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:49:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:58 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2709377726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb36143c-3d96-4a50-af1b-e1c3cdbd5681"}]: dispatch 2024-03-26T18:49:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:58 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb36143c-3d96-4a50-af1b-e1c3cdbd5681"}]: dispatch 2024-03-26T18:49:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:58 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb36143c-3d96-4a50-af1b-e1c3cdbd5681"}]': finished 2024-03-26T18:49:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:58 smithi097 ceph-mon[31955]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T18:49:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:49:58.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:58 smithi144 ceph-mon[36053]: Standby manager daemon y started 2024-03-26T18:49:58.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:58 smithi144 ceph-mon[36053]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T18:49:58.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:58 smithi144 ceph-mon[36053]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:49:58.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:58 smithi144 ceph-mon[36053]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T18:49:58.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:58 smithi144 ceph-mon[36053]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:49:58.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:58 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2709377726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb36143c-3d96-4a50-af1b-e1c3cdbd5681"}]: dispatch 2024-03-26T18:49:58.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:58 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb36143c-3d96-4a50-af1b-e1c3cdbd5681"}]: dispatch 2024-03-26T18:49:58.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:58 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb36143c-3d96-4a50-af1b-e1c3cdbd5681"}]': finished 2024-03-26T18:49:58.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:58 smithi144 ceph-mon[36053]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T18:49:58.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:49:58.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:58 smithi105 ceph-mon[36270]: Standby manager daemon y started 2024-03-26T18:49:58.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:58 smithi105 ceph-mon[36270]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T18:49:58.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:58 smithi105 ceph-mon[36270]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:49:58.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:58 smithi105 ceph-mon[36270]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T18:49:58.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:58 smithi105 ceph-mon[36270]: from='mgr.? 172.21.15.105:0/2051219791' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:49:58.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:58 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2709377726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb36143c-3d96-4a50-af1b-e1c3cdbd5681"}]: dispatch 2024-03-26T18:49:58.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:58 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb36143c-3d96-4a50-af1b-e1c3cdbd5681"}]: dispatch 2024-03-26T18:49:58.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:58 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb36143c-3d96-4a50-af1b-e1c3cdbd5681"}]': finished 2024-03-26T18:49:58.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:58 smithi105 ceph-mon[36270]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T18:49:58.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:49:59.601 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:59 smithi097 ceph-mon[31955]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-03-26T18:49:59.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:59 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T18:49:59.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:59 smithi097 ceph-mon[31955]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:59.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:49:59 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1351971114' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:49:59.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:59 smithi144 ceph-mon[36053]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-03-26T18:49:59.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:59 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T18:49:59.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:59 smithi144 ceph-mon[36053]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:59.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:49:59 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1351971114' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:49:59.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:59 smithi105 ceph-mon[36270]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-03-26T18:49:59.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:59 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T18:49:59.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:59 smithi105 ceph-mon[36270]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:49:59.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:49:59 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1351971114' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:50:00.485 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:00 smithi097 ceph-mon[31955]: overall HEALTH_OK 2024-03-26T18:50:00.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:00 smithi144 ceph-mon[36053]: overall HEALTH_OK 2024-03-26T18:50:00.648 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:00 smithi105 ceph-mon[36270]: overall HEALTH_OK 2024-03-26T18:50:01.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:01 smithi097 ceph-mon[31955]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:01.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:01 smithi144 ceph-mon[36053]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:01.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:01 smithi105 ceph-mon[36270]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:04.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:03 smithi097 ceph-mon[31955]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:04.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:03 smithi144 ceph-mon[36053]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:04.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:03 smithi105 ceph-mon[36270]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:06.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:05 smithi097 ceph-mon[31955]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:06.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:05 smithi144 ceph-mon[36053]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:06.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:05 smithi105 ceph-mon[36270]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:08.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:07 smithi144 ceph-mon[36053]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:08.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T18:50:08.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:08.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:07 smithi144 ceph-mon[36053]: Deploying daemon osd.0 on smithi097 2024-03-26T18:50:08.181 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:07 smithi097 ceph-mon[31955]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:08.181 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T18:50:08.181 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:08.181 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:07 smithi097 ceph-mon[31955]: Deploying daemon osd.0 on smithi097 2024-03-26T18:50:08.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:07 smithi105 ceph-mon[36270]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:08.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T18:50:08.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:08.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:07 smithi105 ceph-mon[36270]: Deploying daemon osd.0 on smithi097 2024-03-26T18:50:10.027 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:09 smithi097 ceph-mon[31955]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:10.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:09 smithi144 ceph-mon[36053]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:10.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:09 smithi105 ceph-mon[36270]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:12.081 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:11 smithi097 ceph-mon[31955]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:12.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:11 smithi144 ceph-mon[36053]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:12.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:11 smithi105 ceph-mon[36270]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:14.101 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:13 smithi097 ceph-mon[31955]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:14.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:13 smithi144 ceph-mon[36053]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:14.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:13 smithi105 ceph-mon[36270]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:15.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:15 smithi097 ceph-mon[31955]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:15.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:15.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:15.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:15.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:15 smithi105 ceph-mon[36270]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:16.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:16.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:16.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:16.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:15 smithi144 ceph-mon[36053]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:18.007 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:17 smithi097 ceph-mon[31955]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:18.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:17 smithi144 ceph-mon[36053]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:18.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:17 smithi105 ceph-mon[36270]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:19.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:19.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:19.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:19.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:19.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:19.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:19 smithi097 ceph-mon[31955]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T18:50:19.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:19 smithi097 ceph-mon[31955]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:19.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:19.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:19.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:19.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:19.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:19.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:19 smithi144 ceph-mon[36053]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T18:50:19.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:19 smithi144 ceph-mon[36053]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:19.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.635+0000 7fb19e000700 1 -- 172.21.15.97:0/4048792597 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb1a00628b0 con 0x7fb188039460 2024-03-26T18:50:19.627 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 0 on host 'smithi097' 2024-03-26T18:50:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 -- 172.21.15.97:0/4048792597 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb188039460 msgr2=0x7fb18803b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:50:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb188039460 0x7fb18803b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00767c0 tx=0x7fb194009260 comp rx=0 tx=0).stop 2024-03-26T18:50:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 -- 172.21.15.97:0/4048792597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a00763e0 msgr2=0x7fb1a019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:50:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a00763e0 0x7fb1a019ada0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b7b0 tx=0x7fb18c00bbb0 comp rx=0 tx=0).stop 2024-03-26T18:50:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 -- 172.21.15.97:0/4048792597 shutdown_connections 2024-03-26T18:50:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb188039460 0x7fb18803b920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1a010a2c0 0x7fb1a01a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1a0074e10 0x7fb1a019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 --2- 172.21.15.97:0/4048792597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a00763e0 0x7fb1a019ada0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 -- 172.21.15.97:0/4048792597 >> 172.21.15.97:0/4048792597 conn(0x7fb1a00fd950 msgr2=0x7fb1a01061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:50:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.638+0000 7fb1a8400700 1 -- 172.21.15.97:0/4048792597 shutdown_connections 2024-03-26T18:50:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:50:19.639+0000 7fb1a8400700 1 -- 172.21.15.97:0/4048792597 wait complete. 2024-03-26T18:50:19.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:19.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:19.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:19.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:19.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:19.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:19 smithi105 ceph-mon[36270]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T18:50:19.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:19 smithi105 ceph-mon[36270]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:20.394 DEBUG:teuthology.orchestra.run.smithi097:osd.0> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.0.service 2024-03-26T18:50:20.399 INFO:tasks.cephadm:Deploying osd.1 on smithi105 with /dev/vg_nvme/lv_4... 2024-03-26T18:50:20.399 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T18:50:20.601 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:20 smithi097 ceph-mon[31955]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T18:50:20.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:20 smithi097 ceph-mon[31955]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T18:50:20.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:20 smithi097 ceph-mon[31955]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:50:20.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:20.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:20.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:20.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:20.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:20 smithi144 ceph-mon[36053]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T18:50:20.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:20 smithi144 ceph-mon[36053]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T18:50:20.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:20 smithi144 ceph-mon[36053]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:50:20.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:20.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:20.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:20.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:20.648 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:50:20.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:20 smithi105 ceph-mon[36270]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T18:50:20.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:20 smithi105 ceph-mon[36270]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T18:50:20.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:20 smithi105 ceph-mon[36270]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:50:20.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:20.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:20.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:20.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:21.195 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 18:50:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0[43972]: 2024-03-26T18:50:20.946+0000 7f4791000700 -1 osd.0 0 waiting for initial osdmap 2024-03-26T18:50:21.195 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 18:50:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0[43972]: 2024-03-26T18:50:20.959+0000 7f4786400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:50:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:21 smithi097 ceph-mon[31955]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:50:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:21 smithi097 ceph-mon[31955]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T18:50:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:21 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:21 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:21 smithi097 ceph-mon[31955]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:21 smithi097 ceph-mon[31955]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' 2024-03-26T18:50:21.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:21 smithi144 ceph-mon[36053]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:50:21.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:21 smithi144 ceph-mon[36053]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T18:50:21.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:21 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:21.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:21 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:21.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:21 smithi144 ceph-mon[36053]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:21.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:21 smithi144 ceph-mon[36053]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' 2024-03-26T18:50:21.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:21 smithi105 ceph-mon[36270]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:50:21.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:21 smithi105 ceph-mon[36270]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T18:50:21.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:21 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:21.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:21 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:21.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:21 smithi105 ceph-mon[36270]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T18:50:21.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:21 smithi105 ceph-mon[36270]: from='osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107]' entity='osd.0' 2024-03-26T18:50:22.397 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T18:50:22.397 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T18:50:22.397 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-26T18:50:22.397 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-26T18:50:22.397 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120492 s, 870 MB/s 2024-03-26T18:50:22.397 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-26T18:50:22.449 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:22 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:50:22.449 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:22 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:50:22.449 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:22 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:22.450 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:22 smithi097 ceph-mon[31955]: osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] boot 2024-03-26T18:50:22.450 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:22 smithi097 ceph-mon[31955]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T18:50:22.450 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:22 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:22.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:22 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:50:22.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:22 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:50:22.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:22.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:22 smithi144 ceph-mon[36053]: osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] boot 2024-03-26T18:50:22.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:22 smithi144 ceph-mon[36053]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T18:50:22.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:22.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:22 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:50:22.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:22 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:50:22.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:22 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:22.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:22 smithi105 ceph-mon[36270]: osd.0 [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] boot 2024-03-26T18:50:22.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:22 smithi105 ceph-mon[36270]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T18:50:22.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:22 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:50:23.026 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_4 2024-03-26T18:50:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:50:23.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:23 smithi097 ceph-mon[31955]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:23.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:23 smithi144 ceph-mon[36053]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:23.677 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:23 smithi105 ceph-mon[36270]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:24.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:24 smithi144 ceph-mon[36053]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T18:50:24.625 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:24 smithi105 ceph-mon[36270]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T18:50:24.710 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:24 smithi097 ceph-mon[31955]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T18:50:24.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.734+0000 7f4a74a9f700 1 -- 172.21.15.105:0/2240122519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a701035d0 msgr2=0x7f4a70103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:50:24.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.734+0000 7f4a74a9f700 1 --2- 172.21.15.105:0/2240122519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a701035d0 0x7f4a70103a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4a600099c0 tx=0x7f4a60009cd0 comp rx=0 tx=0).stop 2024-03-26T18:50:24.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.734+0000 7f4a74a9f700 1 -- 172.21.15.105:0/2240122519 shutdown_connections 2024-03-26T18:50:24.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.734+0000 7f4a74a9f700 1 --2- 172.21.15.105:0/2240122519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a70103f70 0x7f4a7010a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:24.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.734+0000 7f4a74a9f700 1 --2- 172.21.15.105:0/2240122519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a701023d0 0x7f4a701027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:24.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.734+0000 7f4a74a9f700 1 --2- 172.21.15.105:0/2240122519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a701035d0 0x7f4a70103a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:24.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.734+0000 7f4a74a9f700 1 -- 172.21.15.105:0/2240122519 >> 172.21.15.105:0/2240122519 conn(0x7f4a700fd990 msgr2=0x7f4a700ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:50:24.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.734+0000 7f4a74a9f700 1 -- 172.21.15.105:0/2240122519 shutdown_connections 2024-03-26T18:50:24.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.734+0000 7f4a74a9f700 1 -- 172.21.15.105:0/2240122519 wait complete. 2024-03-26T18:50:24.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a74a9f700 1 Processor -- start 2024-03-26T18:50:24.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a74a9f700 1 -- start start 2024-03-26T18:50:24.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a74a9f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a701023d0 0x7f4a70198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:50:24.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a74a9f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a701035d0 0x7f4a70198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:50:24.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a74a9f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a70103f70 0x7f4a7019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:50:24.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a74a9f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010e050 con 0x7f4a70103f70 2024-03-26T18:50:24.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a74a9f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010ded0 con 0x7f4a701035d0 2024-03-26T18:50:24.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a74a9f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010e1d0 con 0x7f4a701023d0 2024-03-26T18:50:24.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a6dd9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a701035d0 0x7f4a70198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a6dd9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a701035d0 0x7f4a70198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51834/0 (socket says 172.21.15.105:51834) 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.735+0000 7f4a6dd9b700 1 -- 172.21.15.105:0/315400778 learned_addr learned my addr 172.21.15.105:0/315400778 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a6e59c700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a701023d0 0x7f4a70198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a6dd9b700 1 -- 172.21.15.105:0/315400778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a701023d0 msgr2=0x7f4a70198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a6dd9b700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a701023d0 0x7f4a70198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a6dd9b700 1 -- 172.21.15.105:0/315400778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a70103f70 msgr2=0x7f4a7019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a6ed9d700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a70103f70 0x7f4a7019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a6dd9b700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a70103f70 0x7f4a7019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a6dd9b700 1 -- 172.21.15.105:0/315400778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a701a02d0 con 0x7f4a701035d0 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a6e59c700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a701023d0 0x7f4a70198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a6dd9b700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a701035d0 0x7f4a70198bb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4a60003890 tx=0x7f4a60003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:50:24.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a5f7fe700 1 -- 172.21.15.105:0/315400778 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60027070 con 0x7f4a701035d0 2024-03-26T18:50:24.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a74a9f700 1 -- 172.21.15.105:0/315400778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a701a0560 con 0x7f4a701035d0 2024-03-26T18:50:24.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a5f7fe700 1 -- 172.21.15.105:0/315400778 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a60022470 con 0x7f4a701035d0 2024-03-26T18:50:24.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a5f7fe700 1 -- 172.21.15.105:0/315400778 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6000f6a0 con 0x7f4a701035d0 2024-03-26T18:50:24.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.736+0000 7f4a74a9f700 1 -- 172.21.15.105:0/315400778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a701a0a90 con 0x7f4a701035d0 2024-03-26T18:50:24.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.737+0000 7f4a74a9f700 1 -- 172.21.15.105:0/315400778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a70069070 con 0x7f4a701035d0 2024-03-26T18:50:24.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.738+0000 7f4a5f7fe700 1 -- 172.21.15.105:0/315400778 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 15) ==== 137798+0+0 (secure 0 0 0) 0x7f4a6002a070 con 0x7f4a701035d0 2024-03-26T18:50:24.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.739+0000 7f4a5f7fe700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4a580a2f20 0x7f4a580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:50:24.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.739+0000 7f4a5f7fe700 1 -- 172.21.15.105:0/315400778 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f4a600ce8b0 con 0x7f4a701035d0 2024-03-26T18:50:24.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.739+0000 7f4a6e59c700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4a580a2f20 0x7f4a580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:50:24.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.740+0000 7f4a6e59c700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4a580a2f20 0x7f4a580a53e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a70103430 tx=0x7f4a64008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:50:24.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.743+0000 7f4a5f7fe700 1 -- 172.21.15.105:0/315400778 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4a60013070 con 0x7f4a701035d0 2024-03-26T18:50:24.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:24.948+0000 7f4a74a9f700 1 -- 172.21.15.105:0/315400778 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4a700627f0 con 0x7f4a580a2f20 2024-03-26T18:50:25.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:25 smithi097 ceph-mon[31955]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:25.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:25 smithi097 ceph-mon[31955]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:50:25.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:25 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:50:25.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:25 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:50:25.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:25 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:25.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:25 smithi144 ceph-mon[36053]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:25.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:25 smithi144 ceph-mon[36053]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:50:25.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:25 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:50:25.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:25 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:50:25.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:25 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:25.650 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:25 smithi105 ceph-mon[36270]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:25.650 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:25 smithi105 ceph-mon[36270]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:50:25.650 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:25 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:50:25.650 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:25 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:50:25.650 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:25 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:26.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:26 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi097 2024-03-26T18:50:26.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:50:26.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:26 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi097 to 17091M 2024-03-26T18:50:26.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:26.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:26.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:26 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi097 2024-03-26T18:50:26.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:50:26.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:26 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi097 to 17091M 2024-03-26T18:50:26.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:26.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:26.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:26 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi097 2024-03-26T18:50:26.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:50:26.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:26 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi097 to 17091M 2024-03-26T18:50:26.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:26.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:26.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:26.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:27.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:27 smithi097 ceph-mon[31955]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:27.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:27 smithi097 ceph-mon[31955]: from='client.? 172.21.15.105:0/1917460711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a895a67a-5554-4bad-ab17-6b6971466aae"}]: dispatch 2024-03-26T18:50:27.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:27 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a895a67a-5554-4bad-ab17-6b6971466aae"}]: dispatch 2024-03-26T18:50:27.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:27 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a895a67a-5554-4bad-ab17-6b6971466aae"}]': finished 2024-03-26T18:50:27.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:27 smithi097 ceph-mon[31955]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T18:50:27.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:27 smithi144 ceph-mon[36053]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:27 smithi144 ceph-mon[36053]: from='client.? 172.21.15.105:0/1917460711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a895a67a-5554-4bad-ab17-6b6971466aae"}]: dispatch 2024-03-26T18:50:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:27 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a895a67a-5554-4bad-ab17-6b6971466aae"}]: dispatch 2024-03-26T18:50:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:27 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a895a67a-5554-4bad-ab17-6b6971466aae"}]': finished 2024-03-26T18:50:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:27 smithi144 ceph-mon[36053]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T18:50:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:27.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:27 smithi105 ceph-mon[36270]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:27.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:27 smithi105 ceph-mon[36270]: from='client.? 172.21.15.105:0/1917460711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a895a67a-5554-4bad-ab17-6b6971466aae"}]: dispatch 2024-03-26T18:50:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:27 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a895a67a-5554-4bad-ab17-6b6971466aae"}]: dispatch 2024-03-26T18:50:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:27 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a895a67a-5554-4bad-ab17-6b6971466aae"}]': finished 2024-03-26T18:50:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:27 smithi105 ceph-mon[36270]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T18:50:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:28.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:28 smithi144 ceph-mon[36053]: from='client.? 172.21.15.105:0/3043416496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:50:28.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:28 smithi105 ceph-mon[36270]: from='client.? 172.21.15.105:0/3043416496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:50:29.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:28 smithi097 ceph-mon[31955]: from='client.? 172.21.15.105:0/3043416496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:50:29.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:29 smithi144 ceph-mon[36053]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:29.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:29 smithi105 ceph-mon[36270]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:30.006 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:29 smithi097 ceph-mon[31955]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:31.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:31 smithi105 ceph-mon[36270]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:32.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:31 smithi097 ceph-mon[31955]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:32.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:31 smithi144 ceph-mon[36053]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:34.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:33 smithi144 ceph-mon[36053]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:34.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:33 smithi105 ceph-mon[36270]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:34.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:33 smithi097 ceph-mon[31955]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:35.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T18:50:35.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:35.194 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T18:50:35.194 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:35.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T18:50:35.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:35.983 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:35 smithi105 ceph-mon[36270]: Deploying daemon osd.1 on smithi105 2024-03-26T18:50:35.983 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:35 smithi105 ceph-mon[36270]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:36.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:35 smithi144 ceph-mon[36053]: Deploying daemon osd.1 on smithi105 2024-03-26T18:50:36.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:35 smithi144 ceph-mon[36053]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:36.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:35 smithi097 ceph-mon[31955]: Deploying daemon osd.1 on smithi105 2024-03-26T18:50:36.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:35 smithi097 ceph-mon[31955]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:38.143 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:37 smithi105 ceph-mon[36270]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:38.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:37 smithi097 ceph-mon[31955]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:38.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:37 smithi144 ceph-mon[36053]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:40.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:39 smithi105 ceph-mon[36270]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:40.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:40.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:40.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:40.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:39 smithi097 ceph-mon[31955]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:40.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:40.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:40.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:39 smithi144 ceph-mon[36053]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:41.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:41 smithi105 ceph-mon[36270]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:42.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:41 smithi097 ceph-mon[31955]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:42.177 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:41 smithi144 ceph-mon[36053]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.239+0000 7f4a5f7fe700 1 -- 172.21.15.105:0/315400778 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a700627f0 con 0x7f4a580a2f20 2024-03-26T18:50:43.231 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 1 on host 'smithi105' 2024-03-26T18:50:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 -- 172.21.15.105:0/315400778 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4a580a2f20 msgr2=0x7f4a580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4a580a2f20 0x7f4a580a53e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a70103430 tx=0x7f4a64008040 comp rx=0 tx=0).stop 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 -- 172.21.15.105:0/315400778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a701035d0 msgr2=0x7f4a70198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a701035d0 0x7f4a70198bb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4a60003890 tx=0x7f4a60003730 comp rx=0 tx=0).stop 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 -- 172.21.15.105:0/315400778 shutdown_connections 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a70103f70 0x7f4a7019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4a580a2f20 0x7f4a580a53e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a701023d0 0x7f4a70198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 --2- 172.21.15.105:0/315400778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a701035d0 0x7f4a70198bb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 -- 172.21.15.105:0/315400778 >> 172.21.15.105:0/315400778 conn(0x7f4a700fd990 msgr2=0x7f4a701068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 -- 172.21.15.105:0/315400778 shutdown_connections 2024-03-26T18:50:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:50:43.241+0000 7f4a74a9f700 1 -- 172.21.15.105:0/315400778 wait complete. 2024-03-26T18:50:43.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:43.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:43.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:43 smithi105 ceph-mon[36270]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:43.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:43.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:43.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:43 smithi097 ceph-mon[31955]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:43 smithi144 ceph-mon[36053]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:43.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:50:43.812 DEBUG:teuthology.orchestra.run.smithi105:osd.1> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.1.service 2024-03-26T18:50:43.814 INFO:tasks.cephadm:Deploying osd.2 on smithi144 with /dev/vg_nvme/lv_4... 2024-03-26T18:50:43.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T18:50:44.058 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:50:44.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:44 smithi105 ceph-mon[36270]: from='osd.1 [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T18:50:44.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:44 smithi105 ceph-mon[36270]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T18:50:44.484 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:44 smithi144 ceph-mon[36053]: from='osd.1 [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T18:50:44.485 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:44 smithi144 ceph-mon[36053]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T18:50:44.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:44 smithi097 ceph-mon[31955]: from='osd.1 [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T18:50:44.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:44 smithi097 ceph-mon[31955]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T18:50:45.491 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:45 smithi144 ceph-mon[36053]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T18:50:45.491 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:45 smithi144 ceph-mon[36053]: from='osd.1 [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:50:45.491 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:45 smithi144 ceph-mon[36053]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T18:50:45.491 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:45.491 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:45 smithi144 ceph-mon[36053]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:50:45.491 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:45 smithi144 ceph-mon[36053]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:45.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:45 smithi097 ceph-mon[31955]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T18:50:45.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:45 smithi097 ceph-mon[31955]: from='osd.1 [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:50:45.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:45 smithi097 ceph-mon[31955]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T18:50:45.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:45.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:45 smithi097 ceph-mon[31955]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:50:45.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:45 smithi097 ceph-mon[31955]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:45.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:45 smithi105 ceph-mon[36270]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T18:50:45.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:45 smithi105 ceph-mon[36270]: from='osd.1 [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:50:45.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:45 smithi105 ceph-mon[36270]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T18:50:45.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:45.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:45 smithi105 ceph-mon[36270]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:50:45.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:45 smithi105 ceph-mon[36270]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:45.857 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T18:50:45.857 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T18:50:45.857 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-26T18:50:45.857 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-26T18:50:45.857 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124368 s, 843 MB/s 2024-03-26T18:50:45.857 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-26T18:50:46.121 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 18:50:45 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1[40864]: 2024-03-26T18:50:45.775+0000 7f58cf5fb700 -1 osd.1 0 waiting for initial osdmap 2024-03-26T18:50:46.121 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 18:50:45 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1[40864]: 2024-03-26T18:50:45.778+0000 7f58c73e6700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:50:46.438 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-03-26T18:50:46.440 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:46 smithi105 ceph-mon[36270]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:50:46.441 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:46 smithi105 ceph-mon[36270]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T18:50:46.441 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:46.441 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:46 smithi105 ceph-mon[36270]: from='osd.1 ' entity='osd.1' 2024-03-26T18:50:46.464 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:46 smithi144 ceph-mon[36053]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:50:46.464 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:46 smithi144 ceph-mon[36053]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T18:50:46.464 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:46.464 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:46 smithi144 ceph-mon[36053]: from='osd.1 ' entity='osd.1' 2024-03-26T18:50:46.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:46 smithi097 ceph-mon[31955]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:50:46.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:46 smithi097 ceph-mon[31955]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T18:50:46.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:46.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:46 smithi097 ceph-mon[31955]: from='osd.1 ' entity='osd.1' 2024-03-26T18:50:46.677 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:50:47.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:50:47.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: osd.1 [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] boot 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi105 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi105 to 17080M 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:50:47.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:50:47.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:47.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: osd.1 [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] boot 2024-03-26T18:50:47.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T18:50:47.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi105 2024-03-26T18:50:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:50:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi105 to 17080M 2024-03-26T18:50:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:50:47.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:50:47.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:47.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: osd.1 [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] boot 2024-03-26T18:50:47.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T18:50:47.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:50:47.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T18:50:47.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi105 2024-03-26T18:50:47.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:50:47.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi105 to 17080M 2024-03-26T18:50:47.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:47.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:47.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:50:47.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:50:48.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/1278729978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f841015d0 msgr2=0x7f6f84101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:50:48.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 --2- 172.21.15.144:0/1278729978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f841015d0 0x7f6f84101a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f780099c0 tx=0x7f6f78009cd0 comp rx=0 tx=0).stop 2024-03-26T18:50:48.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/1278729978 shutdown_connections 2024-03-26T18:50:48.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 --2- 172.21.15.144:0/1278729978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f84101f70 0x7f6f84109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:48.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 --2- 172.21.15.144:0/1278729978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f84102890 0x7f6f84102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:48.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 --2- 172.21.15.144:0/1278729978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f841015d0 0x7f6f84101a30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:48.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/1278729978 >> 172.21.15.144:0/1278729978 conn(0x7f6f840fc000 msgr2=0x7f6f840fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:50:48.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/1278729978 shutdown_connections 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/1278729978 wait complete. 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 Processor -- start 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.131+0000 7f6f8a8f4700 1 -- start start 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f8a8f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f841015d0 0x7f6f84197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f8a8f4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f84101f70 0x7f6f84197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f8a8f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f84102890 0x7f6f8419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f8a8f4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8410cf80 con 0x7f6f841015d0 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f8a8f4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8410ce00 con 0x7f6f84101f70 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f8a8f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8410d100 con 0x7f6f84102890 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f88e91700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f84102890 0x7f6f8419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f88e91700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f84102890 0x7f6f8419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54254/0 (socket says 172.21.15.144:54254) 2024-03-26T18:50:48.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f88e91700 1 -- 172.21.15.144:0/2682445752 learned_addr learned my addr 172.21.15.144:0/2682445752 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T18:50:48.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f837fe700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f84101f70 0x7f6f84197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:50:48.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f88e91700 1 -- 172.21.15.144:0/2682445752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f84101f70 msgr2=0x7f6f84197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:50:48.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f88e91700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f84101f70 0x7f6f84197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:48.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f88e91700 1 -- 172.21.15.144:0/2682445752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f841015d0 msgr2=0x7f6f84197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:50:48.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f88e91700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f841015d0 0x7f6f84197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:50:48.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f88e91700 1 -- 172.21.15.144:0/2682445752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8419f1f0 con 0x7f6f84102890 2024-03-26T18:50:48.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f837fe700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f84101f70 0x7f6f84197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:50:48.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f88e91700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f84102890 0x7f6f8419eb50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c007e60 tx=0x7f6f7c00e330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:50:48.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f817fa700 1 -- 172.21.15.144:0/2682445752 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00c2e0 con 0x7f6f84102890 2024-03-26T18:50:48.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f817fa700 1 -- 172.21.15.144:0/2682445752 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f7c00a600 con 0x7f6f84102890 2024-03-26T18:50:48.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/2682445752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f84100520 con 0x7f6f84102890 2024-03-26T18:50:48.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f817fa700 1 -- 172.21.15.144:0/2682445752 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00c480 con 0x7f6f84102890 2024-03-26T18:50:48.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.132+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/2682445752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f84100ae0 con 0x7f6f84102890 2024-03-26T18:50:48.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.134+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/2682445752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f84069070 con 0x7f6f84102890 2024-03-26T18:50:48.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.135+0000 7f6f817fa700 1 -- 172.21.15.144:0/2682445752 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) ==== 137798+0+0 (secure 0 0 0) 0x7f6f7c00a7b0 con 0x7f6f84102890 2024-03-26T18:50:48.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.136+0000 7f6f817fa700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6f6c0a2f20 0x7f6f6c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:50:48.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.136+0000 7f6f817fa700 1 -- 172.21.15.144:0/2682445752 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f6f7c0cfe50 con 0x7f6f84102890 2024-03-26T18:50:48.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.136+0000 7f6f83fff700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6f6c0a2f20 0x7f6f6c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:50:48.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.137+0000 7f6f83fff700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6f6c0a2f20 0x7f6f6c0a53e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f8406a8b0 tx=0x7f6f74006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:50:48.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.140+0000 7f6f817fa700 1 -- 172.21.15.144:0/2682445752 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6f7c09e4d0 con 0x7f6f84102890 2024-03-26T18:50:48.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:50:48.345+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/2682445752 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6f84100d40 con 0x7f6f6c0a2f20 2024-03-26T18:50:49.575 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:49 smithi144 ceph-mon[36053]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:50:49.575 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:49 smithi144 ceph-mon[36053]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T18:50:49.575 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:50:49.576 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:50:49.576 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:49.576 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:49 smithi144 ceph-mon[36053]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:49.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:49 smithi105 ceph-mon[36270]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:50:49.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:49 smithi105 ceph-mon[36270]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T18:50:49.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:50:49.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:50:49.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:49.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:49 smithi105 ceph-mon[36270]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:49 smithi097 ceph-mon[31955]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:50:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:49 smithi097 ceph-mon[31955]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T18:50:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:50:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:50:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:49.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:49 smithi097 ceph-mon[31955]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:52.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:51 smithi144 ceph-mon[36053]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:52.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:51 smithi144 ceph-mon[36053]: from='client.? 172.21.15.144:0/2567734220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f46098e-3956-46ba-8a63-14e6557ef35f"}]: dispatch 2024-03-26T18:50:52.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:51 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f46098e-3956-46ba-8a63-14e6557ef35f"}]: dispatch 2024-03-26T18:50:52.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:51 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f46098e-3956-46ba-8a63-14e6557ef35f"}]': finished 2024-03-26T18:50:52.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:51 smithi144 ceph-mon[36053]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T18:50:52.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:51 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:50:52.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:51 smithi144 ceph-mon[36053]: from='client.? 172.21.15.144:0/1699972148' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:50:52.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:51 smithi105 ceph-mon[36270]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:52.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:51 smithi105 ceph-mon[36270]: from='client.? 172.21.15.144:0/2567734220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f46098e-3956-46ba-8a63-14e6557ef35f"}]: dispatch 2024-03-26T18:50:52.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:51 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f46098e-3956-46ba-8a63-14e6557ef35f"}]: dispatch 2024-03-26T18:50:52.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:51 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f46098e-3956-46ba-8a63-14e6557ef35f"}]': finished 2024-03-26T18:50:52.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:51 smithi105 ceph-mon[36270]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T18:50:52.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:51 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:50:52.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:51 smithi105 ceph-mon[36270]: from='client.? 172.21.15.144:0/1699972148' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:50:52.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:51 smithi097 ceph-mon[31955]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:52.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:51 smithi097 ceph-mon[31955]: from='client.? 172.21.15.144:0/2567734220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f46098e-3956-46ba-8a63-14e6557ef35f"}]: dispatch 2024-03-26T18:50:52.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:51 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f46098e-3956-46ba-8a63-14e6557ef35f"}]: dispatch 2024-03-26T18:50:52.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:51 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f46098e-3956-46ba-8a63-14e6557ef35f"}]': finished 2024-03-26T18:50:52.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:51 smithi097 ceph-mon[31955]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T18:50:52.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:51 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:50:52.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:51 smithi097 ceph-mon[31955]: from='client.? 172.21.15.144:0/1699972148' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:50:54.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:53 smithi144 ceph-mon[36053]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:54.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:53 smithi105 ceph-mon[36270]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:54.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:53 smithi097 ceph-mon[31955]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:56.029 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:55 smithi144 ceph-mon[36053]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:56.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:55 smithi105 ceph-mon[36270]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:56.351 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:55 smithi097 ceph-mon[31955]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:58.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:57 smithi144 ceph-mon[36053]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:58.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:57 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T18:50:58.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:57 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:58.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:57 smithi105 ceph-mon[36270]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:57 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T18:50:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:57 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:58.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:57 smithi097 ceph-mon[31955]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:50:58.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:57 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T18:50:58.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:57 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:50:59.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:58 smithi144 ceph-mon[36053]: Deploying daemon osd.2 on smithi144 2024-03-26T18:50:59.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:58 smithi105 ceph-mon[36270]: Deploying daemon osd.2 on smithi144 2024-03-26T18:50:59.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:58 smithi097 ceph-mon[31955]: Deploying daemon osd.2 on smithi144 2024-03-26T18:51:00.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:50:59 smithi144 ceph-mon[36053]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:00.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:50:59 smithi105 ceph-mon[36270]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:00.253 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:50:59 smithi097 ceph-mon[31955]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:02.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:01 smithi144 ceph-mon[36053]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:02.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:01 smithi105 ceph-mon[36270]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:02.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:01 smithi097 ceph-mon[31955]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:03 smithi097 ceph-mon[31955]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:04.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:03 smithi144 ceph-mon[36053]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:04.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:04.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:04.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:04.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:03 smithi105 ceph-mon[36270]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:04.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:04.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:04.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:06.253 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:05 smithi144 ceph-mon[36053]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:06.253 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:06.253 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:06.253 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:06.253 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:06.254 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:06.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:05 smithi097 ceph-mon[31955]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:06.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:06.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:06.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:06.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:06.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:06.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:05 smithi105 ceph-mon[36270]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:06.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:06.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:06.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:06.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:06.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:06.569 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 2 on host 'smithi144' 2024-03-26T18:51:06.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.576+0000 7f6f817fa700 1 -- 172.21.15.144:0/2682445752 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6f84100d40 con 0x7f6f6c0a2f20 2024-03-26T18:51:06.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/2682445752 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6f6c0a2f20 msgr2=0x7f6f6c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:51:06.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6f6c0a2f20 0x7f6f6c0a53e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f8406a8b0 tx=0x7f6f74006d20 comp rx=0 tx=0).stop 2024-03-26T18:51:06.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/2682445752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f84102890 msgr2=0x7f6f8419eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:51:06.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f84102890 0x7f6f8419eb50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c007e60 tx=0x7f6f7c00e330 comp rx=0 tx=0).stop 2024-03-26T18:51:06.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/2682445752 shutdown_connections 2024-03-26T18:51:06.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6f6c0a2f20 0x7f6f6c0a53e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:06.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f84102890 0x7f6f8419eb50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:06.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f841015d0 0x7f6f84197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:06.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 --2- 172.21.15.144:0/2682445752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f84101f70 0x7f6f84197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:06.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/2682445752 >> 172.21.15.144:0/2682445752 conn(0x7f6f840fc000 msgr2=0x7f6f841058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:51:06.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/2682445752 shutdown_connections 2024-03-26T18:51:06.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:51:06.579+0000 7f6f8a8f4700 1 -- 172.21.15.144:0/2682445752 wait complete. 2024-03-26T18:51:07.167 DEBUG:teuthology.orchestra.run.smithi144:osd.2> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.2.service 2024-03-26T18:51:07.211 INFO:tasks.cephadm:Deploying osd.3 on smithi097 with /dev/vg_nvme/lv_3... 2024-03-26T18:51:07.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T18:51:07.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:07.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:07.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:07.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:07 smithi097 ceph-mon[31955]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:07.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:07 smithi097 ceph-mon[31955]: from='osd.2 [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T18:51:07.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:07 smithi097 ceph-mon[31955]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T18:51:07.653 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:07.654 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:07.654 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:07.654 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:07 smithi144 ceph-mon[36053]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:07.654 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:07 smithi144 ceph-mon[36053]: from='osd.2 [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T18:51:07.654 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:07 smithi144 ceph-mon[36053]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T18:51:07.672 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:51:07.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:07.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:07.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:07 smithi105 ceph-mon[36270]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:07 smithi105 ceph-mon[36270]: from='osd.2 [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T18:51:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:07 smithi105 ceph-mon[36270]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T18:51:08.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:08 smithi144 ceph-mon[36053]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T18:51:08.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:08 smithi144 ceph-mon[36053]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T18:51:08.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:08 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:08.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:08 smithi144 ceph-mon[36053]: from='osd.2 [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:51:08.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:08 smithi144 ceph-mon[36053]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:51:08.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:08 smithi097 ceph-mon[31955]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T18:51:08.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:08 smithi097 ceph-mon[31955]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T18:51:08.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:08 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:08.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:08 smithi097 ceph-mon[31955]: from='osd.2 [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:51:08.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:08 smithi097 ceph-mon[31955]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:51:08.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:08 smithi105 ceph-mon[36270]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T18:51:08.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:08 smithi105 ceph-mon[36270]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T18:51:08.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:08 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:08.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:08 smithi105 ceph-mon[36270]: from='osd.2 [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:51:08.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:08 smithi105 ceph-mon[36270]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:51:09.375 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:51:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2[41198]: 2024-03-26T18:51:09.021+0000 7f70fad72700 -1 osd.2 0 waiting for initial osdmap 2024-03-26T18:51:09.375 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:51:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2[41198]: 2024-03-26T18:51:09.025+0000 7f70f2b5d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:51:09.599 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:09 smithi097 ceph-mon[31955]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:51:09.599 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:09 smithi097 ceph-mon[31955]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T18:51:09.600 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:09.600 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:09.600 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:09 smithi097 ceph-mon[31955]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:09.600 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:09 smithi097 ceph-mon[31955]: from='osd.2 ' entity='osd.2' 2024-03-26T18:51:09.873 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:09 smithi144 ceph-mon[36053]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:51:09.873 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:09 smithi144 ceph-mon[36053]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T18:51:09.873 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:09.874 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:09.874 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:09 smithi144 ceph-mon[36053]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:09.874 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:09 smithi144 ceph-mon[36053]: from='osd.2 ' entity='osd.2' 2024-03-26T18:51:09.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:09 smithi105 ceph-mon[36270]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:51:09.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:09 smithi105 ceph-mon[36270]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T18:51:09.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:09.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:09.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:09 smithi105 ceph-mon[36270]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-26T18:51:09.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:09 smithi105 ceph-mon[36270]: from='osd.2 ' entity='osd.2' 2024-03-26T18:51:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T18:51:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T18:51:10.566 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-26T18:51:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-26T18:51:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0373561 s, 281 MB/s 2024-03-26T18:51:10.567 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-26T18:51:10.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:51:10.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:51:10.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:10.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: osd.2 [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] boot 2024-03-26T18:51:10.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T18:51:10.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:10.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.854 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.854 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:51:10.854 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.854 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:10.854 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:10.854 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:51:10.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: osd.2 [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] boot 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:10.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: osd.2 [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] boot 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:10.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:11.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-03-26T18:51:11.783 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:51:11.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:11 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi144 2024-03-26T18:51:11.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:11 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi144 to 17080M 2024-03-26T18:51:11.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:11 smithi097 ceph-mon[31955]: pgmap v89: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:11.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:51:11.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:11 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi144 2024-03-26T18:51:11.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:11 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi144 to 17080M 2024-03-26T18:51:11.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:11 smithi144 ceph-mon[36053]: pgmap v89: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:11.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:51:11.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:11 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi144 2024-03-26T18:51:11.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:11 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi144 to 17080M 2024-03-26T18:51:11.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:11 smithi105 ceph-mon[36270]: pgmap v89: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:11.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:51:12.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:12 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T18:51:12.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:12 smithi144 ceph-mon[36053]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T18:51:12.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:12 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:51:12.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:12 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T18:51:12.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:12 smithi105 ceph-mon[36270]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T18:51:12.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:12 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:51:12.987 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:12 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T18:51:12.987 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:12 smithi097 ceph-mon[31955]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T18:51:12.987 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:12 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:51:13.727 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:13 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T18:51:13.727 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:13 smithi097 ceph-mon[31955]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T18:51:13.727 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:13 smithi097 ceph-mon[31955]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:13.727 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:13 smithi097 ceph-mon[31955]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T18:51:13.727 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 18:51:13 smithi097 sudo[48507]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T18:51:13.727 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 18:51:13 smithi097 sudo[48507]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:13.727 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 18:51:13 smithi097 sudo[48507]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:13.727 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 18:51:13 smithi097 sudo[48507]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:13.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:13 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T18:51:13.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:13 smithi144 ceph-mon[36053]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T18:51:13.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:13 smithi144 ceph-mon[36053]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:13.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:13 smithi144 ceph-mon[36053]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T18:51:13.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.893+0000 7f4181400700 1 -- 172.21.15.97:0/52645362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f417c1035f0 msgr2=0x7f417c103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:51:13.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.893+0000 7f4181400700 1 --2- 172.21.15.97:0/52645362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f417c1035f0 0x7f417c103a50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f41680099c0 tx=0x7f4168009cd0 comp rx=0 tx=0).stop 2024-03-26T18:51:13.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.893+0000 7f4181400700 1 -- 172.21.15.97:0/52645362 shutdown_connections 2024-03-26T18:51:13.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.893+0000 7f4181400700 1 --2- 172.21.15.97:0/52645362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c103f90 0x7f417c10a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:13.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.893+0000 7f4181400700 1 --2- 172.21.15.97:0/52645362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417c1023f0 0x7f417c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:13.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.893+0000 7f4181400700 1 --2- 172.21.15.97:0/52645362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f417c1035f0 0x7f417c103a50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:13.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.893+0000 7f4181400700 1 -- 172.21.15.97:0/52645362 >> 172.21.15.97:0/52645362 conn(0x7f417c0fd950 msgr2=0x7f417c0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:51:13.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.893+0000 7f4181400700 1 -- 172.21.15.97:0/52645362 shutdown_connections 2024-03-26T18:51:13.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.893+0000 7f4181400700 1 -- 172.21.15.97:0/52645362 wait complete. 2024-03-26T18:51:13.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.894+0000 7f4181400700 1 Processor -- start 2024-03-26T18:51:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f4181400700 1 -- start start 2024-03-26T18:51:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f4181400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f417c1023f0 0x7f417c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:51:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f4181400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417c1035f0 0x7f417c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:51:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f4181400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c103f90 0x7f417c1a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:51:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f4181400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10e010 con 0x7f417c1023f0 2024-03-26T18:51:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f4181400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10de90 con 0x7f417c103f90 2024-03-26T18:51:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f4181400700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10e190 con 0x7f417c1035f0 2024-03-26T18:51:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f417a800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f417c1023f0 0x7f417c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:51:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f4179e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417c1035f0 0x7f417c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f417a800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f417c1023f0 0x7f417c19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57676/0 (socket says 172.21.15.97:57676) 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f4179e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417c1035f0 0x7f417c19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53438/0 (socket says 172.21.15.97:53438) 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f417a800700 1 -- 172.21.15.97:0/1324057401 learned_addr learned my addr 172.21.15.97:0/1324057401 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.895+0000 7f417b200700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c103f90 0x7f417c1a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.896+0000 7f4179e00700 1 -- 172.21.15.97:0/1324057401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c103f90 msgr2=0x7f417c1a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.896+0000 7f4179e00700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c103f90 0x7f417c1a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.896+0000 7f4179e00700 1 -- 172.21.15.97:0/1324057401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f417c1023f0 msgr2=0x7f417c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.896+0000 7f4179e00700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f417c1023f0 0x7f417c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.896+0000 7f4179e00700 1 -- 172.21.15.97:0/1324057401 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c1a2470 con 0x7f417c1035f0 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.896+0000 7f4179e00700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417c1035f0 0x7f417c19ad50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f41680099c0 tx=0x7f416800bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:51:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.896+0000 7f4173400700 1 -- 172.21.15.97:0/1324057401 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168003e30 con 0x7f417c1035f0 2024-03-26T18:51:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.896+0000 7f4181400700 1 -- 172.21.15.97:0/1324057401 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417c1a2700 con 0x7f417c1035f0 2024-03-26T18:51:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.896+0000 7f4173400700 1 -- 172.21.15.97:0/1324057401 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f416800f460 con 0x7f417c1035f0 2024-03-26T18:51:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.896+0000 7f4173400700 1 -- 172.21.15.97:0/1324057401 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168017670 con 0x7f417c1035f0 2024-03-26T18:51:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.897+0000 7f4181400700 1 -- 172.21.15.97:0/1324057401 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417c1a2b50 con 0x7f417c1035f0 2024-03-26T18:51:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.898+0000 7f4181400700 1 -- 172.21.15.97:0/1324057401 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f417c069070 con 0x7f417c1035f0 2024-03-26T18:51:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.900+0000 7f4173400700 1 -- 172.21.15.97:0/1324057401 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f416800f9c0 con 0x7f417c1035f0 2024-03-26T18:51:13.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.901+0000 7f4173400700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f41600a2fb0 0x7f41600a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:51:13.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.901+0000 7f4173400700 1 -- 172.21.15.97:0/1324057401 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f41680cee40 con 0x7f417c1035f0 2024-03-26T18:51:13.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.902+0000 7f417a800700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f41600a2fb0 0x7f41600a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:51:13.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.902+0000 7f417a800700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f41600a2fb0 0x7f41600a5470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f416c005950 tx=0x7f416c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:51:13.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:13.904+0000 7f4173400700 1 -- 172.21.15.97:0/1324057401 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4168013070 con 0x7f417c1035f0 2024-03-26T18:51:13.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:13 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T18:51:13.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:13 smithi105 ceph-mon[36270]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T18:51:13.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:13 smithi105 ceph-mon[36270]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:13.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:13 smithi105 ceph-mon[36270]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T18:51:14.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:14.141+0000 7f4181400700 1 -- 172.21.15.97:0/1324057401 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f417c075d00 con 0x7f41600a2fb0 2024-03-26T18:51:14.353 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 18:51:13 smithi097 sudo[48537]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T18:51:14.354 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 18:51:13 smithi097 sudo[48537]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:14.354 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 18:51:13 smithi097 sudo[48537]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:14.354 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 18:51:14 smithi097 sudo[48537]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:14.520 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 18:51:14 smithi105 sudo[44575]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T18:51:14.520 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 18:51:14 smithi105 sudo[44575]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:14.520 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 18:51:14 smithi105 sudo[44575]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:14.520 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 18:51:14 smithi105 sudo[44575]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:14.875 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:51:14 smithi144 sudo[45272]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T18:51:14.875 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:51:14 smithi144 sudo[45272]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:14.875 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:51:14 smithi144 sudo[45272]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:14.876 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:51:14 smithi144 sudo[45272]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:14.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:14 smithi144 ceph-mon[36053]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T18:51:14.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:14 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:14.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:14 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:51:14.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:14 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:51:14.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:14 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:14.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:14 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:14.959 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 18:51:14 smithi105 sudo[44578]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T18:51:14.959 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 18:51:14 smithi105 sudo[44578]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:14.960 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 18:51:14 smithi105 sudo[44578]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:14.960 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 18:51:14 smithi105 sudo[44578]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:14.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:14 smithi105 ceph-mon[36270]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T18:51:14.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:14 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:14.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:14 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:51:14.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:14 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:51:14.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:14 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:14.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:14 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:15.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:14 smithi097 ceph-mon[31955]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T18:51:15.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:15.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:51:15.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:51:15.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:15.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:15.375 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:51:15 smithi144 sudo[45275]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T18:51:15.375 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:51:15 smithi144 sudo[45275]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:15.375 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:51:15 smithi144 sudo[45275]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:15.375 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:51:15 smithi144 sudo[45275]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:15.930 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:15 smithi097 sudo[48676]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T18:51:15.930 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:15 smithi097 sudo[48676]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:15.930 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:15 smithi097 sudo[48676]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:15.930 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:15 smithi097 sudo[48676]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:16.591 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:16 smithi097 sudo[48682]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T18:51:16.591 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:16 smithi097 sudo[48682]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:16.591 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:16 smithi097 sudo[48682]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:16.591 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:16 smithi097 sudo[48682]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:16.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:16 smithi144 sudo[45278]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T18:51:16.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:16 smithi144 sudo[45278]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:16.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:16 smithi144 sudo[45278]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:16.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:16 smithi144 sudo[45278]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:16.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:16 smithi144 sudo[45281]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T18:51:16.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:16 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:16.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:16 smithi097 ceph-mon[31955]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T18:51:17.017 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:16 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:17.017 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:16 smithi105 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T18:51:17.017 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 sudo[44581]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T18:51:17.017 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 sudo[44581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:17.017 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 sudo[44581]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:17.017 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 sudo[44581]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:17.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:16 smithi144 sudo[45281]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:17.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:16 smithi144 sudo[45281]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:17.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:16 smithi144 sudo[45281]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:17.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:16 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:17.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:16 smithi144 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T18:51:17.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 sudo[44584]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T18:51:17.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 sudo[44584]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T18:51:17.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 sudo[44584]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T18:51:17.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 sudo[44584]: pam_unix(sudo:session): session closed for user root 2024-03-26T18:51:18.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:51:18.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:18.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T18:51:18.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:51:18.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:51:18.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:18.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:17 smithi097 ceph-mon[31955]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T18:51:18.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:51:18.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:18.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T18:51:18.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:51:18.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:51:18.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:18.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:17 smithi144 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:51:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T18:51:18.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T18:51:18.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:51:18.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:51:18.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:51:18.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:18.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:17 smithi105 ceph-mon[36270]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T18:51:19.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:18 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2541898978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0937aa7b-00ea-417f-b74f-442ca05ac527"}]: dispatch 2024-03-26T18:51:19.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:18 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0937aa7b-00ea-417f-b74f-442ca05ac527"}]: dispatch 2024-03-26T18:51:19.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:18 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0937aa7b-00ea-417f-b74f-442ca05ac527"}]': finished 2024-03-26T18:51:19.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:18 smithi097 ceph-mon[31955]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T18:51:19.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:18 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:19.112 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:18 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2541898978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0937aa7b-00ea-417f-b74f-442ca05ac527"}]: dispatch 2024-03-26T18:51:19.112 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:18 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0937aa7b-00ea-417f-b74f-442ca05ac527"}]: dispatch 2024-03-26T18:51:19.112 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:18 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0937aa7b-00ea-417f-b74f-442ca05ac527"}]': finished 2024-03-26T18:51:19.112 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:18 smithi105 ceph-mon[36270]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T18:51:19.112 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:18 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:19.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:18 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2541898978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0937aa7b-00ea-417f-b74f-442ca05ac527"}]: dispatch 2024-03-26T18:51:19.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:18 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0937aa7b-00ea-417f-b74f-442ca05ac527"}]: dispatch 2024-03-26T18:51:19.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:18 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0937aa7b-00ea-417f-b74f-442ca05ac527"}]': finished 2024-03-26T18:51:19.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:18 smithi144 ceph-mon[36053]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T18:51:19.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:18 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:20.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:19 smithi097 ceph-mon[31955]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:20.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:19 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/549868357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:51:20.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:19 smithi144 ceph-mon[36053]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:20.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:19 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/549868357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:51:20.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:19 smithi105 ceph-mon[36270]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:20.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:19 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/549868357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:51:22.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:21 smithi144 ceph-mon[36053]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:22.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:21 smithi105 ceph-mon[36270]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:22.240 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:21 smithi097 ceph-mon[31955]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:24.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:23 smithi144 ceph-mon[36053]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:24.196 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:23 smithi097 ceph-mon[31955]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:24.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:23 smithi105 ceph-mon[36270]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:26.011 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:25 smithi097 ceph-mon[31955]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:26.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:25 smithi144 ceph-mon[36053]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:26.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:25 smithi105 ceph-mon[36270]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:28.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:27 smithi105 ceph-mon[36270]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:28.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T18:51:28.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:28.218 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:27 smithi097 ceph-mon[31955]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:28.218 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T18:51:28.218 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:28.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:27 smithi144 ceph-mon[36053]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:28.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T18:51:28.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:29.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:28 smithi105 ceph-mon[36270]: Deploying daemon osd.3 on smithi097 2024-03-26T18:51:29.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:28 smithi097 ceph-mon[31955]: Deploying daemon osd.3 on smithi097 2024-03-26T18:51:29.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:28 smithi144 ceph-mon[36053]: Deploying daemon osd.3 on smithi097 2024-03-26T18:51:30.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:29 smithi097 ceph-mon[31955]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:30.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:29 smithi105 ceph-mon[36270]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:30.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:29 smithi144 ceph-mon[36053]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:32.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:31 smithi097 ceph-mon[31955]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:32.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:31 smithi105 ceph-mon[36270]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:32.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:31 smithi144 ceph-mon[36053]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:34.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:33 smithi105 ceph-mon[36270]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:34.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:33 smithi097 ceph-mon[31955]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:34.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:33 smithi144 ceph-mon[36053]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:36.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:35 smithi105 ceph-mon[36270]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:36.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:36.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:36.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:36.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:35 smithi097 ceph-mon[31955]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:36.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:36.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:36.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:36.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:35 smithi144 ceph-mon[36053]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:36.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:36.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:36.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:38.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:37 smithi105 ceph-mon[36270]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:38.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:37 smithi097 ceph-mon[31955]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:38.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:37 smithi144 ceph-mon[36053]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:40.180 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:39 smithi097 ceph-mon[31955]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:40.180 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:40.180 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:40.180 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:40.180 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:40.181 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:40.181 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:39 smithi097 ceph-mon[31955]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T18:51:40.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:39 smithi105 ceph-mon[36270]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:40.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:40.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:39 smithi105 ceph-mon[36270]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T18:51:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:39 smithi144 ceph-mon[36053]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:39 smithi144 ceph-mon[36053]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T18:51:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.645+0000 7f4173400700 1 -- 172.21.15.97:0/1324057401 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f417c075d00 con 0x7f41600a2fb0 2024-03-26T18:51:40.639 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 3 on host 'smithi097' 2024-03-26T18:51:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 -- 172.21.15.97:0/1324057401 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f41600a2fb0 msgr2=0x7f41600a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:51:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f41600a2fb0 0x7f41600a5470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f416c005950 tx=0x7f416c009260 comp rx=0 tx=0).stop 2024-03-26T18:51:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 -- 172.21.15.97:0/1324057401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417c1035f0 msgr2=0x7f417c19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:51:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417c1035f0 0x7f417c19ad50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f41680099c0 tx=0x7f416800bee0 comp rx=0 tx=0).stop 2024-03-26T18:51:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 -- 172.21.15.97:0/1324057401 shutdown_connections 2024-03-26T18:51:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c103f90 0x7f417c1a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f417c1023f0 0x7f417c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f41600a2fb0 0x7f41600a5470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 --2- 172.21.15.97:0/1324057401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417c1035f0 0x7f417c19ad50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 -- 172.21.15.97:0/1324057401 >> 172.21.15.97:0/1324057401 conn(0x7f417c0fd950 msgr2=0x7f417c0745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:51:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 -- 172.21.15.97:0/1324057401 shutdown_connections 2024-03-26T18:51:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:51:40.650+0000 7f4181400700 1 -- 172.21.15.97:0/1324057401 wait complete. 2024-03-26T18:51:41.516 DEBUG:teuthology.orchestra.run.smithi097:osd.3> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.3.service 2024-03-26T18:51:41.521 INFO:tasks.cephadm:Deploying osd.4 on smithi105 with /dev/vg_nvme/lv_3... 2024-03-26T18:51:41.521 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T18:51:41.596 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:41 smithi097 ceph-mon[31955]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T18:51:41.596 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:41 smithi097 ceph-mon[31955]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T18:51:41.596 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:41 smithi097 ceph-mon[31955]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:51:41.596 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:41.597 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:41.597 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:41.597 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:41.597 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:41 smithi097 ceph-mon[31955]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:41.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:41 smithi144 ceph-mon[36053]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T18:51:41.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:41 smithi144 ceph-mon[36053]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T18:51:41.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:41 smithi144 ceph-mon[36053]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:51:41.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:41.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:41.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:41.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:41.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:41 smithi144 ceph-mon[36053]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:41.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:41 smithi105 ceph-mon[36270]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T18:51:41.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:41 smithi105 ceph-mon[36270]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T18:51:41.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:41 smithi105 ceph-mon[36270]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:51:41.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:41.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:41.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:41.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:51:41.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:41 smithi105 ceph-mon[36270]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:41.764 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:51:42.215 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 18:51:42 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3[50794]: 2024-03-26T18:51:42.001+0000 7f0fb6c00700 -1 osd.3 0 waiting for initial osdmap 2024-03-26T18:51:42.215 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 18:51:42 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3[50794]: 2024-03-26T18:51:42.017+0000 7f0faca00700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:51:42.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:42 smithi105 ceph-mon[36270]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:51:42.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:42 smithi105 ceph-mon[36270]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T18:51:42.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:42 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:42.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:42 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:42.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:42 smithi105 ceph-mon[36270]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' 2024-03-26T18:51:42.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:42 smithi097 ceph-mon[31955]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:51:42.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:42 smithi097 ceph-mon[31955]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T18:51:42.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:42 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:42.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:42 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:42.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:42 smithi097 ceph-mon[31955]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' 2024-03-26T18:51:42.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:42 smithi144 ceph-mon[36053]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:51:42.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:42 smithi144 ceph-mon[36053]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T18:51:42.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:42 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:42.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:42 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:42.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:42 smithi144 ceph-mon[36053]: from='osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710]' entity='osd.3' 2024-03-26T18:51:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:43 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:51:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:43 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:51:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:43 smithi097 ceph-mon[31955]: osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] boot 2024-03-26T18:51:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:43 smithi097 ceph-mon[31955]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T18:51:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:43 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:43 smithi097 ceph-mon[31955]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:43 smithi097 ceph-mon[31955]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T18:51:43.604 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T18:51:43.604 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T18:51:43.604 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-26T18:51:43.604 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-26T18:51:43.605 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124416 s, 843 MB/s 2024-03-26T18:51:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-26T18:51:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:43 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:51:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:43 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:51:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:43 smithi144 ceph-mon[36053]: osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] boot 2024-03-26T18:51:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:43 smithi144 ceph-mon[36053]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T18:51:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:43 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:43 smithi144 ceph-mon[36053]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:43.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:43 smithi144 ceph-mon[36053]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T18:51:43.628 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:43 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:51:43.628 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:43 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:51:43.628 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:43 smithi105 ceph-mon[36270]: osd.3 [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] boot 2024-03-26T18:51:43.628 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:43 smithi105 ceph-mon[36270]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T18:51:43.628 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:43 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:51:43.629 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:43 smithi105 ceph-mon[36270]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-26T18:51:43.629 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:43 smithi105 ceph-mon[36270]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T18:51:44.273 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_3 2024-03-26T18:51:44.514 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:51:45.458 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:45 smithi105 ceph-mon[36270]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T18:51:45.458 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:45 smithi105 ceph-mon[36270]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T18:51:45.458 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:45 smithi105 ceph-mon[36270]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:51:45.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:45 smithi097 ceph-mon[31955]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T18:51:45.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:45 smithi097 ceph-mon[31955]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T18:51:45.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:45 smithi097 ceph-mon[31955]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:51:45.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:45 smithi144 ceph-mon[36053]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T18:51:45.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:45 smithi144 ceph-mon[36053]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T18:51:45.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:45 smithi144 ceph-mon[36053]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:51:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.972+0000 7fc3f375e700 1 -- 172.21.15.105:0/2820168351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ec101350 msgr2=0x7fc3ec101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:51:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.972+0000 7fc3f375e700 1 --2- 172.21.15.105:0/2820168351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ec101350 0x7fc3ec101770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0099c0 tx=0x7fc3dc009cd0 comp rx=0 tx=0).stop 2024-03-26T18:51:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.972+0000 7fc3f375e700 1 -- 172.21.15.105:0/2820168351 shutdown_connections 2024-03-26T18:51:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.972+0000 7fc3f375e700 1 --2- 172.21.15.105:0/2820168351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec102ef0 0x7fc3ec1097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.972+0000 7fc3f375e700 1 --2- 172.21.15.105:0/2820168351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ec101350 0x7fc3ec101770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.972+0000 7fc3f375e700 1 --2- 172.21.15.105:0/2820168351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3ec102550 0x7fc3ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.972+0000 7fc3f375e700 1 -- 172.21.15.105:0/2820168351 >> 172.21.15.105:0/2820168351 conn(0x7fc3ec0fc910 msgr2=0x7fc3ec0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:51:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.972+0000 7fc3f375e700 1 -- 172.21.15.105:0/2820168351 shutdown_connections 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.972+0000 7fc3f375e700 1 -- 172.21.15.105:0/2820168351 wait complete. 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f375e700 1 Processor -- start 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f375e700 1 -- start start 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f375e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3ec101350 0x7fc3ec06b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f375e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ec102550 0x7fc3ec19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f375e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec102ef0 0x7fc3ec19f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f375e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec10cf40 con 0x7fc3ec102ef0 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f375e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec10cdc0 con 0x7fc3ec102550 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f375e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec10d0c0 con 0x7fc3ec101350 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f0cf9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ec102550 0x7fc3ec19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f0cf9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ec102550 0x7fc3ec19d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39248/0 (socket says 172.21.15.105:39248) 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f0cf9700 1 -- 172.21.15.105:0/2651658251 learned_addr learned my addr 172.21.15.105:0/2651658251 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:51:45.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.973+0000 7fc3f14fa700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3ec101350 0x7fc3ec06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3f0cf9700 1 -- 172.21.15.105:0/2651658251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3ec101350 msgr2=0x7fc3ec06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3f0cf9700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3ec101350 0x7fc3ec06b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3f0cf9700 1 -- 172.21.15.105:0/2651658251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec102ef0 msgr2=0x7fc3ec19f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3f0cf9700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec102ef0 0x7fc3ec19f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3f0cf9700 1 -- 172.21.15.105:0/2651658251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec1a02c0 con 0x7fc3ec102550 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3f0cf9700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ec102550 0x7fc3ec19d3b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800d980 tx=0x7fc3e800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3e27fc700 1 -- 172.21.15.105:0/2651658251 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8017070 con 0x7fc3ec102550 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3e27fc700 1 -- 172.21.15.105:0/2651658251 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e8004d10 con 0x7fc3ec102550 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3f375e700 1 -- 172.21.15.105:0/2651658251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec1a05b0 con 0x7fc3ec102550 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3e27fc700 1 -- 172.21.15.105:0/2651658251 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e80055c0 con 0x7fc3ec102550 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.974+0000 7fc3f375e700 1 -- 172.21.15.105:0/2651658251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec1a0b70 con 0x7fc3ec102550 2024-03-26T18:51:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.975+0000 7fc3f375e700 1 -- 172.21.15.105:0/2651658251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec069070 con 0x7fc3ec102550 2024-03-26T18:51:45.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.976+0000 7fc3e27fc700 1 -- 172.21.15.105:0/2651658251 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fc3e8004750 con 0x7fc3ec102550 2024-03-26T18:51:45.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.977+0000 7fc3e27fc700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc3d80a3030 0x7fc3d80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:51:45.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.977+0000 7fc3e27fc700 1 -- 172.21.15.105:0/2651658251 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fc3e80ce0f0 con 0x7fc3ec102550 2024-03-26T18:51:45.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.977+0000 7fc3f14fa700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc3d80a3030 0x7fc3d80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:51:45.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.978+0000 7fc3f14fa700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc3d80a3030 0x7fc3d80a54f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b580 tx=0x7fc3dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:51:45.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:45.981+0000 7fc3e27fc700 1 -- 172.21.15.105:0/2651658251 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc3e8020020 con 0x7fc3ec102550 2024-03-26T18:51:46.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:51:46.183+0000 7fc3f375e700 1 -- 172.21.15.105:0/2651658251 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc3ec0627f0 con 0x7fc3d80a3030 2024-03-26T18:51:46.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:46.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:46 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:46.946 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:46 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:47.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T18:51:47.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:51:47.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:51:47.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:47.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:47.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:47.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:51:47.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:51:47.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:47.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:47.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:51:47.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:46 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:51:47.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:47 smithi105 ceph-mon[36270]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:51:47.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:47 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi097 2024-03-26T18:51:47.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:47 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi097 to 8545M 2024-03-26T18:51:47.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:47 smithi105 ceph-mon[36270]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:51:48.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:47 smithi097 ceph-mon[31955]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:51:48.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:47 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi097 2024-03-26T18:51:48.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:47 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi097 to 8545M 2024-03-26T18:51:48.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:47 smithi097 ceph-mon[31955]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:51:48.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:47 smithi144 ceph-mon[36053]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:51:48.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:47 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi097 2024-03-26T18:51:48.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:47 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi097 to 8545M 2024-03-26T18:51:48.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:47 smithi144 ceph-mon[36053]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:51:49.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:48 smithi097 ceph-mon[31955]: from='client.? 172.21.15.105:0/1889545579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f20c598-4cd5-40d8-befd-e5e50d759bc6"}]: dispatch 2024-03-26T18:51:49.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:48 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f20c598-4cd5-40d8-befd-e5e50d759bc6"}]: dispatch 2024-03-26T18:51:49.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:48 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f20c598-4cd5-40d8-befd-e5e50d759bc6"}]': finished 2024-03-26T18:51:49.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:48 smithi097 ceph-mon[31955]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T18:51:49.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:48 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:51:49.120 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:48 smithi105 ceph-mon[36270]: from='client.? 172.21.15.105:0/1889545579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f20c598-4cd5-40d8-befd-e5e50d759bc6"}]: dispatch 2024-03-26T18:51:49.121 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:48 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f20c598-4cd5-40d8-befd-e5e50d759bc6"}]: dispatch 2024-03-26T18:51:49.121 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:48 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f20c598-4cd5-40d8-befd-e5e50d759bc6"}]': finished 2024-03-26T18:51:49.121 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:48 smithi105 ceph-mon[36270]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T18:51:49.121 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:48 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:51:49.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:48 smithi144 ceph-mon[36053]: from='client.? 172.21.15.105:0/1889545579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f20c598-4cd5-40d8-befd-e5e50d759bc6"}]: dispatch 2024-03-26T18:51:49.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:48 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f20c598-4cd5-40d8-befd-e5e50d759bc6"}]: dispatch 2024-03-26T18:51:49.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:48 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f20c598-4cd5-40d8-befd-e5e50d759bc6"}]': finished 2024-03-26T18:51:49.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:48 smithi144 ceph-mon[36053]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T18:51:49.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:48 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:51:50.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:49 smithi097 ceph-mon[31955]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-26T18:51:50.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:49 smithi097 ceph-mon[31955]: from='client.? 172.21.15.105:0/3409137081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:51:50.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:49 smithi144 ceph-mon[36053]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-26T18:51:50.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:49 smithi144 ceph-mon[36053]: from='client.? 172.21.15.105:0/3409137081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:51:50.129 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:49 smithi105 ceph-mon[36270]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-26T18:51:50.130 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:49 smithi105 ceph-mon[36270]: from='client.? 172.21.15.105:0/3409137081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:51:52.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:51 smithi097 ceph-mon[31955]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-26T18:51:52.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:51 smithi144 ceph-mon[36053]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-26T18:51:52.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:51 smithi105 ceph-mon[36270]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-26T18:51:54.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:53 smithi097 ceph-mon[31955]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:51:54.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:53 smithi144 ceph-mon[36053]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:51:54.165 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:53 smithi105 ceph-mon[36270]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:51:56.100 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:55 smithi105 ceph-mon[36270]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-26T18:51:56.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:55 smithi097 ceph-mon[31955]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-26T18:51:56.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:55 smithi144 ceph-mon[36053]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-26T18:51:57.061 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:56 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T18:51:57.062 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:56 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:57.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:56 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T18:51:57.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:56 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:57.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:56 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T18:51:57.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:56 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:51:58.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:57 smithi144 ceph-mon[36053]: Deploying daemon osd.4 on smithi105 2024-03-26T18:51:58.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:57 smithi144 ceph-mon[36053]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T18:51:58.168 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:57 smithi105 ceph-mon[36270]: Deploying daemon osd.4 on smithi105 2024-03-26T18:51:58.168 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:57 smithi105 ceph-mon[36270]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T18:51:58.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:57 smithi097 ceph-mon[31955]: Deploying daemon osd.4 on smithi105 2024-03-26T18:51:58.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:57 smithi097 ceph-mon[31955]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T18:52:00.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:51:59 smithi144 ceph-mon[36053]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:00.155 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:51:59 smithi097 ceph-mon[31955]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:00.168 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:51:59 smithi105 ceph-mon[36270]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:02.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:01 smithi097 ceph-mon[31955]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:02.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:02.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:02.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:02.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:01 smithi144 ceph-mon[36053]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:02.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:02.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:02.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:02.193 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:01 smithi105 ceph-mon[36270]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:02.193 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:02.193 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:02.193 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:03.945 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:03 smithi105 ceph-mon[36270]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:04.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:03 smithi144 ceph-mon[36053]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:04.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:03 smithi097 ceph-mon[31955]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:04.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.718+0000 7fc3e27fc700 1 -- 172.21.15.105:0/2651658251 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc3ec0627f0 con 0x7fc3d80a3030 2024-03-26T18:52:04.711 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 4 on host 'smithi105' 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.720+0000 7fc3f375e700 1 -- 172.21.15.105:0/2651658251 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc3d80a3030 msgr2=0x7fc3d80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.720+0000 7fc3f375e700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc3d80a3030 0x7fc3d80a54f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b580 tx=0x7fc3dc005fb0 comp rx=0 tx=0).stop 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.720+0000 7fc3f375e700 1 -- 172.21.15.105:0/2651658251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ec102550 msgr2=0x7fc3ec19d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.720+0000 7fc3f375e700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ec102550 0x7fc3ec19d3b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800d980 tx=0x7fc3e800dd40 comp rx=0 tx=0).stop 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.720+0000 7fc3f375e700 1 -- 172.21.15.105:0/2651658251 shutdown_connections 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.720+0000 7fc3f375e700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec102ef0 0x7fc3ec19f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.720+0000 7fc3f375e700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc3d80a3030 0x7fc3d80a54f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.720+0000 7fc3f375e700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3ec101350 0x7fc3ec06b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.720+0000 7fc3f375e700 1 --2- 172.21.15.105:0/2651658251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ec102550 0x7fc3ec19d3b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.720+0000 7fc3f375e700 1 -- 172.21.15.105:0/2651658251 >> 172.21.15.105:0/2651658251 conn(0x7fc3ec0fc910 msgr2=0x7fc3ec1058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.721+0000 7fc3f375e700 1 -- 172.21.15.105:0/2651658251 shutdown_connections 2024-03-26T18:52:04.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:52:04.721+0000 7fc3f375e700 1 -- 172.21.15.105:0/2651658251 wait complete. 2024-03-26T18:52:05.638 DEBUG:teuthology.orchestra.run.smithi105:osd.4> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.4.service 2024-03-26T18:52:05.641 INFO:tasks.cephadm:Deploying osd.5 on smithi144 with /dev/vg_nvme/lv_3... 2024-03-26T18:52:05.641 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T18:52:05.890 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:52:05.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:05.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:05.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:05 smithi105 ceph-mon[36270]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:05.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:05 smithi105 ceph-mon[36270]: from='osd.4 [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T18:52:05.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:05 smithi105 ceph-mon[36270]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T18:52:06.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:06.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:06.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:05 smithi144 ceph-mon[36053]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:06.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:05 smithi144 ceph-mon[36053]: from='osd.4 [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T18:52:06.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:05 smithi144 ceph-mon[36053]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T18:52:06.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:06.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:06.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:05 smithi097 ceph-mon[31955]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:06.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:05 smithi097 ceph-mon[31955]: from='osd.4 [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T18:52:06.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:05 smithi097 ceph-mon[31955]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T18:52:06.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:06 smithi105 ceph-mon[36270]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T18:52:06.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:06 smithi105 ceph-mon[36270]: from='osd.4 [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:52:06.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:06 smithi105 ceph-mon[36270]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T18:52:06.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:06.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:06 smithi105 ceph-mon[36270]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:52:06.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:06.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:06.960 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 18:52:06 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4[47371]: 2024-03-26T18:52:06.732+0000 7fba6c28c700 -1 osd.4 0 waiting for initial osdmap 2024-03-26T18:52:06.960 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 18:52:06 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4[47371]: 2024-03-26T18:52:06.738+0000 7fba6807f700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:52:07.081 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:06 smithi144 ceph-mon[36053]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T18:52:07.081 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:06 smithi144 ceph-mon[36053]: from='osd.4 [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:52:07.081 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:06 smithi144 ceph-mon[36053]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T18:52:07.081 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:07.081 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:06 smithi144 ceph-mon[36053]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:52:07.081 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:07.082 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:07.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:06 smithi097 ceph-mon[31955]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T18:52:07.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:06 smithi097 ceph-mon[31955]: from='osd.4 [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:52:07.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:06 smithi097 ceph-mon[31955]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T18:52:07.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:07.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:06 smithi097 ceph-mon[31955]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:52:07.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:07.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:07.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:07 smithi105 ceph-mon[36270]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:52:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:07 smithi105 ceph-mon[36270]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T18:52:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:07 smithi105 ceph-mon[36270]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:07.974 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T18:52:07.974 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T18:52:07.974 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-26T18:52:07.975 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-26T18:52:07.975 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012526 s, 837 MB/s 2024-03-26T18:52:07.975 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-26T18:52:07.975 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:07 smithi144 ceph-mon[36053]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:52:07.975 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:07 smithi144 ceph-mon[36053]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T18:52:07.975 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:07.975 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:07 smithi144 ceph-mon[36053]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:07.975 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:08.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:07 smithi097 ceph-mon[31955]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:52:08.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:07 smithi097 ceph-mon[31955]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T18:52:08.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:08.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:07 smithi097 ceph-mon[31955]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-26T18:52:08.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:08.561 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-03-26T18:52:08.803 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:52:08.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:08 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:52:08.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:08 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:52:08.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:08 smithi144 ceph-mon[36053]: osd.4 [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] boot 2024-03-26T18:52:08.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:08 smithi144 ceph-mon[36053]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T18:52:08.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:08 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:09.078 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:08 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:52:09.078 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:08 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:52:09.078 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:08 smithi105 ceph-mon[36270]: osd.4 [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] boot 2024-03-26T18:52:09.078 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:08 smithi105 ceph-mon[36270]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T18:52:09.079 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:08 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:09.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:08 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:52:09.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:08 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:52:09.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:08 smithi097 ceph-mon[31955]: osd.4 [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] boot 2024-03-26T18:52:09.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:08 smithi097 ceph-mon[31955]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T18:52:09.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:08 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:52:09.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:09.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T18:52:10.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:10.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:10.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:10.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:10.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:10.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:10.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:10.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:10.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:10.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:10.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.439+0000 7fc313949700 1 -- 172.21.15.144:0/170632149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30c1024e0 msgr2=0x7fc30c102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:52:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.439+0000 7fc313949700 1 --2- 172.21.15.144:0/170632149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30c1024e0 0x7fc30c102940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009990 tx=0x7fc2fc009ca0 comp rx=0 tx=0).stop 2024-03-26T18:52:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.439+0000 7fc313949700 1 -- 172.21.15.144:0/170632149 shutdown_connections 2024-03-26T18:52:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.439+0000 7fc313949700 1 --2- 172.21.15.144:0/170632149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc30c102e80 0x7fc30c109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.439+0000 7fc313949700 1 --2- 172.21.15.144:0/170632149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc30c1012e0 0x7fc30c101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.439+0000 7fc313949700 1 --2- 172.21.15.144:0/170632149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30c1024e0 0x7fc30c102940 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.439+0000 7fc313949700 1 -- 172.21.15.144:0/170632149 >> 172.21.15.144:0/170632149 conn(0x7fc30c0fc8a0 msgr2=0x7fc30c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:52:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.439+0000 7fc313949700 1 -- 172.21.15.144:0/170632149 shutdown_connections 2024-03-26T18:52:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.439+0000 7fc313949700 1 -- 172.21.15.144:0/170632149 wait complete. 2024-03-26T18:52:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.439+0000 7fc313949700 1 Processor -- start 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc313949700 1 -- start start 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc313949700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc30c1012e0 0x7fc30c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc313949700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30c1024e0 0x7fc30c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc313949700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc30c102e80 0x7fc30c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc313949700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c10cfc0 con 0x7fc30c102e80 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc313949700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c10ce40 con 0x7fc30c1012e0 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc313949700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c10d140 con 0x7fc30c1024e0 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc310ee4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30c1024e0 0x7fc30c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc310ee4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30c1024e0 0x7fc30c198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53928/0 (socket says 172.21.15.144:53928) 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc310ee4700 1 -- 172.21.15.144:0/2885147313 learned_addr learned my addr 172.21.15.144:0/2885147313 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc3116e5700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc30c1012e0 0x7fc30c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:52:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc310ee4700 1 -- 172.21.15.144:0/2885147313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc30c1012e0 msgr2=0x7fc30c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:52:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc310ee4700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc30c1012e0 0x7fc30c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc310ee4700 1 -- 172.21.15.144:0/2885147313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc30c102e80 msgr2=0x7fc30c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:52:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc310ee4700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc30c102e80 0x7fc30c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc310ee4700 1 -- 172.21.15.144:0/2885147313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c1a03b0 con 0x7fc30c1024e0 2024-03-26T18:52:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc310ee4700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30c1024e0 0x7fc30c198ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0047b0 tx=0x7fc2fc003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:52:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.440+0000 7fc3027fc700 1 -- 172.21.15.144:0/2885147313 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc027070 con 0x7fc30c1024e0 2024-03-26T18:52:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.441+0000 7fc313949700 1 -- 172.21.15.144:0/2885147313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30c1a0640 con 0x7fc30c1024e0 2024-03-26T18:52:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.441+0000 7fc3027fc700 1 -- 172.21.15.144:0/2885147313 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2fc022470 con 0x7fc30c1024e0 2024-03-26T18:52:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.441+0000 7fc3027fc700 1 -- 172.21.15.144:0/2885147313 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00f6c0 con 0x7fc30c1024e0 2024-03-26T18:52:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.441+0000 7fc313949700 1 -- 172.21.15.144:0/2885147313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30c1a0b70 con 0x7fc30c1024e0 2024-03-26T18:52:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.442+0000 7fc313949700 1 -- 172.21.15.144:0/2885147313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30c069070 con 0x7fc30c1024e0 2024-03-26T18:52:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.443+0000 7fc3027fc700 1 -- 172.21.15.144:0/2885147313 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fc2fc02a070 con 0x7fc30c1024e0 2024-03-26T18:52:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.443+0000 7fc3027fc700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2f80a2fb0 0x7fc2f80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:52:10.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.444+0000 7fc3027fc700 1 -- 172.21.15.144:0/2885147313 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fc2fc0cf920 con 0x7fc30c1024e0 2024-03-26T18:52:10.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.444+0000 7fc3116e5700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2f80a2fb0 0x7fc2f80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:52:10.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.445+0000 7fc3116e5700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2f80a2fb0 0x7fc2f80a5470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc308006fd0 tx=0x7fc308008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:52:10.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.447+0000 7fc3027fc700 1 -- 172.21.15.144:0/2885147313 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc2fc031070 con 0x7fc30c1024e0 2024-03-26T18:52:10.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:10.651+0000 7fc313949700 1 -- 172.21.15.144:0/2885147313 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc30c0feac0 con 0x7fc2f80a2fb0 2024-03-26T18:52:11.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:10 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi105 2024-03-26T18:52:11.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:10 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi105 to 8540M 2024-03-26T18:52:11.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:52:11.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:52:11.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:10 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:11.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:10 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi105 2024-03-26T18:52:11.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:10 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi105 to 8540M 2024-03-26T18:52:11.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:52:11.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:52:11.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:10 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:11.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:10 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi105 2024-03-26T18:52:11.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:10 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi105 to 8540M 2024-03-26T18:52:11.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:52:11.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:52:11.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:10 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:11.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:11 smithi144 ceph-mon[36053]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:52:11.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:11 smithi144 ceph-mon[36053]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:12.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:11 smithi097 ceph-mon[31955]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:52:12.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:11 smithi097 ceph-mon[31955]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:11 smithi105 ceph-mon[36270]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:52:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:11 smithi105 ceph-mon[36270]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:14.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:13 smithi105 ceph-mon[36270]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:14.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:13 smithi105 ceph-mon[36270]: from='client.? 172.21.15.144:0/1088362549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62"}]: dispatch 2024-03-26T18:52:14.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:13 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62"}]: dispatch 2024-03-26T18:52:14.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:13 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62"}]': finished 2024-03-26T18:52:14.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:13 smithi105 ceph-mon[36270]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T18:52:14.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:13 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:14.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:13 smithi097 ceph-mon[31955]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:14.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:13 smithi097 ceph-mon[31955]: from='client.? 172.21.15.144:0/1088362549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62"}]: dispatch 2024-03-26T18:52:14.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:13 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62"}]: dispatch 2024-03-26T18:52:14.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:13 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62"}]': finished 2024-03-26T18:52:14.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:13 smithi097 ceph-mon[31955]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T18:52:14.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:13 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:14.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:13 smithi144 ceph-mon[36053]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:14.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:13 smithi144 ceph-mon[36053]: from='client.? 172.21.15.144:0/1088362549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62"}]: dispatch 2024-03-26T18:52:14.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:13 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62"}]: dispatch 2024-03-26T18:52:14.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:13 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62"}]': finished 2024-03-26T18:52:14.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:13 smithi144 ceph-mon[36053]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T18:52:14.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:13 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:15.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:14 smithi105 ceph-mon[36270]: from='client.? 172.21.15.144:0/1844262206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:52:15.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:14 smithi097 ceph-mon[31955]: from='client.? 172.21.15.144:0/1844262206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:52:15.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:14 smithi144 ceph-mon[36053]: from='client.? 172.21.15.144:0/1844262206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:52:16.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:15 smithi105 ceph-mon[36270]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:16.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:15 smithi097 ceph-mon[31955]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:16.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:15 smithi144 ceph-mon[36053]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:18.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:17 smithi144 ceph-mon[36053]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:18.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:17 smithi105 ceph-mon[36270]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:18.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:17 smithi097 ceph-mon[31955]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:20.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:19 smithi105 ceph-mon[36270]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:20.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:19 smithi097 ceph-mon[31955]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:20.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:19 smithi144 ceph-mon[36053]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:21.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T18:52:21.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:21.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T18:52:21.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:21.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T18:52:21.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:22.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:21 smithi144 ceph-mon[36053]: Deploying daemon osd.5 on smithi144 2024-03-26T18:52:22.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:21 smithi144 ceph-mon[36053]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:22.208 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:21 smithi105 ceph-mon[36270]: Deploying daemon osd.5 on smithi144 2024-03-26T18:52:22.208 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:21 smithi105 ceph-mon[36270]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:22.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:21 smithi097 ceph-mon[31955]: Deploying daemon osd.5 on smithi144 2024-03-26T18:52:22.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:21 smithi097 ceph-mon[31955]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:24.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:23 smithi144 ceph-mon[36053]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:24.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:23 smithi105 ceph-mon[36270]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:24.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:23 smithi097 ceph-mon[31955]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:26.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:25 smithi144 ceph-mon[36053]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:26.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:25 smithi105 ceph-mon[36270]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:26.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:25 smithi097 ceph-mon[31955]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:27.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:27.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:27.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:27.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:27 smithi097 ceph-mon[31955]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:27 smithi144 ceph-mon[36053]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:27 smithi105 ceph-mon[36270]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:29.960 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:29 smithi144 ceph-mon[36053]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:29.960 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:29.960 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:29.960 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:29.960 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:29.960 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:29 smithi105 ceph-mon[36270]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:30.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.335+0000 7fc3027fc700 1 -- 172.21.15.144:0/2885147313 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc30c0feac0 con 0x7fc2f80a2fb0 2024-03-26T18:52:30.328 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 5 on host 'smithi144' 2024-03-26T18:52:30.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 -- 172.21.15.144:0/2885147313 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2f80a2fb0 msgr2=0x7fc2f80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:52:30.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2f80a2fb0 0x7fc2f80a5470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc308006fd0 tx=0x7fc308008040 comp rx=0 tx=0).stop 2024-03-26T18:52:30.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 -- 172.21.15.144:0/2885147313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30c1024e0 msgr2=0x7fc30c198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:52:30.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30c1024e0 0x7fc30c198ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0047b0 tx=0x7fc2fc003680 comp rx=0 tx=0).stop 2024-03-26T18:52:30.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 -- 172.21.15.144:0/2885147313 shutdown_connections 2024-03-26T18:52:30.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc30c102e80 0x7fc30c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:30.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc30c1012e0 0x7fc30c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:30.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2f80a2fb0 0x7fc2f80a5470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:30.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 --2- 172.21.15.144:0/2885147313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30c1024e0 0x7fc30c198ce0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:30.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 -- 172.21.15.144:0/2885147313 >> 172.21.15.144:0/2885147313 conn(0x7fc30c0fc8a0 msgr2=0x7fc30c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:52:30.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 -- 172.21.15.144:0/2885147313 shutdown_connections 2024-03-26T18:52:30.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:52:30.337+0000 7fc313949700 1 -- 172.21.15.144:0/2885147313 wait complete. 2024-03-26T18:52:30.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:29 smithi097 ceph-mon[31955]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:30.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:30.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:30.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:30.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:30.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:30.974 DEBUG:teuthology.orchestra.run.smithi144:osd.5> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.5.service 2024-03-26T18:52:30.976 INFO:tasks.cephadm:Deploying osd.6 on smithi097 with /dev/vg_nvme/lv_2... 2024-03-26T18:52:30.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T18:52:31.401 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:52:31.545 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:31.545 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:31.545 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:31.545 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:31 smithi097 ceph-mon[31955]: from='osd.5 [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T18:52:31.545 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:31 smithi097 ceph-mon[31955]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T18:52:31.545 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:31 smithi097 ceph-mon[31955]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:31.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:31 smithi144 ceph-mon[36053]: from='osd.5 [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T18:52:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:31 smithi144 ceph-mon[36053]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T18:52:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:31 smithi144 ceph-mon[36053]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:31.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:31.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:31.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:52:31.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:31 smithi105 ceph-mon[36270]: from='osd.5 [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T18:52:31.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:31 smithi105 ceph-mon[36270]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T18:52:31.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:31 smithi105 ceph-mon[36270]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:32.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:32 smithi097 ceph-mon[31955]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T18:52:32.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:32 smithi097 ceph-mon[31955]: from='osd.5 [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:52:32.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:32 smithi097 ceph-mon[31955]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T18:52:32.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:32 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:32.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:32 smithi097 ceph-mon[31955]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:52:32.585 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:32 smithi144 ceph-mon[36053]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T18:52:32.585 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:32 smithi144 ceph-mon[36053]: from='osd.5 [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:52:32.585 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:32 smithi144 ceph-mon[36053]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T18:52:32.585 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:32 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:32.585 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:32 smithi144 ceph-mon[36053]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:52:32.586 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:52:32 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T18:52:32.302+0000 7fe651506700 -1 osd.5 0 waiting for initial osdmap 2024-03-26T18:52:32.586 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:52:32 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T18:52:32.312+0000 7fe64d2f9700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:52:32.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:32 smithi105 ceph-mon[36270]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T18:52:32.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:32 smithi105 ceph-mon[36270]: from='osd.5 [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:52:32.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:32 smithi105 ceph-mon[36270]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T18:52:32.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:32 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:32.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:32 smithi105 ceph-mon[36270]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:52:33.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:33 smithi097 ceph-mon[31955]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:52:33.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:33 smithi097 ceph-mon[31955]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T18:52:33.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:33 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:33.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:33 smithi097 ceph-mon[31955]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:33.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:33 smithi097 ceph-mon[31955]: osd.5 [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] boot 2024-03-26T18:52:33.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:33 smithi097 ceph-mon[31955]: osdmap e38: 6 total, 6 up, 6 in 2024-03-26T18:52:33.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:33 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:33.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:33 smithi144 ceph-mon[36053]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:52:33.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:33 smithi144 ceph-mon[36053]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T18:52:33.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:33 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:33.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:33 smithi144 ceph-mon[36053]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:33.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:33 smithi144 ceph-mon[36053]: osd.5 [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] boot 2024-03-26T18:52:33.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:33 smithi144 ceph-mon[36053]: osdmap e38: 6 total, 6 up, 6 in 2024-03-26T18:52:33.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:33 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:33.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:33 smithi105 ceph-mon[36270]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:52:33.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:33 smithi105 ceph-mon[36270]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T18:52:33.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:33 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:33.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:33 smithi105 ceph-mon[36270]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T18:52:33.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:33 smithi105 ceph-mon[36270]: osd.5 [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] boot 2024-03-26T18:52:33.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:33 smithi105 ceph-mon[36270]: osdmap e38: 6 total, 6 up, 6 in 2024-03-26T18:52:33.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:33 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:52:34.574 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:52:34.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:52:34.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:34.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:34.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:34.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:34.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:34 smithi097 ceph-mon[31955]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T18:52:34.575 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T18:52:34.575 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T18:52:34.575 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-26T18:52:34.575 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-26T18:52:34.576 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0350793 s, 299 MB/s 2024-03-26T18:52:34.576 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-26T18:52:34.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:52:34.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:52:34.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:34.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:34.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:34.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:34.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:34 smithi144 ceph-mon[36053]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T18:52:34.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:52:34.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:52:34.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:34.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:52:34.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:34.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:52:34.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:52:34.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:34 smithi105 ceph-mon[36270]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T18:52:35.505 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-03-26T18:52:35.590 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:35 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi144 2024-03-26T18:52:35.590 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:35 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi144 to 8540M 2024-03-26T18:52:35.590 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:35 smithi097 ceph-mon[31955]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:35.590 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:35 smithi097 ceph-mon[31955]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T18:52:35.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:35 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi144 2024-03-26T18:52:35.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:35 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi144 to 8540M 2024-03-26T18:52:35.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:35 smithi144 ceph-mon[36053]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:35.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:35 smithi144 ceph-mon[36053]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T18:52:35.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:35 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi144 2024-03-26T18:52:35.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:35 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi144 to 8540M 2024-03-26T18:52:35.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:35 smithi105 ceph-mon[36270]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:35.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:35 smithi105 ceph-mon[36270]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T18:52:35.942 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:52:37.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:37 smithi097 ceph-mon[31955]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T18:52:37.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:37 smithi097 ceph-mon[31955]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:37.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:37 smithi144 ceph-mon[36053]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T18:52:37.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:37 smithi144 ceph-mon[36053]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:37.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:37 smithi105 ceph-mon[36270]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T18:52:37.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:37 smithi105 ceph-mon[36270]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:38.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.255+0000 7f0b18e00700 1 -- 172.21.15.97:0/2672542735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b14103f80 msgr2=0x7f0b1410a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:52:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.255+0000 7f0b18e00700 1 --2- 172.21.15.97:0/2672542735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b14103f80 0x7f0b1410a830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00b0d0 tx=0x7f0b0c00b3e0 comp rx=0 tx=0).stop 2024-03-26T18:52:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.255+0000 7f0b18e00700 1 -- 172.21.15.97:0/2672542735 shutdown_connections 2024-03-26T18:52:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.255+0000 7f0b18e00700 1 --2- 172.21.15.97:0/2672542735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b14103f80 0x7f0b1410a830 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.255+0000 7f0b18e00700 1 --2- 172.21.15.97:0/2672542735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b141023e0 0x7f0b14102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.255+0000 7f0b18e00700 1 --2- 172.21.15.97:0/2672542735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b141035e0 0x7f0b14103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.255+0000 7f0b18e00700 1 -- 172.21.15.97:0/2672542735 >> 172.21.15.97:0/2672542735 conn(0x7f0b140fd9c0 msgr2=0x7f0b140ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:52:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.255+0000 7f0b18e00700 1 -- 172.21.15.97:0/2672542735 shutdown_connections 2024-03-26T18:52:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.255+0000 7f0b18e00700 1 -- 172.21.15.97:0/2672542735 wait complete. 2024-03-26T18:52:38.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.256+0000 7f0b18e00700 1 Processor -- start 2024-03-26T18:52:38.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.256+0000 7f0b18e00700 1 -- start start 2024-03-26T18:52:38.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b18e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b141023e0 0x7f0b141081e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:52:38.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b18e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b141035e0 0x7f0b14108720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:52:38.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b18e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b14103f80 0x7f0b14104d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:52:38.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b18e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1410dfd0 con 0x7f0b14103f80 2024-03-26T18:52:38.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b18e00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1410de50 con 0x7f0b141035e0 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b18e00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1410e150 con 0x7f0b141023e0 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b12800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b14103f80 0x7f0b14104d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b11400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b141035e0 0x7f0b14108720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b11e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b141023e0 0x7f0b141081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b11e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b141023e0 0x7f0b141081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:43212/0 (socket says 172.21.15.97:43212) 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b11400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b141035e0 0x7f0b14108720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:55008/0 (socket says 172.21.15.97:55008) 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b11e00700 1 -- 172.21.15.97:0/996949276 learned_addr learned my addr 172.21.15.97:0/996949276 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b11400700 1 -- 172.21.15.97:0/996949276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b141023e0 msgr2=0x7f0b141081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b11400700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b141023e0 0x7f0b141081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b11400700 1 -- 172.21.15.97:0/996949276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b14103f80 msgr2=0x7f0b14104d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b11400700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b14103f80 0x7f0b14104d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b11400700 1 -- 172.21.15.97:0/996949276 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b14105630 con 0x7f0b141035e0 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.257+0000 7f0b12800700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b14103f80 0x7f0b14104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:52:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.258+0000 7f0b11400700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b141035e0 0x7f0b14108720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000e9f0 tx=0x7f0b0000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:52:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.258+0000 7f0b0aa00700 1 -- 172.21.15.97:0/996949276 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000c9c0 con 0x7f0b141035e0 2024-03-26T18:52:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.258+0000 7f0b18e00700 1 -- 172.21.15.97:0/996949276 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b141a6c20 con 0x7f0b141035e0 2024-03-26T18:52:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.258+0000 7f0b0aa00700 1 -- 172.21.15.97:0/996949276 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b0000cb60 con 0x7f0b141035e0 2024-03-26T18:52:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.258+0000 7f0b0aa00700 1 -- 172.21.15.97:0/996949276 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00018680 con 0x7f0b141035e0 2024-03-26T18:52:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.258+0000 7f0b18e00700 1 -- 172.21.15.97:0/996949276 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b141a70d0 con 0x7f0b141035e0 2024-03-26T18:52:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.260+0000 7f0b18e00700 1 -- 172.21.15.97:0/996949276 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b14069070 con 0x7f0b141035e0 2024-03-26T18:52:38.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.261+0000 7f0b0aa00700 1 -- 172.21.15.97:0/996949276 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f0b00010430 con 0x7f0b141035e0 2024-03-26T18:52:38.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.263+0000 7f0b0aa00700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0af80a2fb0 0x7f0af80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:52:38.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.263+0000 7f0b0aa00700 1 -- 172.21.15.97:0/996949276 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f0b000ce6c0 con 0x7f0b141035e0 2024-03-26T18:52:38.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.263+0000 7f0b11e00700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0af80a2fb0 0x7f0af80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:52:38.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.263+0000 7f0b11e00700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0af80a2fb0 0x7f0af80a5470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005950 tx=0x7f0b04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:52:38.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.267+0000 7f0b0aa00700 1 -- 172.21.15.97:0/996949276 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0b0009c310 con 0x7f0b141035e0 2024-03-26T18:52:38.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:52:38.502+0000 7f0b18e00700 1 -- 172.21.15.97:0/996949276 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0b14075d00 con 0x7f0af80a2fb0 2024-03-26T18:52:38.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:38 smithi097 ceph-mon[31955]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T18:52:38.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:38 smithi144 ceph-mon[36053]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T18:52:38.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:38 smithi105 ceph-mon[36270]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T18:52:39.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:39 smithi097 ceph-mon[31955]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:52:39.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:52:39.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:52:39.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:39.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:39 smithi097 ceph-mon[31955]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-26T18:52:39.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:39 smithi144 ceph-mon[36053]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:52:39.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:52:39.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:52:39.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:39.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:39 smithi144 ceph-mon[36053]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-26T18:52:39.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:39 smithi105 ceph-mon[36270]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:52:39.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:52:39.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:52:39.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:39.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:39 smithi105 ceph-mon[36270]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-26T18:52:42.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:41 smithi105 ceph-mon[36270]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T18:52:42.329 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:41 smithi144 ceph-mon[36053]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T18:52:42.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:41 smithi097 ceph-mon[31955]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T18:52:43.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:42 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3194295978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b32d8dc-0311-47c2-a2c4-4e324b70eefd"}]: dispatch 2024-03-26T18:52:43.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:42 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3194295978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b32d8dc-0311-47c2-a2c4-4e324b70eefd"}]': finished 2024-03-26T18:52:43.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:42 smithi097 ceph-mon[31955]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T18:52:43.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:42 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:52:43.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:42 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3194295978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b32d8dc-0311-47c2-a2c4-4e324b70eefd"}]: dispatch 2024-03-26T18:52:43.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:42 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3194295978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b32d8dc-0311-47c2-a2c4-4e324b70eefd"}]': finished 2024-03-26T18:52:43.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:42 smithi105 ceph-mon[36270]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T18:52:43.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:42 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:52:43.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:42 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3194295978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b32d8dc-0311-47c2-a2c4-4e324b70eefd"}]: dispatch 2024-03-26T18:52:43.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:42 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3194295978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b32d8dc-0311-47c2-a2c4-4e324b70eefd"}]': finished 2024-03-26T18:52:43.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:42 smithi144 ceph-mon[36053]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T18:52:43.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:42 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:52:44.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:43 smithi105 ceph-mon[36270]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-26T18:52:44.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:43 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1749395455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:52:44.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:43 smithi097 ceph-mon[31955]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-26T18:52:44.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:43 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1749395455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:52:44.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:43 smithi144 ceph-mon[36053]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-26T18:52:44.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:43 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1749395455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:52:46.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:45 smithi105 ceph-mon[36270]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:52:46.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:45 smithi097 ceph-mon[31955]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:52:46.378 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:45 smithi144 ceph-mon[36053]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:52:48.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:47 smithi097 ceph-mon[31955]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-26T18:52:48.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:47 smithi105 ceph-mon[36270]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-26T18:52:48.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:47 smithi144 ceph-mon[36053]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-26T18:52:50.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:49 smithi097 ceph-mon[31955]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:50.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:49 smithi105 ceph-mon[36270]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:50.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:49 smithi144 ceph-mon[36053]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:52.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:51 smithi097 ceph-mon[31955]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:52.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:51 smithi105 ceph-mon[36270]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:52.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:51 smithi144 ceph-mon[36053]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:53.183 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:52 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T18:52:53.183 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:52 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:53.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:52 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T18:52:53.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:52 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:53.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:52 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T18:52:53.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:52 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:52:54.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:53 smithi105 ceph-mon[36270]: Deploying daemon osd.6 on smithi097 2024-03-26T18:52:54.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:53 smithi105 ceph-mon[36270]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:54.228 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:53 smithi097 ceph-mon[31955]: Deploying daemon osd.6 on smithi097 2024-03-26T18:52:54.229 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:53 smithi097 ceph-mon[31955]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:54.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:53 smithi144 ceph-mon[36053]: Deploying daemon osd.6 on smithi097 2024-03-26T18:52:54.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:53 smithi144 ceph-mon[36053]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:56.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:55 smithi105 ceph-mon[36270]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:56.210 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:55 smithi097 ceph-mon[31955]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:56.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:55 smithi144 ceph-mon[36053]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:57 smithi105 ceph-mon[36270]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:58.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:57 smithi097 ceph-mon[31955]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:52:58.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:57 smithi144 ceph-mon[36053]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:00.094 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:52:59 smithi097 ceph-mon[31955]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:00.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:52:59 smithi105 ceph-mon[36270]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:00.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:52:59 smithi144 ceph-mon[36053]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:01.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:01.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:01.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:01.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:01 smithi097 ceph-mon[31955]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:01.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:01.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:01.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:01.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:01 smithi105 ceph-mon[36270]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:02.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:02.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:02.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:02.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:01 smithi144 ceph-mon[36053]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:03.945 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:03 smithi097 ceph-mon[31955]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:04.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:03 smithi105 ceph-mon[36270]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:04.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:03 smithi144 ceph-mon[36053]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:05.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:05.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:05.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:05 smithi097 ceph-mon[31955]: from='osd.6 [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T18:53:05.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:05 smithi097 ceph-mon[31955]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T18:53:05.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:05 smithi097 ceph-mon[31955]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:05.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:05.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:05.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:05 smithi144 ceph-mon[36053]: from='osd.6 [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T18:53:05.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:05 smithi144 ceph-mon[36053]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T18:53:05.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:05 smithi144 ceph-mon[36053]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:05.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:05.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:05.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:05 smithi105 ceph-mon[36270]: from='osd.6 [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T18:53:05.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:05 smithi105 ceph-mon[36270]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T18:53:05.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:05 smithi105 ceph-mon[36270]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:05.966 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 6 on host 'smithi097' 2024-03-26T18:53:05.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.974+0000 7f0b0aa00700 1 -- 172.21.15.97:0/996949276 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0b14075d00 con 0x7f0af80a2fb0 2024-03-26T18:53:05.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.977+0000 7f0b18e00700 1 -- 172.21.15.97:0/996949276 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0af80a2fb0 msgr2=0x7f0af80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:05.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.977+0000 7f0b18e00700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0af80a2fb0 0x7f0af80a5470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005950 tx=0x7f0b04009260 comp rx=0 tx=0).stop 2024-03-26T18:53:05.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.977+0000 7f0b18e00700 1 -- 172.21.15.97:0/996949276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b141035e0 msgr2=0x7f0b14108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:05.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.977+0000 7f0b18e00700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b141035e0 0x7f0b14108720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000e9f0 tx=0x7f0b0000ed00 comp rx=0 tx=0).stop 2024-03-26T18:53:05.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.978+0000 7f0b18e00700 1 -- 172.21.15.97:0/996949276 shutdown_connections 2024-03-26T18:53:05.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.978+0000 7f0b18e00700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b14103f80 0x7f0b14104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:05.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.978+0000 7f0b18e00700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0af80a2fb0 0x7f0af80a5470 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:05.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.978+0000 7f0b18e00700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b141023e0 0x7f0b141081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:05.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.978+0000 7f0b18e00700 1 --2- 172.21.15.97:0/996949276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b141035e0 0x7f0b14108720 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:05.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.978+0000 7f0b18e00700 1 -- 172.21.15.97:0/996949276 >> 172.21.15.97:0/996949276 conn(0x7f0b140fd9c0 msgr2=0x7f0b140745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:53:05.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.978+0000 7f0b18e00700 1 -- 172.21.15.97:0/996949276 shutdown_connections 2024-03-26T18:53:05.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:53:05.978+0000 7f0b18e00700 1 -- 172.21.15.97:0/996949276 wait complete. 2024-03-26T18:53:06.750 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T18:53:06.750 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: from='osd.6 [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:53:06.751 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T18:53:06.751 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:06.751 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:53:06.751 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:06.751 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:06.751 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:06.751 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:06.751 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:06.751 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:06.807 DEBUG:teuthology.orchestra.run.smithi097:osd.6> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.6.service 2024-03-26T18:53:06.811 INFO:tasks.cephadm:Deploying osd.7 on smithi105 with /dev/vg_nvme/lv_2... 2024-03-26T18:53:06.811 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T18:53:06.841 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T18:53:06.841 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: from='osd.6 [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:53:06.841 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T18:53:06.841 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:06.841 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:53:06.841 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:06.841 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:06.842 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:06.842 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:06.842 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:06.842 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: from='osd.6 [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:06.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:07.056 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:53:07.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:07 smithi097 ceph-mon[31955]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:53:07.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:07 smithi097 ceph-mon[31955]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T18:53:07.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:07.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:07.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:07 smithi097 ceph-mon[31955]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:07.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:07 smithi144 ceph-mon[36053]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:53:07.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:07 smithi144 ceph-mon[36053]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T18:53:07.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:07.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:07.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:07 smithi144 ceph-mon[36053]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:07 smithi105 ceph-mon[36270]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:53:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:07 smithi105 ceph-mon[36270]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T18:53:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:07 smithi105 ceph-mon[36270]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-26T18:53:08.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:08 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:53:08.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:08 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:53:08.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:08 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:08.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:08 smithi105 ceph-mon[36270]: osd.6 [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] boot 2024-03-26T18:53:08.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:08 smithi105 ceph-mon[36270]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T18:53:08.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:08 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:08.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:08 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:53:08.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:08 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:53:08.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:08 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:08.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:08 smithi097 ceph-mon[31955]: osd.6 [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] boot 2024-03-26T18:53:08.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:08 smithi097 ceph-mon[31955]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T18:53:08.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:08 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:08.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:08 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:53:08.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:08 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:53:08.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:08 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:08.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:08 smithi144 ceph-mon[36053]: osd.6 [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] boot 2024-03-26T18:53:08.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:08 smithi144 ceph-mon[36053]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T18:53:08.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:08 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:53:08.988 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T18:53:08.988 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T18:53:08.988 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-26T18:53:08.988 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-26T18:53:08.988 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121154 s, 865 MB/s 2024-03-26T18:53:08.988 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-26T18:53:09.636 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_2 2024-03-26T18:53:09.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:09 smithi097 ceph-mon[31955]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T18:53:09.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:09 smithi097 ceph-mon[31955]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:09.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:09 smithi144 ceph-mon[36053]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T18:53:09.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:09 smithi144 ceph-mon[36053]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:09.921 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:53:09.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:09 smithi105 ceph-mon[36270]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T18:53:09.959 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:09 smithi105 ceph-mon[36270]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:11.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.595+0000 7fe574ce1700 1 -- 172.21.15.105:0/1008232449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe570102e70 msgr2=0x7fe570109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:11.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.595+0000 7fe574ce1700 1 --2- 172.21.15.105:0/1008232449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe570102e70 0x7fe570109720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe56800b0d0 tx=0x7fe56800b3e0 comp rx=0 tx=0).stop 2024-03-26T18:53:11.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.595+0000 7fe574ce1700 1 -- 172.21.15.105:0/1008232449 shutdown_connections 2024-03-26T18:53:11.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.595+0000 7fe574ce1700 1 --2- 172.21.15.105:0/1008232449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe570102e70 0x7fe570109720 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:11.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.595+0000 7fe574ce1700 1 --2- 172.21.15.105:0/1008232449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5701012d0 0x7fe5701016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:11.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.595+0000 7fe574ce1700 1 --2- 172.21.15.105:0/1008232449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5701024d0 0x7fe570102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:11.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.595+0000 7fe574ce1700 1 -- 172.21.15.105:0/1008232449 >> 172.21.15.105:0/1008232449 conn(0x7fe5700fc870 msgr2=0x7fe5700fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:53:11.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.595+0000 7fe574ce1700 1 -- 172.21.15.105:0/1008232449 shutdown_connections 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.595+0000 7fe574ce1700 1 -- 172.21.15.105:0/1008232449 wait complete. 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.595+0000 7fe574ce1700 1 Processor -- start 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe574ce1700 1 -- start start 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe574ce1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5701012d0 0x7fe570197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe574ce1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5701024d0 0x7fe570197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe574ce1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe570102e70 0x7fe57019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe574ce1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe57010cf30 con 0x7fe5701024d0 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe574ce1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe57010cdb0 con 0x7fe570102e70 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe574ce1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe57010d0b0 con 0x7fe5701012d0 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe56ed9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe570102e70 0x7fe57019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe56ed9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe570102e70 0x7fe57019ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42702/0 (socket says 172.21.15.105:42702) 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe56ed9d700 1 -- 172.21.15.105:0/4070894347 learned_addr learned my addr 172.21.15.105:0/4070894347 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:53:11.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe56e59c700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5701012d0 0x7fe570197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe56ed9d700 1 -- 172.21.15.105:0/4070894347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5701012d0 msgr2=0x7fe570197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe56ed9d700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5701012d0 0x7fe570197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe56ed9d700 1 -- 172.21.15.105:0/4070894347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5701024d0 msgr2=0x7fe570197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe56ed9d700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5701024d0 0x7fe570197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.596+0000 7fe56ed9d700 1 -- 172.21.15.105:0/4070894347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57019f0b0 con 0x7fe570102e70 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.597+0000 7fe56ed9d700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe570102e70 0x7fe57019ea10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe5680062a0 tx=0x7fe568007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.597+0000 7fe567fff700 1 -- 172.21.15.105:0/4070894347 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568003c00 con 0x7fe570102e70 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.597+0000 7fe567fff700 1 -- 172.21.15.105:0/4070894347 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe568003da0 con 0x7fe570102e70 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.597+0000 7fe574ce1700 1 -- 172.21.15.105:0/4070894347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57019f340 con 0x7fe570102e70 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.597+0000 7fe567fff700 1 -- 172.21.15.105:0/4070894347 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568005560 con 0x7fe570102e70 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.597+0000 7fe574ce1700 1 -- 172.21.15.105:0/4070894347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57019f8a0 con 0x7fe570102e70 2024-03-26T18:53:11.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.598+0000 7fe574ce1700 1 -- 172.21.15.105:0/4070894347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe570069070 con 0x7fe570102e70 2024-03-26T18:53:11.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.599+0000 7fe567fff700 1 -- 172.21.15.105:0/4070894347 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fe568005700 con 0x7fe570102e70 2024-03-26T18:53:11.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.599+0000 7fe567fff700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe55c0a2fb0 0x7fe55c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:53:11.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.599+0000 7fe567fff700 1 -- 172.21.15.105:0/4070894347 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7fe56800f020 con 0x7fe570102e70 2024-03-26T18:53:11.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.600+0000 7fe56e59c700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe55c0a2fb0 0x7fe55c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:53:11.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.601+0000 7fe56e59c700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe55c0a2fb0 0x7fe55c0a5470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe560006fd0 tx=0x7fe560008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:53:11.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.604+0000 7fe567fff700 1 -- 172.21.15.105:0/4070894347 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe56802b020 con 0x7fe570102e70 2024-03-26T18:53:11.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:11.809+0000 7fe574ce1700 1 -- 172.21.15.105:0/4070894347 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe5700627f0 con 0x7fe55c0a2fb0 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:53:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:53:12.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:12.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:12.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:12.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:12.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:12.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:12.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:12.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:53:12.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:53:12.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:12.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:53:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:13.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:12 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi097 2024-03-26T18:53:13.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:12 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi097 to 5697M 2024-03-26T18:53:13.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:12 smithi105 ceph-mon[36270]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:53:13.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:12 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi097 2024-03-26T18:53:13.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:12 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi097 to 5697M 2024-03-26T18:53:13.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:12 smithi097 ceph-mon[31955]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:53:13.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:12 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi097 2024-03-26T18:53:13.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:12 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi097 to 5697M 2024-03-26T18:53:13.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:12 smithi144 ceph-mon[36053]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:53:14.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:13 smithi105 ceph-mon[36270]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:14.241 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:13 smithi097 ceph-mon[31955]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:14.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:13 smithi144 ceph-mon[36053]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:15.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:14 smithi105 ceph-mon[36270]: from='client.? 172.21.15.105:0/2663672847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf3ac56-47cf-4b95-ba5f-5d8e72185128"}]: dispatch 2024-03-26T18:53:15.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:14 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf3ac56-47cf-4b95-ba5f-5d8e72185128"}]: dispatch 2024-03-26T18:53:15.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:14 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbf3ac56-47cf-4b95-ba5f-5d8e72185128"}]': finished 2024-03-26T18:53:15.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:14 smithi105 ceph-mon[36270]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T18:53:15.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:14 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:15.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:14 smithi097 ceph-mon[31955]: from='client.? 172.21.15.105:0/2663672847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf3ac56-47cf-4b95-ba5f-5d8e72185128"}]: dispatch 2024-03-26T18:53:15.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:14 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf3ac56-47cf-4b95-ba5f-5d8e72185128"}]: dispatch 2024-03-26T18:53:15.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:14 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbf3ac56-47cf-4b95-ba5f-5d8e72185128"}]': finished 2024-03-26T18:53:15.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:14 smithi097 ceph-mon[31955]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T18:53:15.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:15.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:14 smithi144 ceph-mon[36053]: from='client.? 172.21.15.105:0/2663672847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf3ac56-47cf-4b95-ba5f-5d8e72185128"}]: dispatch 2024-03-26T18:53:15.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:14 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf3ac56-47cf-4b95-ba5f-5d8e72185128"}]: dispatch 2024-03-26T18:53:15.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:14 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbf3ac56-47cf-4b95-ba5f-5d8e72185128"}]': finished 2024-03-26T18:53:15.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:14 smithi144 ceph-mon[36053]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T18:53:15.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:14 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:16.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:15 smithi105 ceph-mon[36270]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:16.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:15 smithi105 ceph-mon[36270]: from='client.? 172.21.15.105:0/2639140085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:53:16.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:15 smithi097 ceph-mon[31955]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:16.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:15 smithi097 ceph-mon[31955]: from='client.? 172.21.15.105:0/2639140085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:53:16.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:15 smithi144 ceph-mon[36053]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:16.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:15 smithi144 ceph-mon[36053]: from='client.? 172.21.15.105:0/2639140085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:53:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:17 smithi105 ceph-mon[36270]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:18.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:17 smithi097 ceph-mon[31955]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:18.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:17 smithi144 ceph-mon[36053]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:20.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:19 smithi105 ceph-mon[36270]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:20.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:19 smithi097 ceph-mon[31955]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:20.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:19 smithi144 ceph-mon[36053]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:22.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:21 smithi105 ceph-mon[36270]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:22.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:21 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T18:53:22.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:21 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:22.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:21 smithi097 ceph-mon[31955]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:22.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:21 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T18:53:22.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:21 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:22.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:21 smithi144 ceph-mon[36053]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:22.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:21 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T18:53:22.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:21 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:23.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:22 smithi105 ceph-mon[36270]: Deploying daemon osd.7 on smithi105 2024-03-26T18:53:23.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:22 smithi097 ceph-mon[31955]: Deploying daemon osd.7 on smithi105 2024-03-26T18:53:23.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:22 smithi144 ceph-mon[36053]: Deploying daemon osd.7 on smithi105 2024-03-26T18:53:24.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:23 smithi105 ceph-mon[36270]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:24.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:23 smithi097 ceph-mon[31955]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:24.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:23 smithi144 ceph-mon[36053]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:26.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:25 smithi105 ceph-mon[36270]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:26.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:25 smithi097 ceph-mon[31955]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:26.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:25 smithi144 ceph-mon[36053]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:28.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:27 smithi105 ceph-mon[36270]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:28.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:28.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:28.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:28.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:27 smithi097 ceph-mon[31955]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:28.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:28.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:28.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:28.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:27 smithi144 ceph-mon[36053]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:28.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:28.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:28.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:29 smithi105 ceph-mon[36270]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:30.266 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:29 smithi097 ceph-mon[31955]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:30.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:29 smithi144 ceph-mon[36053]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.089+0000 7fe567fff700 1 -- 172.21.15.105:0/4070894347 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe5700627f0 con 0x7fe55c0a2fb0 2024-03-26T18:53:31.082 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 7 on host 'smithi105' 2024-03-26T18:53:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.091+0000 7fe574ce1700 1 -- 172.21.15.105:0/4070894347 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe55c0a2fb0 msgr2=0x7fe55c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.091+0000 7fe574ce1700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe55c0a2fb0 0x7fe55c0a5470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe560006fd0 tx=0x7fe560008040 comp rx=0 tx=0).stop 2024-03-26T18:53:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.091+0000 7fe574ce1700 1 -- 172.21.15.105:0/4070894347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe570102e70 msgr2=0x7fe57019ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.091+0000 7fe574ce1700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe570102e70 0x7fe57019ea10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe5680062a0 tx=0x7fe568007e40 comp rx=0 tx=0).stop 2024-03-26T18:53:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.091+0000 7fe574ce1700 1 -- 172.21.15.105:0/4070894347 shutdown_connections 2024-03-26T18:53:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.091+0000 7fe574ce1700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe570102e70 0x7fe57019ea10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:31.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.091+0000 7fe574ce1700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe55c0a2fb0 0x7fe55c0a5470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:31.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.091+0000 7fe574ce1700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5701012d0 0x7fe570197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:31.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.091+0000 7fe574ce1700 1 --2- 172.21.15.105:0/4070894347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5701024d0 0x7fe570197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:31.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.091+0000 7fe574ce1700 1 -- 172.21.15.105:0/4070894347 >> 172.21.15.105:0/4070894347 conn(0x7fe5700fc870 msgr2=0x7fe570105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:53:31.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.092+0000 7fe574ce1700 1 -- 172.21.15.105:0/4070894347 shutdown_connections 2024-03-26T18:53:31.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:53:31.092+0000 7fe574ce1700 1 -- 172.21.15.105:0/4070894347 wait complete. 2024-03-26T18:53:31.455 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.455 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.456 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:31.456 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:31.456 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.456 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:31 smithi105 ceph-mon[36270]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:31.456 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.456 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.456 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:31.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:31.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:31.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:31 smithi097 ceph-mon[31955]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:31.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:31 smithi144 ceph-mon[36053]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:31.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:31.757 DEBUG:teuthology.orchestra.run.smithi105:osd.7> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.7.service 2024-03-26T18:53:31.759 INFO:tasks.cephadm:Deploying osd.8 on smithi144 with /dev/vg_nvme/lv_2... 2024-03-26T18:53:31.759 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T18:53:32.006 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:53:32.259 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:32 smithi105 ceph-mon[36270]: from='osd.7 [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T18:53:32.259 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:32 smithi105 ceph-mon[36270]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T18:53:32.259 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:32 smithi105 ceph-mon[36270]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T18:53:32.260 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:32 smithi105 ceph-mon[36270]: from='osd.7 [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:53:32.260 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:32 smithi105 ceph-mon[36270]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T18:53:32.260 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:32 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:32.260 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:32 smithi105 ceph-mon[36270]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:53:32.487 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:32 smithi144 ceph-mon[36053]: from='osd.7 [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T18:53:32.487 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:32 smithi144 ceph-mon[36053]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T18:53:32.488 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:32 smithi144 ceph-mon[36053]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T18:53:32.488 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:32 smithi144 ceph-mon[36053]: from='osd.7 [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:53:32.488 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:32 smithi144 ceph-mon[36053]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T18:53:32.488 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:32 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:32.488 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:32 smithi144 ceph-mon[36053]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:53:32.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:32 smithi097 ceph-mon[31955]: from='osd.7 [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T18:53:32.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:32 smithi097 ceph-mon[31955]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T18:53:32.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:32 smithi097 ceph-mon[31955]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T18:53:32.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:32 smithi097 ceph-mon[31955]: from='osd.7 [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:53:32.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:32 smithi097 ceph-mon[31955]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T18:53:32.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:32 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:32.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:32 smithi097 ceph-mon[31955]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:53:33.368 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 18:53:33 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T18:53:33.062+0000 7f383e596700 -1 osd.7 0 waiting for initial osdmap 2024-03-26T18:53:33.368 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 18:53:33 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T18:53:33.071+0000 7f383a389700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:53:33.963 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T18:53:33.963 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T18:53:33.963 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-26T18:53:33.963 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-26T18:53:33.963 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130021 s, 806 MB/s 2024-03-26T18:53:33.963 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-26T18:53:34.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:34 smithi144 ceph-mon[36053]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:34.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:34 smithi144 ceph-mon[36053]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:53:34.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:34 smithi144 ceph-mon[36053]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T18:53:34.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:34.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:34 smithi097 ceph-mon[31955]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:34.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:34 smithi097 ceph-mon[31955]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:53:34.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:34 smithi097 ceph-mon[31955]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T18:53:34.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:34.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:34 smithi105 ceph-mon[36270]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-26T18:53:34.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:34 smithi105 ceph-mon[36270]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:53:34.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:34 smithi105 ceph-mon[36270]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T18:53:34.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:34.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-03-26T18:53:34.928 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:53:35.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:53:35.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:53:35.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: osd.7 [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] boot 2024-03-26T18:53:35.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: osdmap e51: 8 total, 8 up, 8 in 2024-03-26T18:53:35.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:35.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T18:53:35.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:35.304 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:35.305 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:35.305 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.305 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:35.305 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:35.305 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:53:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:53:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: osd.7 [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] boot 2024-03-26T18:53:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: osdmap e51: 8 total, 8 up, 8 in 2024-03-26T18:53:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T18:53:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:35.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:35.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:35.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:35.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:35.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:53:35.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:53:35.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: osd.7 [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] boot 2024-03-26T18:53:35.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: osdmap e51: 8 total, 8 up, 8 in 2024-03-26T18:53:35.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:53:35.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T18:53:35.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:35.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:35.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:53:35.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:35.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:35.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:35.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:36.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:36 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi105 2024-03-26T18:53:36.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:36 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi105 to 5693M 2024-03-26T18:53:36.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:36 smithi097 ceph-mon[31955]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:36.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:36 smithi097 ceph-mon[31955]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T18:53:36.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:36 smithi097 ceph-mon[31955]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-26T18:53:36.373 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:36 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi105 2024-03-26T18:53:36.373 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:36 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi105 to 5693M 2024-03-26T18:53:36.373 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:36 smithi144 ceph-mon[36053]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:36.373 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:36 smithi144 ceph-mon[36053]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T18:53:36.373 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:36 smithi144 ceph-mon[36053]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-26T18:53:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:36 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi105 2024-03-26T18:53:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:36 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi105 to 5693M 2024-03-26T18:53:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:36 smithi105 ceph-mon[36270]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:36 smithi105 ceph-mon[36270]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T18:53:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:36 smithi105 ceph-mon[36270]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-26T18:53:36.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 -- 172.21.15.144:0/2758592695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc8103600 msgr2=0x7f2fc8103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:36.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 --2- 172.21.15.144:0/2758592695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc8103600 0x7f2fc8103a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009990 tx=0x7f2fb4009ca0 comp rx=0 tx=0).stop 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 -- 172.21.15.144:0/2758592695 shutdown_connections 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 --2- 172.21.15.144:0/2758592695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc8103fa0 0x7f2fc810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 --2- 172.21.15.144:0/2758592695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8102400 0x7f2fc8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 --2- 172.21.15.144:0/2758592695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc8103600 0x7f2fc8103a60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 -- 172.21.15.144:0/2758592695 >> 172.21.15.144:0/2758592695 conn(0x7f2fc80fd9c0 msgr2=0x7f2fc80ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 -- 172.21.15.144:0/2758592695 shutdown_connections 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 -- 172.21.15.144:0/2758592695 wait complete. 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 Processor -- start 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 -- start start 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8102400 0x7f2fc8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc8103600 0x7f2fc8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:53:36.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc8103fa0 0x7f2fc819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc810e060 con 0x7f2fc8103600 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc810dee0 con 0x7f2fc8102400 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.595+0000 7f2fcf089700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc810e1e0 con 0x7f2fc8103fa0 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcd626700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc8103fa0 0x7f2fc819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcd626700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc8103fa0 0x7f2fc819fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41818/0 (socket says 172.21.15.144:41818) 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcd626700 1 -- 172.21.15.144:0/3294821357 learned_addr learned my addr 172.21.15.144:0/3294821357 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcce25700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8102400 0x7f2fc8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcd626700 1 -- 172.21.15.144:0/3294821357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8102400 msgr2=0x7f2fc8198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcd626700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8102400 0x7f2fc8198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcd626700 1 -- 172.21.15.144:0/3294821357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc8103600 msgr2=0x7f2fc8198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcd626700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc8103600 0x7f2fc8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcd626700 1 -- 172.21.15.144:0/3294821357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc81a02b0 con 0x7f2fc8103fa0 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcd626700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc8103fa0 0x7f2fc819fc10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000b9a0 tx=0x7f2fc000bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:53:36.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fc5ffb700 1 -- 172.21.15.144:0/3294821357 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0019070 con 0x7f2fc8103fa0 2024-03-26T18:53:36.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fc5ffb700 1 -- 172.21.15.144:0/3294821357 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fc000ce10 con 0x7f2fc8103fa0 2024-03-26T18:53:36.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcf089700 1 -- 172.21.15.144:0/3294821357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc81a05a0 con 0x7f2fc8103fa0 2024-03-26T18:53:36.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fc5ffb700 1 -- 172.21.15.144:0/3294821357 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00073d0 con 0x7f2fc8103fa0 2024-03-26T18:53:36.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.596+0000 7f2fcf089700 1 -- 172.21.15.144:0/3294821357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc81a0b60 con 0x7f2fc8103fa0 2024-03-26T18:53:36.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.597+0000 7f2fcf089700 1 -- 172.21.15.144:0/3294821357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc8069070 con 0x7f2fc8103fa0 2024-03-26T18:53:36.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.598+0000 7f2fc5ffb700 1 -- 172.21.15.144:0/3294821357 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f2fc001c070 con 0x7f2fc8103fa0 2024-03-26T18:53:36.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.599+0000 7f2fc5ffb700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2fb80a2fb0 0x7f2fb80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:53:36.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.599+0000 7f2fc5ffb700 1 -- 172.21.15.144:0/3294821357 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f2fc00cdbf0 con 0x7f2fc8103fa0 2024-03-26T18:53:36.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.599+0000 7f2fcce25700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2fb80a2fb0 0x7f2fb80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:53:36.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.600+0000 7f2fcce25700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2fb80a2fb0 0x7f2fb80a5470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8103460 tx=0x7f2fbc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:53:36.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.603+0000 7f2fc5ffb700 1 -- 172.21.15.144:0/3294821357 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2fc009b560 con 0x7f2fc8103fa0 2024-03-26T18:53:36.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:36.810+0000 7f2fcf089700 1 -- 172.21.15.144:0/3294821357 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2fc80627f0 con 0x7f2fb80a2fb0 2024-03-26T18:53:37.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:53:37.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:53:37.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:37.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:53:37.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:53:37.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:37.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:53:37.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:53:37.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:38.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:38 smithi097 ceph-mon[31955]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:53:38.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:38 smithi097 ceph-mon[31955]: pgmap v197: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:38 smithi097 ceph-mon[31955]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T18:53:38.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:38 smithi144 ceph-mon[36053]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:53:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:38 smithi144 ceph-mon[36053]: pgmap v197: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:38 smithi144 ceph-mon[36053]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T18:53:38.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:38 smithi105 ceph-mon[36270]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:53:38.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:38 smithi105 ceph-mon[36270]: pgmap v197: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:38.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:38 smithi105 ceph-mon[36270]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T18:53:39.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:39 smithi097 ceph-mon[31955]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T18:53:39.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:39 smithi144 ceph-mon[36053]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T18:53:39.459 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:39 smithi105 ceph-mon[36270]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T18:53:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:40 smithi097 ceph-mon[31955]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:40 smithi097 ceph-mon[31955]: from='client.? 172.21.15.144:0/243025547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c94dd159-fde0-4285-9b5d-e837afcb8f7f"}]: dispatch 2024-03-26T18:53:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:40 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c94dd159-fde0-4285-9b5d-e837afcb8f7f"}]: dispatch 2024-03-26T18:53:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:40 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c94dd159-fde0-4285-9b5d-e837afcb8f7f"}]': finished 2024-03-26T18:53:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:40 smithi097 ceph-mon[31955]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T18:53:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:40 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:40 smithi144 ceph-mon[36053]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:40 smithi144 ceph-mon[36053]: from='client.? 172.21.15.144:0/243025547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c94dd159-fde0-4285-9b5d-e837afcb8f7f"}]: dispatch 2024-03-26T18:53:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:40 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c94dd159-fde0-4285-9b5d-e837afcb8f7f"}]: dispatch 2024-03-26T18:53:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:40 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c94dd159-fde0-4285-9b5d-e837afcb8f7f"}]': finished 2024-03-26T18:53:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:40 smithi144 ceph-mon[36053]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T18:53:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:40 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:40 smithi105 ceph-mon[36270]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:40 smithi105 ceph-mon[36270]: from='client.? 172.21.15.144:0/243025547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c94dd159-fde0-4285-9b5d-e837afcb8f7f"}]: dispatch 2024-03-26T18:53:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:40 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c94dd159-fde0-4285-9b5d-e837afcb8f7f"}]: dispatch 2024-03-26T18:53:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:40 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c94dd159-fde0-4285-9b5d-e837afcb8f7f"}]': finished 2024-03-26T18:53:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:40 smithi105 ceph-mon[36270]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T18:53:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:40 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:41.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:41 smithi144 ceph-mon[36053]: from='client.? 172.21.15.144:0/1040308378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:53:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:41 smithi105 ceph-mon[36270]: from='client.? 172.21.15.144:0/1040308378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:53:41.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:41 smithi097 ceph-mon[31955]: from='client.? 172.21.15.144:0/1040308378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:53:42.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:42 smithi097 ceph-mon[31955]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:42 smithi097 ceph-mon[31955]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-26T18:53:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:42 smithi097 ceph-mon[31955]: Cluster is now healthy 2024-03-26T18:53:42.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:42 smithi144 ceph-mon[36053]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:42.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:42 smithi144 ceph-mon[36053]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-26T18:53:42.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:42 smithi144 ceph-mon[36053]: Cluster is now healthy 2024-03-26T18:53:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:42 smithi105 ceph-mon[36270]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:42 smithi105 ceph-mon[36270]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-26T18:53:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:42 smithi105 ceph-mon[36270]: Cluster is now healthy 2024-03-26T18:53:44.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:44 smithi144 ceph-mon[36053]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:44.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:44 smithi105 ceph-mon[36270]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:44.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:44 smithi097 ceph-mon[31955]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:46.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:46 smithi144 ceph-mon[36053]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:46 smithi105 ceph-mon[36270]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:46.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:46 smithi097 ceph-mon[31955]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:47.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T18:53:47.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:47 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:47.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T18:53:47.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:47 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:47.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T18:53:47.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:47 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:48 smithi105 ceph-mon[36270]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:48 smithi105 ceph-mon[36270]: Deploying daemon osd.8 on smithi144 2024-03-26T18:53:48.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:48 smithi144 ceph-mon[36053]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:48.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:48 smithi144 ceph-mon[36053]: Deploying daemon osd.8 on smithi144 2024-03-26T18:53:48.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:48 smithi097 ceph-mon[31955]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:48.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:48 smithi097 ceph-mon[31955]: Deploying daemon osd.8 on smithi144 2024-03-26T18:53:50.276 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:50 smithi144 ceph-mon[36053]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:50.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:50 smithi105 ceph-mon[36270]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:50.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:50 smithi097 ceph-mon[31955]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:52.289 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:52 smithi144 ceph-mon[36053]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:52.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:52 smithi105 ceph-mon[36270]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:52.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:52 smithi097 ceph-mon[31955]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:53.852 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:53.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:53.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:53.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:53 smithi097 ceph-mon[31955]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:53.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:53.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:53.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:53.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:53 smithi144 ceph-mon[36053]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:53.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:53.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:53.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:53.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:53 smithi105 ceph-mon[36270]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:56.123 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:55 smithi144 ceph-mon[36053]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:56.123 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:56.123 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:56.123 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:56.123 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:56.124 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:56.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:55 smithi105 ceph-mon[36270]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:56.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:56.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:56.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:56.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:56.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:56.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:55 smithi097 ceph-mon[31955]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:56.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:56.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:56.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:53:56.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:53:56.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:56.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.505+0000 7f2fc5ffb700 1 -- 172.21.15.144:0/3294821357 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2fc80627f0 con 0x7f2fb80a2fb0 2024-03-26T18:53:56.498 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 8 on host 'smithi144' 2024-03-26T18:53:56.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 -- 172.21.15.144:0/3294821357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2fb80a2fb0 msgr2=0x7f2fb80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2fb80a2fb0 0x7f2fb80a5470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8103460 tx=0x7f2fbc00b410 comp rx=0 tx=0).stop 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 -- 172.21.15.144:0/3294821357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc8103fa0 msgr2=0x7f2fc819fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc8103fa0 0x7f2fc819fc10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000b9a0 tx=0x7f2fc000bcb0 comp rx=0 tx=0).stop 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 -- 172.21.15.144:0/3294821357 shutdown_connections 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2fb80a2fb0 0x7f2fb80a5470 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc8103fa0 0x7f2fc819fc10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8102400 0x7f2fc8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 --2- 172.21.15.144:0/3294821357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc8103600 0x7f2fc8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 -- 172.21.15.144:0/3294821357 >> 172.21.15.144:0/3294821357 conn(0x7f2fc80fd9c0 msgr2=0x7f2fc8106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 -- 172.21.15.144:0/3294821357 shutdown_connections 2024-03-26T18:53:56.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:53:56.508+0000 7f2fcf089700 1 -- 172.21.15.144:0/3294821357 wait complete. 2024-03-26T18:53:57.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:56 smithi144 ceph-mon[36053]: from='osd.8 [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T18:53:57.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:56 smithi144 ceph-mon[36053]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T18:53:57.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:56 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:57.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:56 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:57.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:56 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:57.171 DEBUG:teuthology.orchestra.run.smithi144:osd.8> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.8.service 2024-03-26T18:53:57.174 INFO:tasks.cephadm:Deploying osd.9 on smithi097 with /dev/vg_nvme/lv_1... 2024-03-26T18:53:57.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T18:53:57.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:56 smithi105 ceph-mon[36270]: from='osd.8 [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T18:53:57.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:56 smithi105 ceph-mon[36270]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T18:53:57.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:56 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:57.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:56 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:57.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:56 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:57.262 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:56 smithi097 ceph-mon[31955]: from='osd.8 [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T18:53:57.263 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:56 smithi097 ceph-mon[31955]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T18:53:57.263 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:56 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:57.263 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:56 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:53:57.263 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:56 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:53:57.614 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:53:58.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:57 smithi097 ceph-mon[31955]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:58.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:57 smithi097 ceph-mon[31955]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T18:53:58.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:57 smithi097 ceph-mon[31955]: from='osd.8 [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:53:58.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:57 smithi097 ceph-mon[31955]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T18:53:58.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:57 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:58.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:57 smithi097 ceph-mon[31955]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:53:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:57 smithi105 ceph-mon[36270]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:57 smithi105 ceph-mon[36270]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T18:53:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:57 smithi105 ceph-mon[36270]: from='osd.8 [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:53:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:57 smithi105 ceph-mon[36270]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T18:53:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:57 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:57 smithi105 ceph-mon[36270]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:53:58.322 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:57 smithi144 ceph-mon[36053]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:53:58.322 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:57 smithi144 ceph-mon[36053]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T18:53:58.322 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:57 smithi144 ceph-mon[36053]: from='osd.8 [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:53:58.322 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:57 smithi144 ceph-mon[36053]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T18:53:58.322 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:57 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:58.322 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:57 smithi144 ceph-mon[36053]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:53:58.608 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:53:58 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T18:53:58.348+0000 7fc25f425700 -1 osd.8 0 waiting for initial osdmap 2024-03-26T18:53:58.608 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:53:58 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T18:53:58.361+0000 7fc257210700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:53:59.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:58 smithi097 ceph-mon[31955]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:53:59.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:58 smithi097 ceph-mon[31955]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T18:53:59.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:59.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:59.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:58 smithi097 ceph-mon[31955]: from='osd.8 ' entity='osd.8' 2024-03-26T18:53:59.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:58 smithi097 ceph-mon[31955]: osd.8 [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] boot 2024-03-26T18:53:59.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:58 smithi097 ceph-mon[31955]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T18:53:59.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:59.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:58 smithi105 ceph-mon[36270]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:53:59.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:58 smithi105 ceph-mon[36270]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T18:53:59.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:59.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:59.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:58 smithi105 ceph-mon[36270]: from='osd.8 ' entity='osd.8' 2024-03-26T18:53:59.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:58 smithi105 ceph-mon[36270]: osd.8 [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] boot 2024-03-26T18:53:59.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:58 smithi105 ceph-mon[36270]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T18:53:59.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:59.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:58 smithi144 ceph-mon[36053]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:53:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:58 smithi144 ceph-mon[36053]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T18:53:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:53:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:58 smithi144 ceph-mon[36053]: from='osd.8 ' entity='osd.8' 2024-03-26T18:53:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:58 smithi144 ceph-mon[36053]: osd.8 [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] boot 2024-03-26T18:53:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:58 smithi144 ceph-mon[36053]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T18:53:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:54:00.039 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:59 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:54:00.039 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:59 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:54:00.039 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:59 smithi144 ceph-mon[36053]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:54:00.039 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:53:59 smithi144 ceph-mon[36053]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T18:54:00.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:59 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:54:00.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:59 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:54:00.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:59 smithi097 ceph-mon[31955]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:54:00.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:53:59 smithi097 ceph-mon[31955]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T18:54:00.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:59 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:54:00.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:59 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:54:00.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:59 smithi105 ceph-mon[36270]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T18:54:00.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:53:59 smithi105 ceph-mon[36270]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T18:54:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T18:54:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T18:54:00.693 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-26T18:54:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-26T18:54:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0322743 s, 325 MB/s 2024-03-26T18:54:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-26T18:54:01.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi144 2024-03-26T18:54:01.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:01.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:01.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:01.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi144 to 5693M 2024-03-26T18:54:01.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:01.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:01.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:01 smithi097 ceph-mon[31955]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T18:54:01.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi144 2024-03-26T18:54:01.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:01.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:01.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:01.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi144 to 5693M 2024-03-26T18:54:01.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:01.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:01.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:01 smithi144 ceph-mon[36053]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T18:54:01.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi144 2024-03-26T18:54:01.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:01.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:01.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:01.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi144 to 5693M 2024-03-26T18:54:01.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:01.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:01.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:01.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:01 smithi105 ceph-mon[36270]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T18:54:01.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-03-26T18:54:02.048 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:54:02.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:02 smithi097 ceph-mon[31955]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:02.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:02 smithi144 ceph-mon[36053]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:02.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:02 smithi105 ceph-mon[36270]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:03.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:03 smithi097 ceph-mon[31955]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T18:54:03.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:03 smithi144 ceph-mon[36053]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T18:54:03.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:03 smithi105 ceph-mon[36270]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T18:54:04.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.306+0000 7fef7ae00700 1 -- 172.21.15.97:0/2722215623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef74102000 msgr2=0x7fef74102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:54:04.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.306+0000 7fef7ae00700 1 --2- 172.21.15.97:0/2722215623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef74102000 0x7fef74102480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00b0a0 tx=0x7fef6c00b3b0 comp rx=0 tx=0).stop 2024-03-26T18:54:04.300 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:04 smithi097 ceph-mon[31955]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:04.300 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:04 smithi097 ceph-mon[31955]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T18:54:04.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.306+0000 7fef7ae00700 1 -- 172.21.15.97:0/2722215623 shutdown_connections 2024-03-26T18:54:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.306+0000 7fef7ae00700 1 --2- 172.21.15.97:0/2722215623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef74102000 0x7fef74102480 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.306+0000 7fef7ae00700 1 --2- 172.21.15.97:0/2722215623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef74100ce0 0x7fef74101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.306+0000 7fef7ae00700 1 --2- 172.21.15.97:0/2722215623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef74101640 0x7fef74101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.306+0000 7fef7ae00700 1 -- 172.21.15.97:0/2722215623 >> 172.21.15.97:0/2722215623 conn(0x7fef740fc880 msgr2=0x7fef740fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:54:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.306+0000 7fef7ae00700 1 -- 172.21.15.97:0/2722215623 shutdown_connections 2024-03-26T18:54:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.306+0000 7fef7ae00700 1 -- 172.21.15.97:0/2722215623 wait complete. 2024-03-26T18:54:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef7ae00700 1 Processor -- start 2024-03-26T18:54:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef7ae00700 1 -- start start 2024-03-26T18:54:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef7ae00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef74100ce0 0x7fef7419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:54:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef7ae00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef74101640 0x7fef7419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:54:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef7ae00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef74102000 0x7fef741a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:54:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef7ae00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef741101b0 con 0x7fef74102000 2024-03-26T18:54:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef7ae00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fef74110030 con 0x7fef74100ce0 2024-03-26T18:54:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef7ae00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef74110330 con 0x7fef74101640 2024-03-26T18:54:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef73e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef74100ce0 0x7fef7419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:54:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef73e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef74100ce0 0x7fef7419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34314/0 (socket says 172.21.15.97:34314) 2024-03-26T18:54:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef73e00700 1 -- 172.21.15.97:0/789492667 learned_addr learned my addr 172.21.15.97:0/789492667 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:54:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.308+0000 7fef78e00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef74102000 0x7fef741a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:54:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.309+0000 7fef73400700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef74101640 0x7fef7419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:54:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.309+0000 7fef78e00700 1 -- 172.21.15.97:0/789492667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef74101640 msgr2=0x7fef7419ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.309+0000 7fef78e00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef74101640 0x7fef7419ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.309+0000 7fef78e00700 1 -- 172.21.15.97:0/789492667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef74100ce0 msgr2=0x7fef7419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.309+0000 7fef78e00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef74100ce0 0x7fef7419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.309+0000 7fef78e00700 1 -- 172.21.15.97:0/789492667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef741a2400 con 0x7fef74102000 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.309+0000 7fef73e00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef74100ce0 0x7fef7419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.309+0000 7fef78e00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef74102000 0x7fef741a1d60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00b070 tx=0x7fef6c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.310+0000 7fef70c00700 1 -- 172.21.15.97:0/789492667 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c021070 con 0x7fef74102000 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.310+0000 7fef70c00700 1 -- 172.21.15.97:0/789492667 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef6c007b80 con 0x7fef74102000 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.310+0000 7fef70c00700 1 -- 172.21.15.97:0/789492667 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c004750 con 0x7fef74102000 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.310+0000 7fef7ae00700 1 -- 172.21.15.97:0/789492667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef741a2690 con 0x7fef74102000 2024-03-26T18:54:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.310+0000 7fef7ae00700 1 -- 172.21.15.97:0/789492667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef741a2af0 con 0x7fef74102000 2024-03-26T18:54:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.311+0000 7fef7ae00700 1 -- 172.21.15.97:0/789492667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef74069070 con 0x7fef74102000 2024-03-26T18:54:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.313+0000 7fef70c00700 1 -- 172.21.15.97:0/789492667 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fef6c003a00 con 0x7fef74102000 2024-03-26T18:54:04.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.315+0000 7fef70c00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fef580a3030 0x7fef580a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:54:04.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.315+0000 7fef70c00700 1 -- 172.21.15.97:0/789492667 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fef6c00f020 con 0x7fef74102000 2024-03-26T18:54:04.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.315+0000 7fef73e00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fef580a3030 0x7fef580a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:54:04.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.316+0000 7fef73e00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fef580a3030 0x7fef580a54f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef64011590 tx=0x7fef64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:54:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.319+0000 7fef70c00700 1 -- 172.21.15.97:0/789492667 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fef6c02a020 con 0x7fef74102000 2024-03-26T18:54:04.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:04 smithi144 ceph-mon[36053]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:04.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:04 smithi144 ceph-mon[36053]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T18:54:04.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:04 smithi105 ceph-mon[36270]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:04.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:04 smithi105 ceph-mon[36270]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T18:54:04.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:04.554+0000 7fef7ae00700 1 -- 172.21.15.97:0/789492667 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fef740627f0 con 0x7fef580a3030 2024-03-26T18:54:05.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:54:05.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:54:05.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:05.427 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:54:05.427 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:54:05.427 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:05.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:54:05.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:54:05.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:06.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:06 smithi144 ceph-mon[36053]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:54:06.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:06 smithi144 ceph-mon[36053]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-26T18:54:06.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:06 smithi105 ceph-mon[36270]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:54:06.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:06 smithi105 ceph-mon[36270]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-26T18:54:06.586 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:06 smithi097 ceph-mon[31955]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:54:06.586 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:06 smithi097 ceph-mon[31955]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-26T18:54:08.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:08 smithi105 ceph-mon[36270]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-26T18:54:08.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:08 smithi097 ceph-mon[31955]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-26T18:54:08.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:08 smithi144 ceph-mon[36053]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-26T18:54:09.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:09 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2376578879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d179a4d5-7751-4d6b-a273-97807334dbe7"}]: dispatch 2024-03-26T18:54:09.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:09 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d179a4d5-7751-4d6b-a273-97807334dbe7"}]: dispatch 2024-03-26T18:54:09.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:09 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d179a4d5-7751-4d6b-a273-97807334dbe7"}]': finished 2024-03-26T18:54:09.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:09 smithi097 ceph-mon[31955]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T18:54:09.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:09.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:09 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2376578879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d179a4d5-7751-4d6b-a273-97807334dbe7"}]: dispatch 2024-03-26T18:54:09.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:09 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d179a4d5-7751-4d6b-a273-97807334dbe7"}]: dispatch 2024-03-26T18:54:09.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:09 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d179a4d5-7751-4d6b-a273-97807334dbe7"}]': finished 2024-03-26T18:54:09.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:09 smithi105 ceph-mon[36270]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T18:54:09.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:09.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:09 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2376578879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d179a4d5-7751-4d6b-a273-97807334dbe7"}]: dispatch 2024-03-26T18:54:09.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:09 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d179a4d5-7751-4d6b-a273-97807334dbe7"}]: dispatch 2024-03-26T18:54:09.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:09 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d179a4d5-7751-4d6b-a273-97807334dbe7"}]': finished 2024-03-26T18:54:09.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:09 smithi144 ceph-mon[36053]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T18:54:09.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:10.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:10 smithi105 ceph-mon[36270]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T18:54:10.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:10 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1080645310' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:54:10.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:10 smithi097 ceph-mon[31955]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T18:54:10.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:10 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1080645310' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:54:10.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:10 smithi144 ceph-mon[36053]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T18:54:10.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:10 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1080645310' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:54:12.428 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:12 smithi144 ceph-mon[36053]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:54:12.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:12 smithi105 ceph-mon[36270]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:54:12.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:12 smithi097 ceph-mon[31955]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:54:14.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:14 smithi105 ceph-mon[36270]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-26T18:54:14.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:14 smithi097 ceph-mon[31955]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-26T18:54:14.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:14 smithi144 ceph-mon[36053]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-26T18:54:16.277 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:16 smithi097 ceph-mon[31955]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:16.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:16 smithi105 ceph-mon[36270]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:16.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:16 smithi144 ceph-mon[36053]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:18.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:18 smithi105 ceph-mon[36270]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:18.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:18 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T18:54:18.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:18 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:18.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:18 smithi097 ceph-mon[31955]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:18.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:18 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T18:54:18.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:18 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:18.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:18 smithi144 ceph-mon[36053]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:18.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:18 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T18:54:18.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:18 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:19.326 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:19 smithi097 ceph-mon[31955]: Deploying daemon osd.9 on smithi097 2024-03-26T18:54:19.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:19 smithi105 ceph-mon[36270]: Deploying daemon osd.9 on smithi097 2024-03-26T18:54:19.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:19 smithi144 ceph-mon[36053]: Deploying daemon osd.9 on smithi097 2024-03-26T18:54:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:20 smithi105 ceph-mon[36270]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:20.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:20 smithi097 ceph-mon[31955]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:20.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:20 smithi144 ceph-mon[36053]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:21.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:21 smithi097 ceph-mon[31955]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:21.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:21 smithi105 ceph-mon[36270]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:21.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:21 smithi144 ceph-mon[36053]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:24.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:23 smithi144 ceph-mon[36053]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:24.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:23 smithi105 ceph-mon[36270]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:24.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:23 smithi097 ceph-mon[31955]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:26.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:25 smithi105 ceph-mon[36270]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:26.211 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:25 smithi097 ceph-mon[31955]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:26.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:25 smithi144 ceph-mon[36053]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:27.749 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:27.749 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:27.749 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:27.749 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:27 smithi097 ceph-mon[31955]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:27.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:27.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:27.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:27.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:27 smithi144 ceph-mon[36053]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:27.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:27 smithi105 ceph-mon[36270]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:29.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:29 smithi097 ceph-mon[31955]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:29 smithi105 ceph-mon[36270]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:29 smithi144 ceph-mon[36053]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:31.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:30 smithi105 ceph-mon[36270]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T18:54:31.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:31.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:31.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:30 smithi097 ceph-mon[31955]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T18:54:31.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:31.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:31.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:30 smithi144 ceph-mon[36053]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T18:54:31.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:31.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:31.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.546+0000 7fef70c00700 1 -- 172.21.15.97:0/789492667 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fef740627f0 con 0x7fef580a3030 2024-03-26T18:54:31.540 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 9 on host 'smithi097' 2024-03-26T18:54:31.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 -- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fef580a3030 msgr2=0x7fef580a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:54:31.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fef580a3030 0x7fef580a54f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef64011590 tx=0x7fef64009260 comp rx=0 tx=0).stop 2024-03-26T18:54:31.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 -- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef74102000 msgr2=0x7fef741a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:54:31.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef74102000 0x7fef741a1d60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00b070 tx=0x7fef6c004ce0 comp rx=0 tx=0).stop 2024-03-26T18:54:31.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 -- 172.21.15.97:0/789492667 shutdown_connections 2024-03-26T18:54:31.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef74102000 0x7fef741a1d60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:31.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef74100ce0 0x7fef7419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:31.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fef580a3030 0x7fef580a54f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:31.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 --2- 172.21.15.97:0/789492667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef74101640 0x7fef7419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:31.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 -- 172.21.15.97:0/789492667 >> 172.21.15.97:0/789492667 conn(0x7fef740fc880 msgr2=0x7fef740fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:54:31.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.549+0000 7fef7ae00700 1 -- 172.21.15.97:0/789492667 shutdown_connections 2024-03-26T18:54:31.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:54:31.550+0000 7fef7ae00700 1 -- 172.21.15.97:0/789492667 wait complete. 2024-03-26T18:54:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:31 smithi105 ceph-mon[36270]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:31 smithi105 ceph-mon[36270]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T18:54:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:31 smithi105 ceph-mon[36270]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T18:54:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:31 smithi105 ceph-mon[36270]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:54:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:31 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:32.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:31 smithi097 ceph-mon[31955]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:32.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:31 smithi097 ceph-mon[31955]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T18:54:32.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:31 smithi097 ceph-mon[31955]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T18:54:32.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:31 smithi097 ceph-mon[31955]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:54:32.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:32.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:32.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:31 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:32.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:31 smithi144 ceph-mon[36053]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:31 smithi144 ceph-mon[36053]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T18:54:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:31 smithi144 ceph-mon[36053]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T18:54:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:31 smithi144 ceph-mon[36053]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-26T18:54:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:31 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:32.425 DEBUG:teuthology.orchestra.run.smithi097:osd.9> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.9.service 2024-03-26T18:54:32.430 INFO:tasks.cephadm:Deploying osd.10 on smithi105 with /dev/vg_nvme/lv_1... 2024-03-26T18:54:32.430 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T18:54:32.684 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:54:32.852 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 18:54:32 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T18:54:32.524+0000 7f05e1e00700 -1 osd.9 0 waiting for initial osdmap 2024-03-26T18:54:32.853 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 18:54:32 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T18:54:32.560+0000 7f05d7c00700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:54:33.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:32 smithi105 ceph-mon[36270]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:54:33.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:32 smithi105 ceph-mon[36270]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T18:54:33.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:32 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:33.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:32 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:33.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:32 smithi105 ceph-mon[36270]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' 2024-03-26T18:54:33.297 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:32 smithi097 ceph-mon[31955]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:54:33.297 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:32 smithi097 ceph-mon[31955]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T18:54:33.297 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:32 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:33.298 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:32 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:33.298 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:32 smithi097 ceph-mon[31955]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' 2024-03-26T18:54:33.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:32 smithi144 ceph-mon[36053]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-26T18:54:33.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:32 smithi144 ceph-mon[36053]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T18:54:33.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:32 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:33.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:32 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:33.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:32 smithi144 ceph-mon[36053]: from='osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599]' entity='osd.9' 2024-03-26T18:54:34.070 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:33 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:54:34.071 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:33 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:54:34.071 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:33 smithi105 ceph-mon[36270]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:34.071 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:33 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:34.071 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:33 smithi105 ceph-mon[36270]: osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] boot 2024-03-26T18:54:34.071 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:33 smithi105 ceph-mon[36270]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T18:54:34.071 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:33 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:34.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:33 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:54:34.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:33 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:54:34.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:33 smithi097 ceph-mon[31955]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:34.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:33 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:34.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:33 smithi097 ceph-mon[31955]: osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] boot 2024-03-26T18:54:34.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:33 smithi097 ceph-mon[31955]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T18:54:34.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:33 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:34.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:33 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:54:34.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:33 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:54:34.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:33 smithi144 ceph-mon[36053]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T18:54:34.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:33 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:34.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:33 smithi144 ceph-mon[36053]: osd.9 [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] boot 2024-03-26T18:54:34.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:33 smithi144 ceph-mon[36053]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T18:54:34.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:33 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:54:34.729 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T18:54:34.729 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T18:54:34.729 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-26T18:54:34.729 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-26T18:54:34.729 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148439 s, 706 MB/s 2024-03-26T18:54:34.729 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-26T18:54:35.431 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_1 2024-03-26T18:54:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:54:36.047 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:35 smithi105 ceph-mon[36270]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:36.048 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:35 smithi105 ceph-mon[36270]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T18:54:36.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:35 smithi097 ceph-mon[31955]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:36.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:35 smithi097 ceph-mon[31955]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T18:54:36.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:35 smithi144 ceph-mon[36053]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:36.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:35 smithi144 ceph-mon[36053]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T18:54:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.357+0000 7f3d5c53c700 1 -- 172.21.15.105:0/2237950899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d541035e0 msgr2=0x7f3d54103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:54:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.357+0000 7f3d5c53c700 1 --2- 172.21.15.105:0/2237950899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d541035e0 0x7f3d54103a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3d440099c0 tx=0x7f3d44009cd0 comp rx=0 tx=0).stop 2024-03-26T18:54:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 -- 172.21.15.105:0/2237950899 shutdown_connections 2024-03-26T18:54:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 --2- 172.21.15.105:0/2237950899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d54103f80 0x7f3d5410a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 --2- 172.21.15.105:0/2237950899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d541023e0 0x7f3d54102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 --2- 172.21.15.105:0/2237950899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d541035e0 0x7f3d54103a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 -- 172.21.15.105:0/2237950899 >> 172.21.15.105:0/2237950899 conn(0x7f3d540fd9c0 msgr2=0x7f3d540ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:54:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 -- 172.21.15.105:0/2237950899 shutdown_connections 2024-03-26T18:54:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 -- 172.21.15.105:0/2237950899 wait complete. 2024-03-26T18:54:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 Processor -- start 2024-03-26T18:54:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 -- start start 2024-03-26T18:54:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d541023e0 0x7f3d541986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:54:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d541035e0 0x7f3d54198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:54:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d54103f80 0x7f3d5419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:54:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5410e0e0 con 0x7f3d541035e0 2024-03-26T18:54:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5410df60 con 0x7f3d54103f80 2024-03-26T18:54:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5c53c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5410e260 con 0x7f3d541023e0 2024-03-26T18:54:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5aad9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d54103f80 0x7f3d5419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:54:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5aad9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d54103f80 0x7f3d5419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59110/0 (socket says 172.21.15.105:59110) 2024-03-26T18:54:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.358+0000 7f3d5aad9700 1 -- 172.21.15.105:0/3927196329 learned_addr learned my addr 172.21.15.105:0/3927196329 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d5a2d8700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d541023e0 0x7f3d541986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d5aad9700 1 -- 172.21.15.105:0/3927196329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d541023e0 msgr2=0x7f3d541986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d5aad9700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d541023e0 0x7f3d541986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d5aad9700 1 -- 172.21.15.105:0/3927196329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d541035e0 msgr2=0x7f3d54198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d5aad9700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d541035e0 0x7f3d54198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d5aad9700 1 -- 172.21.15.105:0/3927196329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d541a0330 con 0x7f3d54103f80 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d5aad9700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d54103f80 0x7f3d5419fc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00b780 tx=0x7f3d4c00bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d4b7fe700 1 -- 172.21.15.105:0/3927196329 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c00f840 con 0x7f3d54103f80 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d4b7fe700 1 -- 172.21.15.105:0/3927196329 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d4c017400 con 0x7f3d54103f80 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d4b7fe700 1 -- 172.21.15.105:0/3927196329 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c00d630 con 0x7f3d54103f80 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d5c53c700 1 -- 172.21.15.105:0/3927196329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d541a0620 con 0x7f3d54103f80 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.359+0000 7f3d5c53c700 1 -- 172.21.15.105:0/3927196329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d541a0b90 con 0x7f3d54103f80 2024-03-26T18:54:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.360+0000 7f3d5c53c700 1 -- 172.21.15.105:0/3927196329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d54069070 con 0x7f3d54103f80 2024-03-26T18:54:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.361+0000 7f3d4b7fe700 1 -- 172.21.15.105:0/3927196329 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f3d4c017940 con 0x7f3d54103f80 2024-03-26T18:54:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.362+0000 7f3d4b7fe700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d400a2fb0 0x7f3d400a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:54:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.362+0000 7f3d4b7fe700 1 -- 172.21.15.105:0/3927196329 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f3d4c0cea20 con 0x7f3d54103f80 2024-03-26T18:54:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.362+0000 7f3d5a2d8700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d400a2fb0 0x7f3d400a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:54:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.363+0000 7f3d5a2d8700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d400a2fb0 0x7f3d400a5470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006fd0 tx=0x7f3d50008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:54:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.366+0000 7f3d4b7fe700 1 -- 172.21.15.105:0/3927196329 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3d4c013030 con 0x7f3d54103f80 2024-03-26T18:54:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:37.569+0000 7f3d5c53c700 1 -- 172.21.15.105:0/3927196329 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3d540627f0 con 0x7f3d400a2fb0 2024-03-26T18:54:38.208 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:38.208 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:54:38.208 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:54:38.208 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:38.208 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.208 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:38.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:38.209 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:54:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:54:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:38.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:38.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:54:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:38.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:38.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:38.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:39.230 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:38 smithi105 ceph-mon[36270]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:54:39.230 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:38 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi097 2024-03-26T18:54:39.230 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:38 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi097 to 4272M 2024-03-26T18:54:39.230 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:39.230 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:39.230 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:39.230 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:38 smithi097 ceph-mon[31955]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:54:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:38 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi097 2024-03-26T18:54:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:38 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi097 to 4272M 2024-03-26T18:54:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:38 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:38 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:38 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:38 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:39.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:38 smithi144 ceph-mon[36053]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:54:39.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:38 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi097 2024-03-26T18:54:39.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:38 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi097 to 4272M 2024-03-26T18:54:39.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:38 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:39.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:38 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:39.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:38 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:39.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:38 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:40.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:40 smithi097 ceph-mon[31955]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:40.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:40 smithi144 ceph-mon[36053]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:40 smithi105 ceph-mon[36270]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:41.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:41 smithi105 ceph-mon[36270]: from='client.? 172.21.15.105:0/4263548235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e5fd1b9-0abc-48b2-991a-2e07f7606bfe"}]: dispatch 2024-03-26T18:54:41.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:41 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e5fd1b9-0abc-48b2-991a-2e07f7606bfe"}]: dispatch 2024-03-26T18:54:41.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:41 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e5fd1b9-0abc-48b2-991a-2e07f7606bfe"}]': finished 2024-03-26T18:54:41.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:41 smithi105 ceph-mon[36270]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T18:54:41.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:41 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:54:41.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:41 smithi097 ceph-mon[31955]: from='client.? 172.21.15.105:0/4263548235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e5fd1b9-0abc-48b2-991a-2e07f7606bfe"}]: dispatch 2024-03-26T18:54:41.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:41 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e5fd1b9-0abc-48b2-991a-2e07f7606bfe"}]: dispatch 2024-03-26T18:54:41.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:41 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e5fd1b9-0abc-48b2-991a-2e07f7606bfe"}]': finished 2024-03-26T18:54:41.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:41 smithi097 ceph-mon[31955]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T18:54:41.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:41 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:54:41.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:41 smithi144 ceph-mon[36053]: from='client.? 172.21.15.105:0/4263548235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e5fd1b9-0abc-48b2-991a-2e07f7606bfe"}]: dispatch 2024-03-26T18:54:41.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:41 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e5fd1b9-0abc-48b2-991a-2e07f7606bfe"}]: dispatch 2024-03-26T18:54:41.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:41 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e5fd1b9-0abc-48b2-991a-2e07f7606bfe"}]': finished 2024-03-26T18:54:41.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:41 smithi144 ceph-mon[36053]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T18:54:41.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:41 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:54:42.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:42 smithi097 ceph-mon[31955]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:42 smithi097 ceph-mon[31955]: from='client.? 172.21.15.105:0/2974588104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:54:42.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:42 smithi144 ceph-mon[36053]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:42 smithi144 ceph-mon[36053]: from='client.? 172.21.15.105:0/2974588104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:54:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:42 smithi105 ceph-mon[36270]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:42 smithi105 ceph-mon[36270]: from='client.? 172.21.15.105:0/2974588104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:54:44.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:44 smithi097 ceph-mon[31955]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:44.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:44 smithi144 ceph-mon[36053]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:44.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:44 smithi105 ceph-mon[36270]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:46.148 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:46 smithi105 ceph-mon[36270]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:46.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:46 smithi097 ceph-mon[31955]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:46.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:46 smithi144 ceph-mon[36053]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:48.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:48 smithi097 ceph-mon[31955]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:48.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:48 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T18:54:48.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:48 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:48.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:48 smithi144 ceph-mon[36053]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:48.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:48 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T18:54:48.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:48 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:48 smithi105 ceph-mon[36270]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:48 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T18:54:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:48 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:49.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:49 smithi105 ceph-mon[36270]: Deploying daemon osd.10 on smithi105 2024-03-26T18:54:49.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:49 smithi097 ceph-mon[31955]: Deploying daemon osd.10 on smithi105 2024-03-26T18:54:49.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:49 smithi144 ceph-mon[36053]: Deploying daemon osd.10 on smithi105 2024-03-26T18:54:50.335 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:50 smithi105 ceph-mon[36270]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:50.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:50 smithi097 ceph-mon[31955]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:50.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:50 smithi144 ceph-mon[36053]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:52.265 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:52 smithi105 ceph-mon[36270]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:52.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:52 smithi097 ceph-mon[31955]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:52.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:52 smithi144 ceph-mon[36053]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:54.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:54 smithi097 ceph-mon[31955]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:54.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:54 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:54.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:54 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:54.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:54 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:54.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:54 smithi144 ceph-mon[36053]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:54.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:54 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:54.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:54 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:54.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:54 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:54.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:54 smithi105 ceph-mon[36270]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:54.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:54 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:54.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:54 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:54.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:54 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:56.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:56 smithi097 ceph-mon[31955]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:56.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:56 smithi144 ceph-mon[36053]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:56 smithi105 ceph-mon[36270]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:58.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.012+0000 7f3d4b7fe700 1 -- 172.21.15.105:0/3927196329 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3d540627f0 con 0x7f3d400a2fb0 2024-03-26T18:54:58.005 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 10 on host 'smithi105' 2024-03-26T18:54:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 -- 172.21.15.105:0/3927196329 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d400a2fb0 msgr2=0x7f3d400a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:54:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d400a2fb0 0x7f3d400a5470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006fd0 tx=0x7f3d50008040 comp rx=0 tx=0).stop 2024-03-26T18:54:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 -- 172.21.15.105:0/3927196329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d54103f80 msgr2=0x7f3d5419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:54:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d54103f80 0x7f3d5419fc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00b780 tx=0x7f3d4c00bb40 comp rx=0 tx=0).stop 2024-03-26T18:54:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 -- 172.21.15.105:0/3927196329 shutdown_connections 2024-03-26T18:54:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d54103f80 0x7f3d5419fc90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d400a2fb0 0x7f3d400a5470 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d541023e0 0x7f3d541986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:58.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 --2- 172.21.15.105:0/3927196329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d541035e0 0x7f3d54198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:54:58.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 -- 172.21.15.105:0/3927196329 >> 172.21.15.105:0/3927196329 conn(0x7f3d540fd9c0 msgr2=0x7f3d54106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:54:58.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 -- 172.21.15.105:0/3927196329 shutdown_connections 2024-03-26T18:54:58.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:54:58.014+0000 7f3d5c53c700 1 -- 172.21.15.105:0/3927196329 wait complete. 2024-03-26T18:54:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:58.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: from='osd.10 [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T18:54:58.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T18:54:58.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:58.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:58.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:58.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:58.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: from='osd.10 [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T18:54:58.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T18:54:58.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:54:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:54:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:54:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: from='osd.10 [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T18:54:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T18:54:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:54:58.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:54:58.704 DEBUG:teuthology.orchestra.run.smithi105:osd.10> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.10.service 2024-03-26T18:54:58.707 INFO:tasks.cephadm:Deploying osd.11 on smithi144 with /dev/vg_nvme/lv_1... 2024-03-26T18:54:58.707 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T18:54:58.955 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:54:59.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:59 smithi105 ceph-mon[36270]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T18:54:59.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:59 smithi105 ceph-mon[36270]: from='osd.10 [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:54:59.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:59 smithi105 ceph-mon[36270]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T18:54:59.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:59 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:54:59.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:54:59 smithi105 ceph-mon[36270]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:54:59.211 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 18:54:59 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T18:54:59.144+0000 7f87adba4700 -1 osd.10 0 waiting for initial osdmap 2024-03-26T18:54:59.211 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 18:54:59 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T18:54:59.155+0000 7f87a9997700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:54:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:59 smithi144 ceph-mon[36053]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T18:54:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:59 smithi144 ceph-mon[36053]: from='osd.10 [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:54:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:59 smithi144 ceph-mon[36053]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T18:54:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:59 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:54:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:54:59 smithi144 ceph-mon[36053]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:54:59.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:59 smithi097 ceph-mon[31955]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T18:54:59.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:59 smithi097 ceph-mon[31955]: from='osd.10 [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:54:59.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:59 smithi097 ceph-mon[31955]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T18:54:59.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:59 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:54:59.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:54:59 smithi097 ceph-mon[31955]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T18:55:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:00 smithi144 ceph-mon[36053]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:55:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:00 smithi144 ceph-mon[36053]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:55:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:00 smithi144 ceph-mon[36053]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T18:55:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:00 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:55:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:00 smithi144 ceph-mon[36053]: osd.10 [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] boot 2024-03-26T18:55:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:00 smithi144 ceph-mon[36053]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T18:55:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:00 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:55:00.424 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:00 smithi105 ceph-mon[36270]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:55:00.424 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:00 smithi105 ceph-mon[36270]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:55:00.424 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:00 smithi105 ceph-mon[36270]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T18:55:00.424 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:00 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:55:00.424 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:00 smithi105 ceph-mon[36270]: osd.10 [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] boot 2024-03-26T18:55:00.424 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:00 smithi105 ceph-mon[36270]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T18:55:00.424 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:00 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:55:00.505 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:00 smithi097 ceph-mon[31955]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T18:55:00.505 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:00 smithi097 ceph-mon[31955]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T18:55:00.505 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:00 smithi097 ceph-mon[31955]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T18:55:00.505 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:00 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:55:00.505 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:00 smithi097 ceph-mon[31955]: osd.10 [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] boot 2024-03-26T18:55:00.505 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:00 smithi097 ceph-mon[31955]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T18:55:00.505 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:00 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:55:00.949 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T18:55:00.949 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T18:55:00.949 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-26T18:55:00.949 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-26T18:55:00.949 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135814 s, 772 MB/s 2024-03-26T18:55:00.949 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-26T18:55:01.651 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_1 2024-03-26T18:55:01.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:01 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:55:01.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:01 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:55:01.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:01 smithi097 ceph-mon[31955]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T18:55:01.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:01 smithi097 ceph-mon[31955]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:01.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:01 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:55:01.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:01 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:55:01.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:01 smithi144 ceph-mon[36053]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T18:55:01.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:01 smithi144 ceph-mon[36053]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:01.896 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:55:01.913 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:01 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:55:01.913 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:01 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:55:01.913 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:01 smithi105 ceph-mon[36270]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T18:55:01.913 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:01 smithi105 ceph-mon[36270]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:02.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T18:55:02.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi105 2024-03-26T18:55:02.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:02.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:02.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:02.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:02.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:02.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:02.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi105 to 4270M 2024-03-26T18:55:02.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:02.877 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:02.877 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:02.877 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:03.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T18:55:03.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi105 2024-03-26T18:55:03.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:03.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:03.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:03.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:03.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi105 to 4270M 2024-03-26T18:55:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:03.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T18:55:03.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi105 2024-03-26T18:55:03.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:03.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:03.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:03.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:03.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:03.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:03.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi105 to 4270M 2024-03-26T18:55:03.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:03.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:03.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:03.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.464+0000 7f170433d700 1 -- 172.21.15.144:0/644786979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16fc1012e0 msgr2=0x7f16fc101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.464+0000 7f170433d700 1 --2- 172.21.15.144:0/644786979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16fc1012e0 0x7f16fc101700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009990 tx=0x7f16f8009ca0 comp rx=0 tx=0).stop 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.464+0000 7f170433d700 1 -- 172.21.15.144:0/644786979 shutdown_connections 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.464+0000 7f170433d700 1 --2- 172.21.15.144:0/644786979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16fc102e80 0x7f16fc109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.464+0000 7f170433d700 1 --2- 172.21.15.144:0/644786979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16fc1012e0 0x7f16fc101700 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.464+0000 7f170433d700 1 --2- 172.21.15.144:0/644786979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16fc1024e0 0x7f16fc102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.464+0000 7f170433d700 1 -- 172.21.15.144:0/644786979 >> 172.21.15.144:0/644786979 conn(0x7f16fc0fc880 msgr2=0x7f16fc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.464+0000 7f170433d700 1 -- 172.21.15.144:0/644786979 shutdown_connections 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.464+0000 7f170433d700 1 -- 172.21.15.144:0/644786979 wait complete. 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f170433d700 1 Processor -- start 2024-03-26T18:55:03.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f170433d700 1 -- start start 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f170433d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16fc1012e0 0x7f16fc198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f170433d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16fc1024e0 0x7f16fc198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f170433d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16fc102e80 0x7f16fc19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f170433d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc10cfa0 con 0x7f16fc102e80 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f170433d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc10ce20 con 0x7f16fc1012e0 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f170433d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc10d120 con 0x7f16fc1024e0 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f17018d8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16fc1024e0 0x7f16fc198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f17018d8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16fc1024e0 0x7f16fc198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51126/0 (socket says 172.21.15.144:51126) 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f17018d8700 1 -- 172.21.15.144:0/1470779828 learned_addr learned my addr 172.21.15.144:0/1470779828 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f17020d9700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16fc1012e0 0x7f16fc198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f17018d8700 1 -- 172.21.15.144:0/1470779828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16fc1012e0 msgr2=0x7f16fc198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f17018d8700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16fc1012e0 0x7f16fc198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f17018d8700 1 -- 172.21.15.144:0/1470779828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16fc102e80 msgr2=0x7f16fc19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:03.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f17018d8700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16fc102e80 0x7f16fc19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:03.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f17018d8700 1 -- 172.21.15.144:0/1470779828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc1a0420 con 0x7f16fc1024e0 2024-03-26T18:55:03.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f17018d8700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16fc1024e0 0x7f16fc198d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16f400ba80 tx=0x7f16f400bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:03.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.465+0000 7f16f37fe700 1 -- 172.21.15.144:0/1470779828 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4019070 con 0x7f16fc1024e0 2024-03-26T18:55:03.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.466+0000 7f16f37fe700 1 -- 172.21.15.144:0/1470779828 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f4014440 con 0x7f16fc1024e0 2024-03-26T18:55:03.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.466+0000 7f16f37fe700 1 -- 172.21.15.144:0/1470779828 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40075d0 con 0x7f16fc1024e0 2024-03-26T18:55:03.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.466+0000 7f170433d700 1 -- 172.21.15.144:0/1470779828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16fc1a0710 con 0x7f16fc1024e0 2024-03-26T18:55:03.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.466+0000 7f170433d700 1 -- 172.21.15.144:0/1470779828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16fc1a0c80 con 0x7f16fc1024e0 2024-03-26T18:55:03.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.467+0000 7f170433d700 1 -- 172.21.15.144:0/1470779828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16fc0690a0 con 0x7f16fc1024e0 2024-03-26T18:55:03.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.468+0000 7f16f37fe700 1 -- 172.21.15.144:0/1470779828 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f16f401c070 con 0x7f16fc1024e0 2024-03-26T18:55:03.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.468+0000 7f16f37fe700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f16e80a3030 0x7f16e80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:03.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.468+0000 7f16f37fe700 1 -- 172.21.15.144:0/1470779828 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f16f40ce1a0 con 0x7f16fc1024e0 2024-03-26T18:55:03.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.469+0000 7f17020d9700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f16e80a3030 0x7f16e80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:03.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.470+0000 7f17020d9700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f16e80a3030 0x7f16e80a54f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16f800b5c0 tx=0x7f16f8003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:03.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.472+0000 7f16f37fe700 1 -- 172.21.15.144:0/1470779828 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f16fc0690a0 con 0x7f16fc1024e0 2024-03-26T18:55:03.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:03.677+0000 7f170433d700 1 -- 172.21.15.144:0/1470779828 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f16fc06a7a0 con 0x7f16e80a3030 2024-03-26T18:55:04.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:03 smithi097 ceph-mon[31955]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T18:55:04.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:03 smithi097 ceph-mon[31955]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:04.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:55:04.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:55:04.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:04.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:03 smithi144 ceph-mon[36053]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T18:55:04.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:03 smithi144 ceph-mon[36053]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:04.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:55:04.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:55:04.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:03 smithi105 ceph-mon[36270]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T18:55:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:03 smithi105 ceph-mon[36270]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T18:55:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T18:55:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:04.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:04 smithi144 ceph-mon[36053]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:55:04.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:04 smithi144 ceph-mon[36053]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T18:55:05.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:04 smithi097 ceph-mon[31955]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:55:05.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:04 smithi097 ceph-mon[31955]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T18:55:05.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:04 smithi105 ceph-mon[36270]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:55:05.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:04 smithi105 ceph-mon[36270]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T18:55:05.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:05 smithi144 ceph-mon[36053]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-03-26T18:55:06.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:05 smithi097 ceph-mon[31955]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-03-26T18:55:06.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:05 smithi105 ceph-mon[36270]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-03-26T18:55:07.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:06 smithi097 ceph-mon[31955]: from='client.? 172.21.15.144:0/1128092289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2347144e-d823-4816-ac8c-65722dd9cc62"}]: dispatch 2024-03-26T18:55:07.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:06 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2347144e-d823-4816-ac8c-65722dd9cc62"}]: dispatch 2024-03-26T18:55:07.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:06 smithi097 ceph-mon[31955]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2347144e-d823-4816-ac8c-65722dd9cc62"}]': finished 2024-03-26T18:55:07.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:06 smithi097 ceph-mon[31955]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T18:55:07.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:06 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:07.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:06 smithi144 ceph-mon[36053]: from='client.? 172.21.15.144:0/1128092289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2347144e-d823-4816-ac8c-65722dd9cc62"}]: dispatch 2024-03-26T18:55:07.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:06 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2347144e-d823-4816-ac8c-65722dd9cc62"}]: dispatch 2024-03-26T18:55:07.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:06 smithi144 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2347144e-d823-4816-ac8c-65722dd9cc62"}]': finished 2024-03-26T18:55:07.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:06 smithi144 ceph-mon[36053]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T18:55:07.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:06 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:07.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:06 smithi105 ceph-mon[36270]: from='client.? 172.21.15.144:0/1128092289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2347144e-d823-4816-ac8c-65722dd9cc62"}]: dispatch 2024-03-26T18:55:07.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:06 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2347144e-d823-4816-ac8c-65722dd9cc62"}]: dispatch 2024-03-26T18:55:07.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:06 smithi105 ceph-mon[36270]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2347144e-d823-4816-ac8c-65722dd9cc62"}]': finished 2024-03-26T18:55:07.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:06 smithi105 ceph-mon[36270]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T18:55:07.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:06 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:08.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:07 smithi097 ceph-mon[31955]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-26T18:55:08.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:07 smithi097 ceph-mon[31955]: from='client.? 172.21.15.144:0/1818407163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:55:08.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:07 smithi144 ceph-mon[36053]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-26T18:55:08.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:07 smithi144 ceph-mon[36053]: from='client.? 172.21.15.144:0/1818407163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:55:08.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:07 smithi105 ceph-mon[36270]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-26T18:55:08.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:07 smithi105 ceph-mon[36270]: from='client.? 172.21.15.144:0/1818407163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T18:55:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:09 smithi105 ceph-mon[36270]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-26T18:55:10.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:09 smithi097 ceph-mon[31955]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-26T18:55:10.362 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:09 smithi144 ceph-mon[36053]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-26T18:55:12.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:11 smithi105 ceph-mon[36270]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:55:12.264 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:11 smithi144 ceph-mon[36053]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:55:12.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:11 smithi097 ceph-mon[31955]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T18:55:14.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:13 smithi105 ceph-mon[36270]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-26T18:55:14.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:13 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T18:55:14.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:13 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:14.319 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:13 smithi144 ceph-mon[36053]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-26T18:55:14.319 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:13 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T18:55:14.319 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:13 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:14.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:13 smithi097 ceph-mon[31955]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-26T18:55:14.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:13 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T18:55:14.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:13 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:15.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:14 smithi105 ceph-mon[36270]: Deploying daemon osd.11 on smithi144 2024-03-26T18:55:15.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:14 smithi097 ceph-mon[31955]: Deploying daemon osd.11 on smithi144 2024-03-26T18:55:15.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:14 smithi144 ceph-mon[36053]: Deploying daemon osd.11 on smithi144 2024-03-26T18:55:16.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:15 smithi105 ceph-mon[36270]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:16.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:15 smithi097 ceph-mon[31955]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:16.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:15 smithi144 ceph-mon[36053]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:17 smithi105 ceph-mon[36270]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:18.214 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:17 smithi144 ceph-mon[36053]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:18.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:17 smithi097 ceph-mon[31955]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:20.109 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:19 smithi144 ceph-mon[36053]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:20.110 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:20.110 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:20.110 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:55:20.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:19 smithi097 ceph-mon[31955]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:20.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:20.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:20.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:55:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:19 smithi105 ceph-mon[36270]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:55:22.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:21 smithi144 ceph-mon[36053]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:22.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:21 smithi097 ceph-mon[31955]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:22.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:21 smithi105 ceph-mon[36270]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:23.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.012+0000 7f16f37fe700 1 -- 172.21.15.144:0/1470779828 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f16fc06a7a0 con 0x7f16e80a3030 2024-03-26T18:55:23.007 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 11 on host 'smithi144' 2024-03-26T18:55:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 -- 172.21.15.144:0/1470779828 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f16e80a3030 msgr2=0x7f16e80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f16e80a3030 0x7f16e80a54f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16f800b5c0 tx=0x7f16f8003960 comp rx=0 tx=0).stop 2024-03-26T18:55:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 -- 172.21.15.144:0/1470779828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16fc1024e0 msgr2=0x7f16fc198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16fc1024e0 0x7f16fc198d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16f400ba80 tx=0x7f16f400bd90 comp rx=0 tx=0).stop 2024-03-26T18:55:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 -- 172.21.15.144:0/1470779828 shutdown_connections 2024-03-26T18:55:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16fc102e80 0x7f16fc19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16fc1012e0 0x7f16fc198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f16e80a3030 0x7f16e80a54f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 --2- 172.21.15.144:0/1470779828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16fc1024e0 0x7f16fc198d50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:23.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 -- 172.21.15.144:0/1470779828 >> 172.21.15.144:0/1470779828 conn(0x7f16fc0fc880 msgr2=0x7f16fc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:23.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 -- 172.21.15.144:0/1470779828 shutdown_connections 2024-03-26T18:55:23.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:23.015+0000 7f170433d700 1 -- 172.21.15.144:0/1470779828 wait complete. 2024-03-26T18:55:23.630 DEBUG:teuthology.orchestra.run.smithi144:osd.11> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.11.service 2024-03-26T18:55:23.632 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-26T18:55:23.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T18:55:24.097 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:55:24.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:23 smithi097 ceph-mon[31955]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:24.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:23 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:23 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:23 smithi097 ceph-mon[31955]: from='osd.11 [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T18:55:24.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:23 smithi097 ceph-mon[31955]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T18:55:24.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:23 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:23 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:23 smithi144 ceph-mon[36053]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:23 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:23 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:23 smithi144 ceph-mon[36053]: from='osd.11 [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T18:55:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:23 smithi144 ceph-mon[36053]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T18:55:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:23 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:23 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:23 smithi105 ceph-mon[36270]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:24.499 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:23 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.499 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:23 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.499 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:23 smithi105 ceph-mon[36270]: from='osd.11 [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T18:55:24.499 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:23 smithi105 ceph-mon[36270]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T18:55:24.499 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:23 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:24.499 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:23 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:25.203 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:25 smithi144 ceph-mon[36053]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T18:55:25.203 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:25 smithi144 ceph-mon[36053]: from='osd.11 [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:55:25.203 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:25 smithi144 ceph-mon[36053]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T18:55:25.203 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:25 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:25.203 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:25 smithi144 ceph-mon[36053]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:55:25.203 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:55:25 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T18:55:25.004+0000 7f43c0775700 -1 osd.11 0 waiting for initial osdmap 2024-03-26T18:55:25.203 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:55:25 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T18:55:25.016+0000 7f43bc568700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T18:55:25.321 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:25 smithi097 ceph-mon[31955]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T18:55:25.321 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:25 smithi097 ceph-mon[31955]: from='osd.11 [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:55:25.321 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:25 smithi097 ceph-mon[31955]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T18:55:25.321 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:25 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:25.321 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:25 smithi097 ceph-mon[31955]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:55:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:25 smithi105 ceph-mon[36270]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T18:55:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:25 smithi105 ceph-mon[36270]: from='osd.11 [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:55:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:25 smithi105 ceph-mon[36270]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T18:55:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:25 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:25 smithi105 ceph-mon[36270]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T18:55:26.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:26 smithi144 ceph-mon[36053]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:26.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:26 smithi144 ceph-mon[36053]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:55:26.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:26 smithi144 ceph-mon[36053]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T18:55:26.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:26.266 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:26 smithi097 ceph-mon[31955]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:26.266 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:26 smithi097 ceph-mon[31955]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:55:26.266 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:26 smithi097 ceph-mon[31955]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T18:55:26.266 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.436+0000 7f79aec00700 1 -- 172.21.15.97:0/2831944583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79a8103ec0 msgr2=0x7f79a810a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.436+0000 7f79aec00700 1 --2- 172.21.15.97:0/2831944583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79a8103ec0 0x7f79a810a770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009fc0 tx=0x7f79a000b330 comp rx=0 tx=0).stop 2024-03-26T18:55:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.436+0000 7f79aec00700 1 -- 172.21.15.97:0/2831944583 shutdown_connections 2024-03-26T18:55:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.436+0000 7f79aec00700 1 --2- 172.21.15.97:0/2831944583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79a8103ec0 0x7f79a810a770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.436+0000 7f79aec00700 1 --2- 172.21.15.97:0/2831944583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a8102320 0x7f79a8102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.436+0000 7f79aec00700 1 --2- 172.21.15.97:0/2831944583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8103520 0x7f79a8103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.436+0000 7f79aec00700 1 -- 172.21.15.97:0/2831944583 >> 172.21.15.97:0/2831944583 conn(0x7f79a80fd8c0 msgr2=0x7f79a80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.436+0000 7f79aec00700 1 -- 172.21.15.97:0/2831944583 shutdown_connections 2024-03-26T18:55:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.436+0000 7f79aec00700 1 -- 172.21.15.97:0/2831944583 wait complete. 2024-03-26T18:55:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79aec00700 1 Processor -- start 2024-03-26T18:55:26.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79aec00700 1 -- start start 2024-03-26T18:55:26.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79aec00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a8102320 0x7f79a8076150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:26.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79aec00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8103520 0x7f79a80747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:26.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79aec00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79a8103ec0 0x7f79a8074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:26.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79aec00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f79a810def0 con 0x7f79a8103520 2024-03-26T18:55:26.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79aec00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f79a810dd70 con 0x7f79a8103ec0 2024-03-26T18:55:26.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79aec00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79a810e070 con 0x7f79a8102320 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79ace00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8103520 0x7f79a80747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79ace00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8103520 0x7f79a80747a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53950/0 (socket says 172.21.15.97:53950) 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.438+0000 7f79ace00700 1 -- 172.21.15.97:0/1641827436 learned_addr learned my addr 172.21.15.97:0/1641827436 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.439+0000 7f79ad800700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a8102320 0x7f79a8076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.439+0000 7f79ae200700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79a8103ec0 0x7f79a8074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.439+0000 7f79ace00700 1 -- 172.21.15.97:0/1641827436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a8102320 msgr2=0x7f79a8076150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.439+0000 7f79ace00700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a8102320 0x7f79a8076150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.439+0000 7f79ace00700 1 -- 172.21.15.97:0/1641827436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79a8103ec0 msgr2=0x7f79a8074ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.439+0000 7f79ace00700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79a8103ec0 0x7f79a8074ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.439+0000 7f79ace00700 1 -- 172.21.15.97:0/1641827436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a8075220 con 0x7f79a8103520 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.439+0000 7f79ad800700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a8102320 0x7f79a8076150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.439+0000 7f79ace00700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8103520 0x7f79a80747a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f799400ec10 tx=0x7f799400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:26.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.440+0000 7f79a6000700 1 -- 172.21.15.97:0/1641827436 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799400cd40 con 0x7f79a8103520 2024-03-26T18:55:26.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.440+0000 7f79a6000700 1 -- 172.21.15.97:0/1641827436 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7994004500 con 0x7f79a8103520 2024-03-26T18:55:26.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.440+0000 7f79aec00700 1 -- 172.21.15.97:0/1641827436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a8075510 con 0x7f79a8103520 2024-03-26T18:55:26.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.440+0000 7f79a6000700 1 -- 172.21.15.97:0/1641827436 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994010690 con 0x7f79a8103520 2024-03-26T18:55:26.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.440+0000 7f79aec00700 1 -- 172.21.15.97:0/1641827436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a81a6f30 con 0x7f79a8103520 2024-03-26T18:55:26.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.441+0000 7f79aec00700 1 -- 172.21.15.97:0/1641827436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a8069070 con 0x7f79a8103520 2024-03-26T18:55:26.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.443+0000 7f79a6000700 1 -- 172.21.15.97:0/1641827436 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f7994003f10 con 0x7f79a8103520 2024-03-26T18:55:26.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.445+0000 7f79a6000700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79900a3030 0x7f79900a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:26.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.446+0000 7f79a6000700 1 -- 172.21.15.97:0/1641827436 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f79940cf430 con 0x7f79a8103520 2024-03-26T18:55:26.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.446+0000 7f79ad800700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79900a3030 0x7f79900a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:26.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.446+0000 7f79ad800700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79900a3030 0x7f79900a54f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f79a8103380 tx=0x7f799c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:26.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.448+0000 7f79a6000700 1 -- 172.21.15.97:0/1641827436 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f799401f340 con 0x7f79a8103520 2024-03-26T18:55:26.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:26 smithi105 ceph-mon[36270]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:26.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:26 smithi105 ceph-mon[36270]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T18:55:26.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:26 smithi105 ceph-mon[36270]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T18:55:26.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:26.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.674+0000 7f79aec00700 1 -- 172.21.15.97:0/1641827436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f79a802dc00 con 0x7f79a8103520 2024-03-26T18:55:26.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.675+0000 7f79a6000700 1 -- 172.21.15.97:0/1641827436 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f799409c660 con 0x7f79a8103520 2024-03-26T18:55:26.668 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:26.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.677+0000 7f79aec00700 1 -- 172.21.15.97:0/1641827436 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79900a3030 msgr2=0x7f79900a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:26.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.677+0000 7f79aec00700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79900a3030 0x7f79900a54f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f79a8103380 tx=0x7f799c009260 comp rx=0 tx=0).stop 2024-03-26T18:55:26.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.677+0000 7f79aec00700 1 -- 172.21.15.97:0/1641827436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8103520 msgr2=0x7f79a80747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:26.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.677+0000 7f79aec00700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8103520 0x7f79a80747a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f799400ec10 tx=0x7f799400ef20 comp rx=0 tx=0).stop 2024-03-26T18:55:26.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.678+0000 7f79aec00700 1 -- 172.21.15.97:0/1641827436 shutdown_connections 2024-03-26T18:55:26.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.678+0000 7f79aec00700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79a8103ec0 0x7f79a8074ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:26.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.678+0000 7f79aec00700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79900a3030 0x7f79900a54f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:26.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.678+0000 7f79aec00700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a8102320 0x7f79a8076150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:26.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.678+0000 7f79aec00700 1 --2- 172.21.15.97:0/1641827436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8103520 0x7f79a80747a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:26.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.678+0000 7f79aec00700 1 -- 172.21.15.97:0/1641827436 >> 172.21.15.97:0/1641827436 conn(0x7f79a80fd8c0 msgr2=0x7f79a81067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:26.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.678+0000 7f79aec00700 1 -- 172.21.15.97:0/1641827436 shutdown_connections 2024-03-26T18:55:26.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:26.678+0000 7f79aec00700 1 -- 172.21.15.97:0/1641827436 wait complete. 2024-03-26T18:55:27.093 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:27 smithi097 ceph-mon[31955]: purged_snaps scrub starts 2024-03-26T18:55:27.093 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:27 smithi097 ceph-mon[31955]: purged_snaps scrub ok 2024-03-26T18:55:27.093 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:27.093 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:27 smithi097 ceph-mon[31955]: osd.11 [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] boot 2024-03-26T18:55:27.093 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:27 smithi097 ceph-mon[31955]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T18:55:27.093 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:27.093 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:27 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1641827436' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T18:55:27.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:27 smithi144 ceph-mon[36053]: purged_snaps scrub starts 2024-03-26T18:55:27.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:27 smithi144 ceph-mon[36053]: purged_snaps scrub ok 2024-03-26T18:55:27.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:27.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:27 smithi144 ceph-mon[36053]: osd.11 [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] boot 2024-03-26T18:55:27.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:27 smithi144 ceph-mon[36053]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T18:55:27.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:27.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:27 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1641827436' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T18:55:27.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:27 smithi105 ceph-mon[36270]: purged_snaps scrub starts 2024-03-26T18:55:27.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:27 smithi105 ceph-mon[36270]: purged_snaps scrub ok 2024-03-26T18:55:27.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:27.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:27 smithi105 ceph-mon[36270]: osd.11 [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] boot 2024-03-26T18:55:27.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:27 smithi105 ceph-mon[36270]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T18:55:27.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:55:27.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:27 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1641827436' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T18:55:27.510 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1711479326,"num_in_osds":12,"osd_in_since":1711479306,"num_remapped_pgs":0} 2024-03-26T18:55:27.511 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T18:55:27.933 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:55:28.296 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:28.296 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T18:55:28.296 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.296 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.296 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.296 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.296 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.296 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.297 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.297 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:28.297 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:28.297 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.297 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:55:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T18:55:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:28.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:28.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:55:28.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-26T18:55:28.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T18:55:28.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:55:28.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:28.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:28.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:28.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:55:29.324 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:29 smithi097 ceph-mon[31955]: Detected new or changed devices on smithi144 2024-03-26T18:55:29.324 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:29 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi144 to 4270M 2024-03-26T18:55:29.325 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:29.325 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:29.325 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:29 smithi144 ceph-mon[36053]: Detected new or changed devices on smithi144 2024-03-26T18:55:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:29 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi144 to 4270M 2024-03-26T18:55:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:29 smithi105 ceph-mon[36270]: Detected new or changed devices on smithi144 2024-03-26T18:55:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:29 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi144 to 4270M 2024-03-26T18:55:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.170+0000 7f24d9800700 1 -- 172.21.15.97:0/1778047529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d4074f00 msgr2=0x7f24d410e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.170+0000 7f24d9800700 1 --2- 172.21.15.97:0/1778047529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d4074f00 0x7f24d410e870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009990 tx=0x7f24c0009ca0 comp rx=0 tx=0).stop 2024-03-26T18:55:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.170+0000 7f24d9800700 1 -- 172.21.15.97:0/1778047529 shutdown_connections 2024-03-26T18:55:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.170+0000 7f24d9800700 1 --2- 172.21.15.97:0/1778047529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d410edb0 0x7f24d41111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.170+0000 7f24d9800700 1 --2- 172.21.15.97:0/1778047529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d40745e0 0x7f24d40749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.170+0000 7f24d9800700 1 --2- 172.21.15.97:0/1778047529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d4074f00 0x7f24d410e870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.170+0000 7f24d9800700 1 -- 172.21.15.97:0/1778047529 >> 172.21.15.97:0/1778047529 conn(0x7f24d40794e0 msgr2=0x7f24d40798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.171+0000 7f24d9800700 1 -- 172.21.15.97:0/1778047529 shutdown_connections 2024-03-26T18:55:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.171+0000 7f24d9800700 1 -- 172.21.15.97:0/1778047529 wait complete. 2024-03-26T18:55:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.172+0000 7f24d9800700 1 Processor -- start 2024-03-26T18:55:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.172+0000 7f24d9800700 1 -- start start 2024-03-26T18:55:30.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d9800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d40745e0 0x7f24d419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:30.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d9800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d4074f00 0x7f24d419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:30.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d9800700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d410edb0 0x7f24d419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:30.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d9800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41148b0 con 0x7f24d4074f00 2024-03-26T18:55:30.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d9800700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4114730 con 0x7f24d40745e0 2024-03-26T18:55:30.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d9800700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4114a30 con 0x7f24d410edb0 2024-03-26T18:55:30.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d2000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d4074f00 0x7f24d419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d2000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d4074f00 0x7f24d419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53966/0 (socket says 172.21.15.97:53966) 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d2000700 1 -- 172.21.15.97:0/1061921556 learned_addr learned my addr 172.21.15.97:0/1061921556 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d2a00700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d40745e0 0x7f24d419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d3400700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d410edb0 0x7f24d419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d2a00700 1 -- 172.21.15.97:0/1061921556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d410edb0 msgr2=0x7f24d419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.173+0000 7f24d2a00700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d410edb0 0x7f24d419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.174+0000 7f24d2a00700 1 -- 172.21.15.97:0/1061921556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d4074f00 msgr2=0x7f24d419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.174+0000 7f24d2a00700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d4074f00 0x7f24d419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.174+0000 7f24d2a00700 1 -- 172.21.15.97:0/1061921556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d419fdd0 con 0x7f24d40745e0 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.174+0000 7f24d2000700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d4074f00 0x7f24d419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.174+0000 7f24d2a00700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d40745e0 0x7f24d419adc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f24c400e9f0 tx=0x7f24c400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.174+0000 7f24cb400700 1 -- 172.21.15.97:0/1061921556 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c400c9c0 con 0x7f24d40745e0 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.174+0000 7f24cb400700 1 -- 172.21.15.97:0/1061921556 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c400cb60 con 0x7f24d40745e0 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.174+0000 7f24cb400700 1 -- 172.21.15.97:0/1061921556 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4018680 con 0x7f24d40745e0 2024-03-26T18:55:30.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.174+0000 7f24d9800700 1 -- 172.21.15.97:0/1061921556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d41a00c0 con 0x7f24d40745e0 2024-03-26T18:55:30.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.174+0000 7f24d9800700 1 -- 172.21.15.97:0/1061921556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d41a7940 con 0x7f24d40745e0 2024-03-26T18:55:30.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.175+0000 7f24d9800700 1 -- 172.21.15.97:0/1061921556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d4069070 con 0x7f24d40745e0 2024-03-26T18:55:30.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.177+0000 7f24cb400700 1 -- 172.21.15.97:0/1061921556 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f24c4010430 con 0x7f24d40745e0 2024-03-26T18:55:30.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.179+0000 7f24cb400700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f24b80a2fb0 0x7f24b80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:30.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.179+0000 7f24cb400700 1 -- 172.21.15.97:0/1061921556 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f24c40cf100 con 0x7f24d40745e0 2024-03-26T18:55:30.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.179+0000 7f24d2000700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f24b80a2fb0 0x7f24b80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:30.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.179+0000 7f24d2000700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f24b80a2fb0 0x7f24b80a5470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f24d4110ad0 tx=0x7f24c0009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:30.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.183+0000 7f24cb400700 1 -- 172.21.15.97:0/1061921556 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f24c409c330 con 0x7f24d40745e0 2024-03-26T18:55:30.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:30 smithi097 ceph-mon[31955]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:30 smithi144 ceph-mon[36053]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:30.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.406+0000 7f24d9800700 1 -- 172.21.15.97:0/1061921556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24d402dc00 con 0x7f24d40745e0 2024-03-26T18:55:30.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.408+0000 7f24cb400700 1 -- 172.21.15.97:0/1061921556 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18431 (secure 0 0 0) 0x7f24c409bc30 con 0x7f24d40745e0 2024-03-26T18:55:30.402 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:30.402 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":81,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:55:27.022168+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:55:30.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 -- 172.21.15.97:0/1061921556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f24b80a2fb0 msgr2=0x7f24b80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f24b80a2fb0 0x7f24b80a5470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f24d4110ad0 tx=0x7f24c0009570 comp rx=0 tx=0).stop 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 -- 172.21.15.97:0/1061921556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d40745e0 msgr2=0x7f24d419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d40745e0 0x7f24d419adc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f24c400e9f0 tx=0x7f24c400ed00 comp rx=0 tx=0).stop 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 -- 172.21.15.97:0/1061921556 shutdown_connections 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f24b80a2fb0 0x7f24b80a5470 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d410edb0 0x7f24d419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d40745e0 0x7f24d419adc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 --2- 172.21.15.97:0/1061921556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d4074f00 0x7f24d419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 -- 172.21.15.97:0/1061921556 >> 172.21.15.97:0/1061921556 conn(0x7f24d40794e0 msgr2=0x7f24d40ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 -- 172.21.15.97:0/1061921556 shutdown_connections 2024-03-26T18:55:30.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:30.411+0000 7f24d9800700 1 -- 172.21.15.97:0/1061921556 wait complete. 2024-03-26T18:55:30.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:30 smithi105 ceph-mon[36270]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:31.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:31 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1061921556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:55:31.264 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T18:51:11.459500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T18:55:31.264 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-26T18:55:31.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:31 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1061921556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:55:31.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:31 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1061921556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:55:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:55:32.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:32 smithi097 ceph-mon[31955]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:32 smithi144 ceph-mon[36053]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:32.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:32 smithi105 ceph-mon[36270]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.026+0000 7f48b5e00700 1 -- 172.21.15.97:0/251780226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48b010a680 msgr2=0x7f48b010aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.026+0000 7f48b5e00700 1 --2- 172.21.15.97:0/251780226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48b010a680 0x7f48b010aa60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f48a00099c0 tx=0x7f48a0009cd0 comp rx=0 tx=0).stop 2024-03-26T18:55:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.026+0000 7f48b5e00700 1 -- 172.21.15.97:0/251780226 shutdown_connections 2024-03-26T18:55:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.026+0000 7f48b5e00700 1 --2- 172.21.15.97:0/251780226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48b0104f90 0x7f48b0111450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.026+0000 7f48b5e00700 1 --2- 172.21.15.97:0/251780226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48b010a680 0x7f48b010aa60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.026+0000 7f48b5e00700 1 --2- 172.21.15.97:0/251780226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b01045d0 0x7f48b0104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.026+0000 7f48b5e00700 1 -- 172.21.15.97:0/251780226 >> 172.21.15.97:0/251780226 conn(0x7f48b0100080 msgr2=0x7f48b01024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.026+0000 7f48b5e00700 1 -- 172.21.15.97:0/251780226 shutdown_connections 2024-03-26T18:55:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.026+0000 7f48b5e00700 1 -- 172.21.15.97:0/251780226 wait complete. 2024-03-26T18:55:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.027+0000 7f48b5e00700 1 Processor -- start 2024-03-26T18:55:34.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.027+0000 7f48b5e00700 1 -- start start 2024-03-26T18:55:34.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48b5e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48b01045d0 0x7f48b01a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:34.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48b5e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b0104f90 0x7f48b01a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48b5e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48b010a680 0x7f48b019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48b5e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48b01148f0 con 0x7f48b010a680 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48b5e00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f48b0114770 con 0x7f48b01045d0 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48b5e00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48b0114a70 con 0x7f48b0104f90 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48af400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48b010a680 0x7f48b019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48aea00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48b01045d0 0x7f48b01a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48aea00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48b01045d0 0x7f48b01a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:40270/0 (socket says 172.21.15.97:40270) 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48aea00700 1 -- 172.21.15.97:0/2109452357 learned_addr learned my addr 172.21.15.97:0/2109452357 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.028+0000 7f48ae000700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b0104f90 0x7f48b01a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48aea00700 1 -- 172.21.15.97:0/2109452357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b0104f90 msgr2=0x7f48b01a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48aea00700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b0104f90 0x7f48b01a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48aea00700 1 -- 172.21.15.97:0/2109452357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48b010a680 msgr2=0x7f48b019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48aea00700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48b010a680 0x7f48b019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48aea00700 1 -- 172.21.15.97:0/2109452357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b019b230 con 0x7f48b01045d0 2024-03-26T18:55:34.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48af400700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48b010a680 0x7f48b019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:55:34.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48aea00700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48b01045d0 0x7f48b01a0910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48a0006010 tx=0x7f48a0003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:34.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48b4a00700 1 -- 172.21.15.97:0/2109452357 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0003680 con 0x7f48b01045d0 2024-03-26T18:55:34.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48b4a00700 1 -- 172.21.15.97:0/2109452357 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48a0003e50 con 0x7f48b01045d0 2024-03-26T18:55:34.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48b4a00700 1 -- 172.21.15.97:0/2109452357 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a000fe10 con 0x7f48b01045d0 2024-03-26T18:55:34.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.029+0000 7f48b5e00700 1 -- 172.21.15.97:0/2109452357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b019b4c0 con 0x7f48b01045d0 2024-03-26T18:55:34.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.030+0000 7f48b5e00700 1 -- 172.21.15.97:0/2109452357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b01a77e0 con 0x7f48b01045d0 2024-03-26T18:55:34.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.030+0000 7f48b5e00700 1 -- 172.21.15.97:0/2109452357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b010cb10 con 0x7f48b01045d0 2024-03-26T18:55:34.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.032+0000 7f48b4a00700 1 -- 172.21.15.97:0/2109452357 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f48a000f460 con 0x7f48b01045d0 2024-03-26T18:55:34.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.034+0000 7f48b4a00700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48940a31d0 0x7f48940a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:34.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.034+0000 7f48b4a00700 1 -- 172.21.15.97:0/2109452357 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f48a00cfbe0 con 0x7f48b01045d0 2024-03-26T18:55:34.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.034+0000 7f48ae000700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48940a31d0 0x7f48940a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:34.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.035+0000 7f48ae000700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48940a31d0 0x7f48940a5690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f48b0109c10 tx=0x7f489c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:34.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.038+0000 7f48b4a00700 1 -- 172.21.15.97:0/2109452357 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f48a0013070 con 0x7f48b01045d0 2024-03-26T18:55:34.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:34 smithi097 ceph-mon[31955]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.270+0000 7f48b5e00700 1 -- 172.21.15.97:0/2109452357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48b019bfc0 con 0x7f48b01045d0 2024-03-26T18:55:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.271+0000 7f48b4a00700 1 -- 172.21.15.97:0/2109452357 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f48a009cea0 con 0x7f48b01045d0 2024-03-26T18:55:34.265 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-26T18:55:34.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 -- 172.21.15.97:0/2109452357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48940a31d0 msgr2=0x7f48940a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:34.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48940a31d0 0x7f48940a5690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f48b0109c10 tx=0x7f489c009260 comp rx=0 tx=0).stop 2024-03-26T18:55:34.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 -- 172.21.15.97:0/2109452357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48b01045d0 msgr2=0x7f48b01a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:34.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48b01045d0 0x7f48b01a0910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48a0006010 tx=0x7f48a0003a90 comp rx=0 tx=0).stop 2024-03-26T18:55:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 -- 172.21.15.97:0/2109452357 shutdown_connections 2024-03-26T18:55:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48b010a680 0x7f48b019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48b01045d0 0x7f48b01a0910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48940a31d0 0x7f48940a5690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 --2- 172.21.15.97:0/2109452357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b0104f90 0x7f48b01a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 -- 172.21.15.97:0/2109452357 >> 172.21.15.97:0/2109452357 conn(0x7f48b0100080 msgr2=0x7f48b0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 -- 172.21.15.97:0/2109452357 shutdown_connections 2024-03-26T18:55:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:34.275+0000 7f48b5e00700 1 -- 172.21.15.97:0/2109452357 wait complete. 2024-03-26T18:55:34.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:34 smithi144 ceph-mon[36053]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:34.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:34 smithi105 ceph-mon[36270]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:35.085 INFO:tasks.cephadm:Adding mds.a on smithi097 2024-03-26T18:55:35.085 INFO:tasks.cephadm:Adding mds.d on smithi097 2024-03-26T18:55:35.086 INFO:tasks.cephadm:Adding mds.g on smithi097 2024-03-26T18:55:35.086 INFO:tasks.cephadm:Adding mds.j on smithi097 2024-03-26T18:55:35.086 INFO:tasks.cephadm:Adding mds.b on smithi105 2024-03-26T18:55:35.086 INFO:tasks.cephadm:Adding mds.e on smithi105 2024-03-26T18:55:35.086 INFO:tasks.cephadm:Adding mds.h on smithi105 2024-03-26T18:55:35.086 INFO:tasks.cephadm:Adding mds.k on smithi105 2024-03-26T18:55:35.086 INFO:tasks.cephadm:Adding mds.c on smithi144 2024-03-26T18:55:35.086 INFO:tasks.cephadm:Adding mds.f on smithi144 2024-03-26T18:55:35.086 INFO:tasks.cephadm:Adding mds.i on smithi144 2024-03-26T18:55:35.086 INFO:tasks.cephadm:Adding mds.l on smithi144 2024-03-26T18:55:35.086 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l' 2024-03-26T18:55:35.335 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:55:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:35 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2109452357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:55:35.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:35 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2109452357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:55:35.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:35 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2109452357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:55:36.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:36 smithi097 ceph-mon[31955]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:36.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:36 smithi144 ceph-mon[36053]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:36 smithi105 ceph-mon[36270]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:36.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.964+0000 7f229c8a5700 1 -- 172.21.15.144:0/4260590078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981024d0 msgr2=0x7f2298102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:36.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.964+0000 7f229c8a5700 1 --2- 172.21.15.144:0/4260590078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981024d0 0x7f2298102930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2288009990 tx=0x7f2288009ca0 comp rx=0 tx=0).stop 2024-03-26T18:55:36.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.964+0000 7f229c8a5700 1 -- 172.21.15.144:0/4260590078 shutdown_connections 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.964+0000 7f229c8a5700 1 --2- 172.21.15.144:0/4260590078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2298102e70 0x7f2298109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.964+0000 7f229c8a5700 1 --2- 172.21.15.144:0/4260590078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22981012d0 0x7f22981016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.964+0000 7f229c8a5700 1 --2- 172.21.15.144:0/4260590078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981024d0 0x7f2298102930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.965+0000 7f229c8a5700 1 -- 172.21.15.144:0/4260590078 >> 172.21.15.144:0/4260590078 conn(0x7f22980fc870 msgr2=0x7f22980fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.965+0000 7f229c8a5700 1 -- 172.21.15.144:0/4260590078 shutdown_connections 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.965+0000 7f229c8a5700 1 -- 172.21.15.144:0/4260590078 wait complete. 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.965+0000 7f229c8a5700 1 Processor -- start 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.965+0000 7f229c8a5700 1 -- start start 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229c8a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981012d0 0x7f2298197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229c8a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22981024d0 0x7f2298197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229c8a5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2298102e70 0x7f229819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229c8a5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f229810cf50 con 0x7f22981024d0 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229c8a5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f229810cdd0 con 0x7f2298102e70 2024-03-26T18:55:36.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229c8a5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f229810d0d0 con 0x7f22981012d0 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981012d0 0x7f2298197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981012d0 0x7f2298197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50468/0 (socket says 172.21.15.144:50468) 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229659c700 1 -- 172.21.15.144:0/2778998279 learned_addr learned my addr 172.21.15.144:0/2778998279 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f2296d9d700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2298102e70 0x7f229819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229659c700 1 -- 172.21.15.144:0/2778998279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2298102e70 msgr2=0x7f229819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229659c700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2298102e70 0x7f229819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229659c700 1 -- 172.21.15.144:0/2778998279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22981024d0 msgr2=0x7f2298197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229659c700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22981024d0 0x7f2298197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229659c700 1 -- 172.21.15.144:0/2778998279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229819f190 con 0x7f22981012d0 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229659c700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981012d0 0x7f2298197530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f228000b530 tx=0x7f228000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f228f7fe700 1 -- 172.21.15.144:0/2778998279 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280015020 con 0x7f22981012d0 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229c8a5700 1 -- 172.21.15.144:0/2778998279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229819f480 con 0x7f22981012d0 2024-03-26T18:55:36.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f228f7fe700 1 -- 172.21.15.144:0/2778998279 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2280011d10 con 0x7f22981012d0 2024-03-26T18:55:36.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.966+0000 7f229c8a5700 1 -- 172.21.15.144:0/2778998279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229819fa90 con 0x7f22981012d0 2024-03-26T18:55:36.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.967+0000 7f228f7fe700 1 -- 172.21.15.144:0/2778998279 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228000f2e0 con 0x7f22981012d0 2024-03-26T18:55:36.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.968+0000 7f229c8a5700 1 -- 172.21.15.144:0/2778998279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2298069070 con 0x7f22981012d0 2024-03-26T18:55:36.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.969+0000 7f228f7fe700 1 -- 172.21.15.144:0/2778998279 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f2280018020 con 0x7f22981012d0 2024-03-26T18:55:36.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.969+0000 7f228f7fe700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22840a3030 0x7f22840a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:36.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.969+0000 7f228f7fe700 1 -- 172.21.15.144:0/2778998279 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f22800ce600 con 0x7f22981012d0 2024-03-26T18:55:36.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.970+0000 7f2295d9b700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22840a3030 0x7f22840a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:36.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.971+0000 7f2295d9b700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22840a3030 0x7f22840a54f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2288005f50 tx=0x7f2288005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:36.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:36.974+0000 7f228f7fe700 1 -- 172.21.15.144:0/2778998279 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f228001f020 con 0x7f22981012d0 2024-03-26T18:55:37.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.179+0000 7f229c8a5700 1 -- 172.21.15.144:0/2778998279 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}) -- 0x7f22980629f0 con 0x7f22840a3030 2024-03-26T18:55:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.194+0000 7f228f7fe700 1 -- 172.21.15.144:0/2778998279 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f22980629f0 con 0x7f22840a3030 2024-03-26T18:55:37.187 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mds.all update... 2024-03-26T18:55:37.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 -- 172.21.15.144:0/2778998279 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22840a3030 msgr2=0x7f22840a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:37.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22840a3030 0x7f22840a54f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2288005f50 tx=0x7f2288005e80 comp rx=0 tx=0).stop 2024-03-26T18:55:37.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 -- 172.21.15.144:0/2778998279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981012d0 msgr2=0x7f2298197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:37.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981012d0 0x7f2298197530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f228000b530 tx=0x7f228000b840 comp rx=0 tx=0).stop 2024-03-26T18:55:37.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 -- 172.21.15.144:0/2778998279 shutdown_connections 2024-03-26T18:55:37.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2298102e70 0x7f229819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:37.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22840a3030 0x7f22840a54f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:37.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981012d0 0x7f2298197530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:37.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 --2- 172.21.15.144:0/2778998279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22981024d0 0x7f2298197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:37.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 -- 172.21.15.144:0/2778998279 >> 172.21.15.144:0/2778998279 conn(0x7f22980fc870 msgr2=0x7f22981057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:37.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 -- 172.21.15.144:0/2778998279 shutdown_connections 2024-03-26T18:55:37.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T18:55:37.197+0000 7f229c8a5700 1 -- 172.21.15.144:0/2778998279 wait complete. 2024-03-26T18:55:37.818 DEBUG:teuthology.orchestra.run.smithi097:mds.a> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.a.service 2024-03-26T18:55:37.825 DEBUG:teuthology.orchestra.run.smithi097:mds.d> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.d.service 2024-03-26T18:55:37.870 DEBUG:teuthology.orchestra.run.smithi097:mds.g> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.g.service 2024-03-26T18:55:37.916 DEBUG:teuthology.orchestra.run.smithi097:mds.j> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.j.service 2024-03-26T18:55:37.962 DEBUG:teuthology.orchestra.run.smithi105:mds.b> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.b.service 2024-03-26T18:55:37.964 DEBUG:teuthology.orchestra.run.smithi105:mds.e> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.e.service 2024-03-26T18:55:38.008 DEBUG:teuthology.orchestra.run.smithi105:mds.h> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.h.service 2024-03-26T18:55:38.050 DEBUG:teuthology.orchestra.run.smithi105:mds.k> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.k.service 2024-03-26T18:55:38.093 DEBUG:teuthology.orchestra.run.smithi144:mds.c> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.c.service 2024-03-26T18:55:38.096 DEBUG:teuthology.orchestra.run.smithi144:mds.f> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.f.service 2024-03-26T18:55:38.138 DEBUG:teuthology.orchestra.run.smithi144:mds.i> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.i.service 2024-03-26T18:55:38.181 DEBUG:teuthology.orchestra.run.smithi144:mds.l> sudo journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.l.service 2024-03-26T18:55:38.224 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-26T18:55:38.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:55:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:38 smithi097 ceph-mon[31955]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:38 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:38 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:55:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:38 smithi144 ceph-mon[36053]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:38 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:38 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:55:38.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:38 smithi105 ceph-mon[36270]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:38.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:38.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:55:38.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.468+0000 7f85be800640 1 -- 172.21.15.97:0/3093803983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b8102ad0 msgr2=0x7f85b8102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.468+0000 7f85be800640 1 --2- 172.21.15.97:0/3093803983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b8102ad0 0x7f85b8102f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009900 tx=0x7f85a402bc30 comp rx=0 tx=0).stop 2024-03-26T18:55:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.468+0000 7f85be800640 1 -- 172.21.15.97:0/3093803983 shutdown_connections 2024-03-26T18:55:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.468+0000 7f85be800640 1 --2- 172.21.15.97:0/3093803983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b8103490 0x7f85b8109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.468+0000 7f85be800640 1 --2- 172.21.15.97:0/3093803983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b8102ad0 0x7f85b8102f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.468+0000 7f85be800640 1 --2- 172.21.15.97:0/3093803983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85b81018e0 0x7f85b8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.468+0000 7f85be800640 1 -- 172.21.15.97:0/3093803983 >> 172.21.15.97:0/3093803983 conn(0x7f85b80fce70 msgr2=0x7f85b80ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.468+0000 7f85be800640 1 -- 172.21.15.97:0/3093803983 shutdown_connections 2024-03-26T18:55:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.468+0000 7f85be800640 1 -- 172.21.15.97:0/3093803983 wait complete. 2024-03-26T18:55:38.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.469+0000 7f85be800640 1 Processor -- start 2024-03-26T18:55:38.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85be800640 1 -- start start 2024-03-26T18:55:38.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85be800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85b81018e0 0x7f85b8197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85be800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b8102ad0 0x7f85b8198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85be800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b8103490 0x7f85b819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85be800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85b810d5c0 con 0x7f85b81018e0 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85be800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85b810d440 con 0x7f85b8103490 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85be800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85b810d740 con 0x7f85b8102ad0 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85b81018e0 0x7f85b8197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85b6a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b8102ad0 0x7f85b8198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85b6a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b8102ad0 0x7f85b8198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52388/0 (socket says 172.21.15.97:52388) 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85b81018e0 0x7f85b8197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41674/0 (socket says 172.21.15.97:41674) 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85b6a00640 1 -- 172.21.15.97:0/3923321371 learned_addr learned my addr 172.21.15.97:0/3923321371 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.470+0000 7f85b7e00640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b8103490 0x7f85b819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85b6a00640 1 -- 172.21.15.97:0/3923321371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b8103490 msgr2=0x7f85b819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85b6a00640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b8103490 0x7f85b819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85b6a00640 1 -- 172.21.15.97:0/3923321371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85b81018e0 msgr2=0x7f85b8197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85b6a00640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85b81018e0 0x7f85b8197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85b6a00640 1 -- 172.21.15.97:0/3923321371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b819f840 con 0x7f85b8102ad0 2024-03-26T18:55:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85b6a00640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b8102ad0 0x7f85b8198170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009230 tx=0x7f85a4005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85afe00640 1 -- 172.21.15.97:0/3923321371 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4041070 con 0x7f85b8102ad0 2024-03-26T18:55:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85afe00640 1 -- 172.21.15.97:0/3923321371 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a40044d0 con 0x7f85b8102ad0 2024-03-26T18:55:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85afe00640 1 -- 172.21.15.97:0/3923321371 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4034440 con 0x7f85b8102ad0 2024-03-26T18:55:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85be800640 1 -- 172.21.15.97:0/3923321371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b819fad0 con 0x7f85b8102ad0 2024-03-26T18:55:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.471+0000 7f85be800640 1 -- 172.21.15.97:0/3923321371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b81a0010 con 0x7f85b8102ad0 2024-03-26T18:55:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.472+0000 7f85be800640 1 -- 172.21.15.97:0/3923321371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b8108c00 con 0x7f85b8102ad0 2024-03-26T18:55:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.474+0000 7f85afe00640 1 -- 172.21.15.97:0/3923321371 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f85a4044070 con 0x7f85b8102ad0 2024-03-26T18:55:38.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.475+0000 7f85afe00640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f859c0a2fa0 0x7f859c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:38.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.476+0000 7f85afe00640 1 -- 172.21.15.97:0/3923321371 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f85a40f0bf0 con 0x7f85b8102ad0 2024-03-26T18:55:38.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.476+0000 7f85b7400640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f859c0a2fa0 0x7f859c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:38.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.477+0000 7f85b7400640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f859c0a2fa0 0x7f859c0a5460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85b8199100 tx=0x7f85a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:38.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.480+0000 7f85afe00640 1 -- 172.21.15.97:0/3923321371 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f85a40bd810 con 0x7f85b8102ad0 2024-03-26T18:55:38.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.680+0000 7f85be800640 1 -- 172.21.15.97:0/3923321371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85b80624f0 con 0x7f85b8102ad0 2024-03-26T18:55:38.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.683+0000 7f85afe00640 1 -- 172.21.15.97:0/3923321371 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18431 (secure 0 0 0) 0x7f85a40c18b0 con 0x7f85b8102ad0 2024-03-26T18:55:38.676 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:38.677 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":81,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:55:27.022168+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 -- 172.21.15.97:0/3923321371 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f859c0a2fa0 msgr2=0x7f859c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f859c0a2fa0 0x7f859c0a5460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85b8199100 tx=0x7f85a8005f90 comp rx=0 tx=0).stop 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 -- 172.21.15.97:0/3923321371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b8102ad0 msgr2=0x7f85b8198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b8102ad0 0x7f85b8198170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009230 tx=0x7f85a4005d50 comp rx=0 tx=0).stop 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 -- 172.21.15.97:0/3923321371 shutdown_connections 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f859c0a2fa0 0x7f859c0a5460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b8103490 0x7f85b819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b8102ad0 0x7f85b8198170 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 --2- 172.21.15.97:0/3923321371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85b81018e0 0x7f85b8197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 -- 172.21.15.97:0/3923321371 >> 172.21.15.97:0/3923321371 conn(0x7f85b80fce70 msgr2=0x7f85b8105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.685+0000 7f85be800640 1 -- 172.21.15.97:0/3923321371 shutdown_connections 2024-03-26T18:55:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.686+0000 7f85be800640 1 -- 172.21.15.97:0/3923321371 wait complete. 2024-03-26T18:55:38.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T18:51:11.459500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T18:55:38.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:55:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.927+0000 7f64b7600640 1 -- 172.21.15.97:0/2371599077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64b0076280 msgr2=0x7f64b00766a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.927+0000 7f64b7600640 1 --2- 172.21.15.97:0/2371599077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64b0076280 0x7f64b00766a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009900 tx=0x7f64a002bc40 comp rx=0 tx=0).stop 2024-03-26T18:55:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.928+0000 7f64b7600640 1 -- 172.21.15.97:0/2371599077 shutdown_connections 2024-03-26T18:55:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.928+0000 7f64b7600640 1 --2- 172.21.15.97:0/2371599077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64b0077e60 0x7f64b0078700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.928+0000 7f64b7600640 1 --2- 172.21.15.97:0/2371599077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64b00774c0 0x7f64b0077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.928+0000 7f64b7600640 1 --2- 172.21.15.97:0/2371599077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64b0076280 0x7f64b00766a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.928+0000 7f64b7600640 1 -- 172.21.15.97:0/2371599077 >> 172.21.15.97:0/2371599077 conn(0x7f64b00ff070 msgr2=0x7f64b01014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.928+0000 7f64b7600640 1 -- 172.21.15.97:0/2371599077 shutdown_connections 2024-03-26T18:55:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.928+0000 7f64b7600640 1 -- 172.21.15.97:0/2371599077 wait complete. 2024-03-26T18:55:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b7600640 1 Processor -- start 2024-03-26T18:55:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b7600640 1 -- start start 2024-03-26T18:55:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b7600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64b0076280 0x7f64b019e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b7600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64b00774c0 0x7f64b019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b7600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64b0077e60 0x7f64b01a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b7600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64b0113bd0 con 0x7f64b0076280 2024-03-26T18:55:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b7600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f64b0113a50 con 0x7f64b00774c0 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b7600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64b0113d50 con 0x7f64b0077e60 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64b0076280 0x7f64b019e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64b0076280 0x7f64b019e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41698/0 (socket says 172.21.15.97:41698) 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b4c00640 1 -- 172.21.15.97:0/821133059 learned_addr learned my addr 172.21.15.97:0/821133059 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.929+0000 7f64b5600640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64b0077e60 0x7f64b01a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64a7e00640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64b00774c0 0x7f64b019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64b4c00640 1 -- 172.21.15.97:0/821133059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64b0077e60 msgr2=0x7f64b01a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64b4c00640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64b0077e60 0x7f64b01a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64b4c00640 1 -- 172.21.15.97:0/821133059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64b00774c0 msgr2=0x7f64b019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64b4c00640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64b00774c0 0x7f64b019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64b4c00640 1 -- 172.21.15.97:0/821133059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b01a5e20 con 0x7f64b0076280 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64b5600640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64b0077e60 0x7f64b01a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64b4c00640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64b0076280 0x7f64b019e200 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f64a002c8f0 tx=0x7f64a002c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64ae000640 1 -- 172.21.15.97:0/821133059 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0041070 con 0x7f64b0076280 2024-03-26T18:55:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64b7600640 1 -- 172.21.15.97:0/821133059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b01a60b0 con 0x7f64b0076280 2024-03-26T18:55:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64ae000640 1 -- 172.21.15.97:0/821133059 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a0004450 con 0x7f64b0076280 2024-03-26T18:55:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.930+0000 7f64ae000640 1 -- 172.21.15.97:0/821133059 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0003df0 con 0x7f64b0076280 2024-03-26T18:55:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.931+0000 7f64b7600640 1 -- 172.21.15.97:0/821133059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b01a65a0 con 0x7f64b0076280 2024-03-26T18:55:38.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.932+0000 7f64b7600640 1 -- 172.21.15.97:0/821133059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b0110610 con 0x7f64b0076280 2024-03-26T18:55:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.934+0000 7f64ae000640 1 -- 172.21.15.97:0/821133059 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f64a0044070 con 0x7f64b0076280 2024-03-26T18:55:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.935+0000 7f64ae000640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64940a2f20 0x7f64940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.935+0000 7f64ae000640 1 -- 172.21.15.97:0/821133059 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f64a00f0a50 con 0x7f64b0076280 2024-03-26T18:55:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.935+0000 7f64a7e00640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64940a2f20 0x7f64940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.936+0000 7f64a7e00640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64940a2f20 0x7f64940a53e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64980095f0 tx=0x7f6498005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:38.940+0000 7f64ae000640 1 -- 172.21.15.97:0/821133059 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f64a00bdd20 con 0x7f64b0076280 2024-03-26T18:55:39.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.140+0000 7f64b7600640 1 -- 172.21.15.97:0/821133059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64b00624c0 con 0x7f64b0076280 2024-03-26T18:55:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.141+0000 7f64ae000640 1 -- 172.21.15.97:0/821133059 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f64a00bd620 con 0x7f64b0076280 2024-03-26T18:55:39.134 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-26T18:55:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.143+0000 7f64b7600640 1 -- 172.21.15.97:0/821133059 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64940a2f20 msgr2=0x7f64940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.143+0000 7f64b7600640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64940a2f20 0x7f64940a53e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64980095f0 tx=0x7f6498005f90 comp rx=0 tx=0).stop 2024-03-26T18:55:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.143+0000 7f64b7600640 1 -- 172.21.15.97:0/821133059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64b0076280 msgr2=0x7f64b019e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.143+0000 7f64b7600640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64b0076280 0x7f64b019e200 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f64a002c8f0 tx=0x7f64a002c660 comp rx=0 tx=0).stop 2024-03-26T18:55:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.144+0000 7f64b7600640 1 -- 172.21.15.97:0/821133059 shutdown_connections 2024-03-26T18:55:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.144+0000 7f64b7600640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64940a2f20 0x7f64940a53e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.144+0000 7f64b7600640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64b0077e60 0x7f64b01a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.144+0000 7f64b7600640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64b00774c0 0x7f64b019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.144+0000 7f64b7600640 1 --2- 172.21.15.97:0/821133059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64b0076280 0x7f64b019e200 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.144+0000 7f64b7600640 1 -- 172.21.15.97:0/821133059 >> 172.21.15.97:0/821133059 conn(0x7f64b00ff070 msgr2=0x7f64b01007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.144+0000 7f64b7600640 1 -- 172.21.15.97:0/821133059 shutdown_connections 2024-03-26T18:55:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.144+0000 7f64b7600640 1 -- 172.21.15.97:0/821133059 wait complete. 2024-03-26T18:55:39.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:55:39.318 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:39 smithi144 ceph-mon[36053]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:55:39.318 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:39 smithi144 ceph-mon[36053]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-03-26T18:55:39.318 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:39.318 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:39.318 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:39.318 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:39.318 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:39.318 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:39 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:39.318 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:39 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3923321371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:55:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:39 smithi097 ceph-mon[31955]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:55:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:39 smithi097 ceph-mon[31955]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-03-26T18:55:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:39 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:39.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:39 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3923321371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:55:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.373+0000 7fea3b600640 1 -- 172.21.15.97:0/1272698565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea34103b00 msgr2=0x7fea34103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.373+0000 7fea3b600640 1 --2- 172.21.15.97:0/1272698565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea34103b00 0x7fea34103f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea28009900 tx=0x7fea2802bc30 comp rx=0 tx=0).stop 2024-03-26T18:55:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.374+0000 7fea3b600640 1 -- 172.21.15.97:0/1272698565 shutdown_connections 2024-03-26T18:55:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.374+0000 7fea3b600640 1 --2- 172.21.15.97:0/1272698565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea341056b0 0x7fea3410bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.374+0000 7fea3b600640 1 --2- 172.21.15.97:0/1272698565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea34104cf0 0x7fea34105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.374+0000 7fea3b600640 1 --2- 172.21.15.97:0/1272698565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea34103b00 0x7fea34103f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.374+0000 7fea3b600640 1 -- 172.21.15.97:0/1272698565 >> 172.21.15.97:0/1272698565 conn(0x7fea340ff070 msgr2=0x7fea341014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.374+0000 7fea3b600640 1 -- 172.21.15.97:0/1272698565 shutdown_connections 2024-03-26T18:55:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.374+0000 7fea3b600640 1 -- 172.21.15.97:0/1272698565 wait complete. 2024-03-26T18:55:39.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea3b600640 1 Processor -- start 2024-03-26T18:55:39.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea3b600640 1 -- start start 2024-03-26T18:55:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea3b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea34103b00 0x7fea34199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea3b600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea34104cf0 0x7fea3419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea3b600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea341056b0 0x7fea341a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea3b600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea3410f7d0 con 0x7fea34103b00 2024-03-26T18:55:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea3b600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea3410f650 con 0x7fea341056b0 2024-03-26T18:55:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea3b600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea3410f950 con 0x7fea34104cf0 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea38c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea34103b00 0x7fea34199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea38c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea34103b00 0x7fea34199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41730/0 (socket says 172.21.15.97:41730) 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.375+0000 7fea38c00640 1 -- 172.21.15.97:0/1304613497 learned_addr learned my addr 172.21.15.97:0/1304613497 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea33e00640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea34104cf0 0x7fea3419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea39600640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea341056b0 0x7fea341a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea38c00640 1 -- 172.21.15.97:0/1304613497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea34104cf0 msgr2=0x7fea3419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea38c00640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea34104cf0 0x7fea3419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea38c00640 1 -- 172.21.15.97:0/1304613497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea341056b0 msgr2=0x7fea341a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea38c00640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea341056b0 0x7fea341a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea38c00640 1 -- 172.21.15.97:0/1304613497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea341a1a00 con 0x7fea34103b00 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea39600640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea341056b0 0x7fea341a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea38c00640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea34103b00 0x7fea34199de0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea28000c00 tx=0x7fea2802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea31600640 1 -- 172.21.15.97:0/1304613497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28043020 con 0x7fea34103b00 2024-03-26T18:55:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea31600640 1 -- 172.21.15.97:0/1304613497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea280054e0 con 0x7fea34103b00 2024-03-26T18:55:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.376+0000 7fea3b600640 1 -- 172.21.15.97:0/1304613497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea341a1c90 con 0x7fea34103b00 2024-03-26T18:55:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.377+0000 7fea31600640 1 -- 172.21.15.97:0/1304613497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28003c40 con 0x7fea34103b00 2024-03-26T18:55:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.377+0000 7fea3b600640 1 -- 172.21.15.97:0/1304613497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea341a2180 con 0x7fea34103b00 2024-03-26T18:55:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.378+0000 7fea3b600640 1 -- 172.21.15.97:0/1304613497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3410ae20 con 0x7fea34103b00 2024-03-26T18:55:39.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.380+0000 7fea31600640 1 -- 172.21.15.97:0/1304613497 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fea28046020 con 0x7fea34103b00 2024-03-26T18:55:39.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.382+0000 7fea31600640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fea1c0a2fa0 0x7fea1c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:39.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.382+0000 7fea31600640 1 -- 172.21.15.97:0/1304613497 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fea280f1080 con 0x7fea34103b00 2024-03-26T18:55:39.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.382+0000 7fea33e00640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fea1c0a2fa0 0x7fea1c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:39.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.383+0000 7fea33e00640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fea1c0a2fa0 0x7fea1c0a5460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea3419b2f0 tx=0x7fea20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:39.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.386+0000 7fea31600640 1 -- 172.21.15.97:0/1304613497 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fea2804d020 con 0x7fea34103b00 2024-03-26T18:55:39.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:39 smithi105 ceph-mon[36270]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:55:39.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:39 smithi105 ceph-mon[36270]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-03-26T18:55:39.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:39.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:55:39.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:39.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:39.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:39.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:39 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:39.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:39 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3923321371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:55:39.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.626+0000 7fea3b600640 1 -- 172.21.15.97:0/1304613497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea340624c0 con 0x7fea34103b00 2024-03-26T18:55:39.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.626+0000 7fea31600640 1 -- 172.21.15.97:0/1304613497 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fea280be2b0 con 0x7fea34103b00 2024-03-26T18:55:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:39.620 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-26T18:55:39.621 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 1 2024-03-26T18:55:39.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 -- 172.21.15.97:0/1304613497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fea1c0a2fa0 msgr2=0x7fea1c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:39.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fea1c0a2fa0 0x7fea1c0a5460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea3419b2f0 tx=0x7fea20005f90 comp rx=0 tx=0).stop 2024-03-26T18:55:39.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 -- 172.21.15.97:0/1304613497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea34103b00 msgr2=0x7fea34199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:39.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea34103b00 0x7fea34199de0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea28000c00 tx=0x7fea2802c650 comp rx=0 tx=0).stop 2024-03-26T18:55:39.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 -- 172.21.15.97:0/1304613497 shutdown_connections 2024-03-26T18:55:39.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fea1c0a2fa0 0x7fea1c0a5460 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea341056b0 0x7fea341a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea34104cf0 0x7fea3419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 --2- 172.21.15.97:0/1304613497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea34103b00 0x7fea34199de0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:39.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 -- 172.21.15.97:0/1304613497 >> 172.21.15.97:0/1304613497 conn(0x7fea340ff070 msgr2=0x7fea34107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:39.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 -- 172.21.15.97:0/1304613497 shutdown_connections 2024-03-26T18:55:39.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:39.629+0000 7fea3b600640 1 -- 172.21.15.97:0/1304613497 wait complete. 2024-03-26T18:55:40.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:40 smithi144 ceph-mon[36053]: Deploying daemon mds.l on smithi144 2024-03-26T18:55:40.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:40 smithi144 ceph-mon[36053]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:40.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:40 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/821133059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:55:40.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:40 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1304613497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:40.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:40 smithi097 ceph-mon[31955]: Deploying daemon mds.l on smithi144 2024-03-26T18:55:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:40 smithi097 ceph-mon[31955]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:40 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/821133059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:55:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:40 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1304613497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:40 smithi105 ceph-mon[36270]: Deploying daemon mds.l on smithi144 2024-03-26T18:55:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:40 smithi105 ceph-mon[36270]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:40 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/821133059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:55:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:40 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1304613497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:40.626 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:55:40 smithi144 systemd[1]: Starting Ceph mds.l for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:55:41.045 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:55:40 smithi144 podman[66036]: 2024-03-26 18:55:40.658086812 +0000 UTC m=+0.017496320 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:55:41.045 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:55:40 smithi144 podman[66036]: 2024-03-26 18:55:40.816680827 +0000 UTC m=+0.176090335 container create ca0f9421c468d7abd974c9aaea97fb818f0edf061f8349b595668f98b9f7d2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:55:41.045 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:55:41 smithi144 podman[66036]: 2024-03-26 18:55:41.052708612 +0000 UTC m=+0.412118128 container init ca0f9421c468d7abd974c9aaea97fb818f0edf061f8349b595668f98b9f7d2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:55:41.376 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:55:41 smithi144 podman[66036]: 2024-03-26 18:55:41.055841864 +0000 UTC m=+0.415251381 container start ca0f9421c468d7abd974c9aaea97fb818f0edf061f8349b595668f98b9f7d2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-26T18:55:41.376 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:55:41 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l[66051]: starting mds.l at 2024-03-26T18:55:41.376 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:55:41 smithi144 bash[66036]: ca0f9421c468d7abd974c9aaea97fb818f0edf061f8349b595668f98b9f7d2c7 2024-03-26T18:55:41.376 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:55:41 smithi144 systemd[1]: Started Ceph mds.l for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:55:41.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:55:41.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.874+0000 7f2873600640 1 -- 172.21.15.97:0/1786069348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f286c1056b0 msgr2=0x7f286c10bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:41.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.874+0000 7f2873600640 1 --2- 172.21.15.97:0/1786069348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f286c1056b0 0x7f286c10bf40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2864009be0 tx=0x7f286402bef0 comp rx=0 tx=0).stop 2024-03-26T18:55:41.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.874+0000 7f2873600640 1 -- 172.21.15.97:0/1786069348 shutdown_connections 2024-03-26T18:55:41.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.874+0000 7f2873600640 1 --2- 172.21.15.97:0/1786069348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f286c1056b0 0x7f286c10bf40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:41.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.874+0000 7f2873600640 1 --2- 172.21.15.97:0/1786069348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c104cf0 0x7f286c105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:41.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.874+0000 7f2873600640 1 --2- 172.21.15.97:0/1786069348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286c103b00 0x7f286c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:41.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.874+0000 7f2873600640 1 -- 172.21.15.97:0/1786069348 >> 172.21.15.97:0/1786069348 conn(0x7f286c0ff0b0 msgr2=0x7f286c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:41.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.874+0000 7f2873600640 1 -- 172.21.15.97:0/1786069348 shutdown_connections 2024-03-26T18:55:41.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.874+0000 7f2873600640 1 -- 172.21.15.97:0/1786069348 wait complete. 2024-03-26T18:55:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f2873600640 1 Processor -- start 2024-03-26T18:55:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f2873600640 1 -- start start 2024-03-26T18:55:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f2873600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f286c103b00 0x7f286c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f2873600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286c104cf0 0x7f286c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f2873600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c1056b0 0x7f286c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f2873600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10f7b0 con 0x7f286c104cf0 2024-03-26T18:55:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f2873600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10f630 con 0x7f286c103b00 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f2873600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10f930 con 0x7f286c1056b0 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f286be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286c104cf0 0x7f286c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f286be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286c104cf0 0x7f286c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34728/0 (socket says 172.21.15.97:34728) 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f286be00640 1 -- 172.21.15.97:0/3502854268 learned_addr learned my addr 172.21.15.97:0/3502854268 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f2871600640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c1056b0 0x7f286c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.875+0000 7f2870c00640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f286c103b00 0x7f286c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f286be00640 1 -- 172.21.15.97:0/3502854268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c1056b0 msgr2=0x7f286c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f286be00640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c1056b0 0x7f286c1a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f286be00640 1 -- 172.21.15.97:0/3502854268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f286c103b00 msgr2=0x7f286c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f286be00640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f286c103b00 0x7f286c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f286be00640 1 -- 172.21.15.97:0/3502854268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c1a19c0 con 0x7f286c104cf0 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f2871600640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c1056b0 0x7f286c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f2870c00640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f286c103b00 0x7f286c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f286be00640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286c104cf0 0x7f286c19a2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f285800e9c0 tx=0x7f285800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f2869600640 1 -- 172.21.15.97:0/3502854268 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285800cde0 con 0x7f286c104cf0 2024-03-26T18:55:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f2869600640 1 -- 172.21.15.97:0/3502854268 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28580035f0 con 0x7f286c104cf0 2024-03-26T18:55:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f2873600640 1 -- 172.21.15.97:0/3502854268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c1a1cb0 con 0x7f286c104cf0 2024-03-26T18:55:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.876+0000 7f2869600640 1 -- 172.21.15.97:0/3502854268 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858010690 con 0x7f286c104cf0 2024-03-26T18:55:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.877+0000 7f2873600640 1 -- 172.21.15.97:0/3502854268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c1a21b0 con 0x7f286c104cf0 2024-03-26T18:55:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.878+0000 7f2873600640 1 -- 172.21.15.97:0/3502854268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286c10ae20 con 0x7f286c104cf0 2024-03-26T18:55:41.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.880+0000 7f2869600640 1 -- 172.21.15.97:0/3502854268 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f28580037a0 con 0x7f286c104cf0 2024-03-26T18:55:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.881+0000 7f2869600640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28540a2f20 0x7f28540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.882+0000 7f2869600640 1 -- 172.21.15.97:0/3502854268 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f28580cf380 con 0x7f286c104cf0 2024-03-26T18:55:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.882+0000 7f2870c00640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28540a2f20 0x7f28540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.882+0000 7f2870c00640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28540a2f20 0x7f28540a53e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f286c19b280 tx=0x7f2860005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:41.886+0000 7f2869600640 1 -- 172.21.15.97:0/3502854268 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f285809c5b0 con 0x7f286c104cf0 2024-03-26T18:55:42.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.126+0000 7f2873600640 1 -- 172.21.15.97:0/3502854268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f286c062560 con 0x7f286c104cf0 2024-03-26T18:55:42.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.127+0000 7f2869600640 1 -- 172.21.15.97:0/3502854268 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f285809beb0 con 0x7f286c104cf0 2024-03-26T18:55:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:42.120 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-26T18:55:42.120 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-03-26T18:55:42.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.129+0000 7f2873600640 1 -- 172.21.15.97:0/3502854268 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28540a2f20 msgr2=0x7f28540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:42.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.129+0000 7f2873600640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28540a2f20 0x7f28540a53e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f286c19b280 tx=0x7f2860005f90 comp rx=0 tx=0).stop 2024-03-26T18:55:42.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.129+0000 7f2873600640 1 -- 172.21.15.97:0/3502854268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286c104cf0 msgr2=0x7f286c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:42.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.129+0000 7f2873600640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286c104cf0 0x7f286c19a2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f285800e9c0 tx=0x7f285800ee80 comp rx=0 tx=0).stop 2024-03-26T18:55:42.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.130+0000 7f2873600640 1 -- 172.21.15.97:0/3502854268 shutdown_connections 2024-03-26T18:55:42.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.130+0000 7f2873600640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28540a2f20 0x7f28540a53e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.130+0000 7f2873600640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c1056b0 0x7f286c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.130+0000 7f2873600640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f286c104cf0 0x7f286c19a2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.130+0000 7f2873600640 1 --2- 172.21.15.97:0/3502854268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f286c103b00 0x7f286c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.130+0000 7f2873600640 1 -- 172.21.15.97:0/3502854268 >> 172.21.15.97:0/3502854268 conn(0x7f286c0ff0b0 msgr2=0x7f286c107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.130+0000 7f2873600640 1 -- 172.21.15.97:0/3502854268 shutdown_connections 2024-03-26T18:55:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:42.130+0000 7f2873600640 1 -- 172.21.15.97:0/3502854268 wait complete. 2024-03-26T18:55:42.138 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:42 smithi105 ceph-mon[36270]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:42.138 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:42 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.144:6832/3099308706,v1:172.21.15.144:6833/3099308706] up:boot 2024-03-26T18:55:42.138 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:42 smithi105 ceph-mon[36270]: fsmap 1 up:standby 2024-03-26T18:55:42.138 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:42 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:55:42.138 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:42 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:42.138 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:42 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:42.138 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:42 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:42.138 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:42 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:42.138 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:42 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:42.139 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:42 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:42 smithi097 ceph-mon[31955]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:42 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.144:6832/3099308706,v1:172.21.15.144:6833/3099308706] up:boot 2024-03-26T18:55:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:42 smithi097 ceph-mon[31955]: fsmap 1 up:standby 2024-03-26T18:55:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:42 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:55:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:42 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:42.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:42 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:42.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:42 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:42.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:42 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:42.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:42 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:42.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:42 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:42 smithi144 ceph-mon[36053]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:42 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.144:6832/3099308706,v1:172.21.15.144:6833/3099308706] up:boot 2024-03-26T18:55:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:42 smithi144 ceph-mon[36053]: fsmap 1 up:standby 2024-03-26T18:55:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:42 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:55:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:42 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:42 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:42 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:42 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:42 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:42 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:43.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:43 smithi105 ceph-mon[36270]: Deploying daemon mds.k on smithi105 2024-03-26T18:55:43.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:43 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3502854268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:43.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:43 smithi144 ceph-mon[36053]: Deploying daemon mds.k on smithi105 2024-03-26T18:55:43.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:43 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3502854268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:43 smithi097 ceph-mon[31955]: Deploying daemon mds.k on smithi105 2024-03-26T18:55:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:43 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3502854268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:43.710 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 18:55:43 smithi105 systemd[1]: Starting Ceph mds.k for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:55:44.010 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 18:55:43 smithi105 podman[65344]: 2024-03-26 18:55:43.764910601 +0000 UTC m=+0.107774379 container create 5a839fe235117cadc51db139b09ff714318f3f7de9ba59a19d13e5bec6622f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:55:44.010 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 18:55:43 smithi105 podman[65344]: 2024-03-26 18:55:43.674290602 +0000 UTC m=+0.017154371 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:55:44.010 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 18:55:43 smithi105 podman[65344]: 2024-03-26 18:55:43.992802191 +0000 UTC m=+0.335665939 container init 5a839fe235117cadc51db139b09ff714318f3f7de9ba59a19d13e5bec6622f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-26T18:55:44.011 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 18:55:43 smithi105 podman[65344]: 2024-03-26 18:55:43.995416173 +0000 UTC m=+0.338279941 container start 5a839fe235117cadc51db139b09ff714318f3f7de9ba59a19d13e5bec6622f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-26T18:55:44.011 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 18:55:44 smithi105 ceph-mds[65363]: -- 172.21.15.105:0/3033387397 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654b878da40 con 0x5654b8760c00 2024-03-26T18:55:44.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:55:44.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.362+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3550320138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c08104cf0 msgr2=0x7f9c08105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:44.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.362+0000 7f9c0fa00640 1 --2- 172.21.15.97:0/3550320138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c08104cf0 0x7f9c08105170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009900 tx=0x7f9bf402bc30 comp rx=0 tx=0).stop 2024-03-26T18:55:44.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.362+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3550320138 shutdown_connections 2024-03-26T18:55:44.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.362+0000 7f9c0fa00640 1 --2- 172.21.15.97:0/3550320138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c081056b0 0x7f9c0810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:44.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.362+0000 7f9c0fa00640 1 --2- 172.21.15.97:0/3550320138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c08104cf0 0x7f9c08105170 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:44.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.362+0000 7f9c0fa00640 1 --2- 172.21.15.97:0/3550320138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c08103b00 0x7f9c08103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:44.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.362+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3550320138 >> 172.21.15.97:0/3550320138 conn(0x7f9c080ff0b0 msgr2=0x7f9c081014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:44.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.362+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3550320138 shutdown_connections 2024-03-26T18:55:44.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.362+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3550320138 wait complete. 2024-03-26T18:55:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.363+0000 7f9c0fa00640 1 Processor -- start 2024-03-26T18:55:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0fa00640 1 -- start start 2024-03-26T18:55:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c08103b00 0x7f9c08199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0fa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c08104cf0 0x7f9c0819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0fa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081056b0 0x7f9c081a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0810f7d0 con 0x7f9c08103b00 2024-03-26T18:55:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0fa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0810f650 con 0x7f9c08104cf0 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0fa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0810f950 con 0x7f9c081056b0 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c08103b00 0x7f9c08199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c08104cf0 0x7f9c0819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c08103b00 0x7f9c08199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34752/0 (socket says 172.21.15.97:34752) 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0d000640 1 -- 172.21.15.97:0/3081477976 learned_addr learned my addr 172.21.15.97:0/3081477976 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c0da00640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081056b0 0x7f9c081a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c07e00640 1 -- 172.21.15.97:0/3081477976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081056b0 msgr2=0x7f9c081a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.364+0000 7f9c07e00640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081056b0 0x7f9c081a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.365+0000 7f9c07e00640 1 -- 172.21.15.97:0/3081477976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c08103b00 msgr2=0x7f9c08199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.365+0000 7f9c07e00640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c08103b00 0x7f9c08199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.365+0000 7f9c07e00640 1 -- 172.21.15.97:0/3081477976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c081a1a40 con 0x7f9c08104cf0 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.365+0000 7f9c07e00640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c08104cf0 0x7f9c0819a360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4036040 tx=0x7f9bf402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.365+0000 7f9c05600640 1 -- 172.21.15.97:0/3081477976 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4043020 con 0x7f9c08104cf0 2024-03-26T18:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.365+0000 7f9c05600640 1 -- 172.21.15.97:0/3081477976 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf40054e0 con 0x7f9c08104cf0 2024-03-26T18:55:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.365+0000 7f9c05600640 1 -- 172.21.15.97:0/3081477976 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4003ca0 con 0x7f9c08104cf0 2024-03-26T18:55:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.365+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3081477976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c081a1cd0 con 0x7f9c08104cf0 2024-03-26T18:55:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.365+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3081477976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c081a21a0 con 0x7f9c08104cf0 2024-03-26T18:55:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.366+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3081477976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0810ae20 con 0x7f9c08104cf0 2024-03-26T18:55:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.369+0000 7f9c05600640 1 -- 172.21.15.97:0/3081477976 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f9bf4046020 con 0x7f9c08104cf0 2024-03-26T18:55:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.370+0000 7f9c05600640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf00a2f20 0x7f9bf00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.370+0000 7f9c05600640 1 -- 172.21.15.97:0/3081477976 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9bf40f0ff0 con 0x7f9c08104cf0 2024-03-26T18:55:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.370+0000 7f9c0d000640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf00a2f20 0x7f9bf00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:44.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.371+0000 7f9c0d000640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf00a2f20 0x7f9bf00a53e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc005fd0 tx=0x7f9bfc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:44.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.374+0000 7f9c05600640 1 -- 172.21.15.97:0/3081477976 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9bf404d020 con 0x7f9c08104cf0 2024-03-26T18:55:44.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:44 smithi144 ceph-mon[36053]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:44.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:44 smithi105 ceph-mon[36270]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:44.460 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 18:55:44 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k[65359]: starting mds.k at 2024-03-26T18:55:44.460 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 18:55:44 smithi105 bash[65344]: 5a839fe235117cadc51db139b09ff714318f3f7de9ba59a19d13e5bec6622f01 2024-03-26T18:55:44.460 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 18:55:44 smithi105 systemd[1]: Started Ceph mds.k for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:55:44.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:44 smithi097 ceph-mon[31955]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:44.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.613+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3081477976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c0802d240 con 0x7f9c08104cf0 2024-03-26T18:55:44.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.614+0000 7f9c05600640 1 -- 172.21.15.97:0/3081477976 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f9bf40be260 con 0x7f9c08104cf0 2024-03-26T18:55:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:44.608 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-26T18:55:44.608 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2024-03-26T18:55:44.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.617+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3081477976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf00a2f20 msgr2=0x7f9bf00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:44.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.617+0000 7f9c0fa00640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf00a2f20 0x7f9bf00a53e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc005fd0 tx=0x7f9bfc00b410 comp rx=0 tx=0).stop 2024-03-26T18:55:44.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.617+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3081477976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c08104cf0 msgr2=0x7f9c0819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:44.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.617+0000 7f9c0fa00640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c08104cf0 0x7f9c0819a360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4036040 tx=0x7f9bf402c6c0 comp rx=0 tx=0).stop 2024-03-26T18:55:44.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.617+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3081477976 shutdown_connections 2024-03-26T18:55:44.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.617+0000 7f9c0fa00640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf00a2f20 0x7f9bf00a53e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:44.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.617+0000 7f9c0fa00640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081056b0 0x7f9c081a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:44.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.617+0000 7f9c0fa00640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c08104cf0 0x7f9c0819a360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:44.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.618+0000 7f9c0fa00640 1 --2- 172.21.15.97:0/3081477976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c08103b00 0x7f9c08199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:44.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.618+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3081477976 >> 172.21.15.97:0/3081477976 conn(0x7f9c080ff0b0 msgr2=0x7f9c08107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:44.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.618+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3081477976 shutdown_connections 2024-03-26T18:55:44.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:44.618+0000 7f9c0fa00640 1 -- 172.21.15.97:0/3081477976 wait complete. 2024-03-26T18:55:45.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:45 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6832/254387341,v1:172.21.15.105:6833/254387341] up:boot 2024-03-26T18:55:45.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:45 smithi144 ceph-mon[36053]: fsmap 2 up:standby 2024-03-26T18:55:45.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:55:45.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:45.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:45.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:45.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:45.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:45.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:45 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:45.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:45 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3081477976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:45.405 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:45 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6832/254387341,v1:172.21.15.105:6833/254387341] up:boot 2024-03-26T18:55:45.405 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:45 smithi105 ceph-mon[36270]: fsmap 2 up:standby 2024-03-26T18:55:45.405 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:55:45.405 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:45.406 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:45.406 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:45.406 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:45.406 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:45.406 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:45 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:45.406 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:45 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3081477976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:45.467 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:45 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6832/254387341,v1:172.21.15.105:6833/254387341] up:boot 2024-03-26T18:55:45.467 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:45 smithi097 ceph-mon[31955]: fsmap 2 up:standby 2024-03-26T18:55:45.467 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:55:45.467 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:45.467 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:45.467 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:45.467 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:45.467 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:45.467 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:45 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:45.468 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:45 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3081477976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:46.334 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:46 smithi097 ceph-mon[31955]: Deploying daemon mds.j on smithi097 2024-03-26T18:55:46.335 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:46 smithi097 ceph-mon[31955]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:46.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:46 smithi144 ceph-mon[36053]: Deploying daemon mds.j on smithi097 2024-03-26T18:55:46.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:46 smithi144 ceph-mon[36053]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:46 smithi105 ceph-mon[36270]: Deploying daemon mds.j on smithi097 2024-03-26T18:55:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:46 smithi105 ceph-mon[36270]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:46.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:55:46.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.954+0000 7f451aa00640 1 -- 172.21.15.97:0/966044215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4514104cd0 msgr2=0x7f4514105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.954+0000 7f451aa00640 1 --2- 172.21.15.97:0/966044215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4514104cd0 0x7f4514105150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4500009900 tx=0x7f450002bc30 comp rx=0 tx=0).stop 2024-03-26T18:55:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.954+0000 7f451aa00640 1 -- 172.21.15.97:0/966044215 shutdown_connections 2024-03-26T18:55:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.954+0000 7f451aa00640 1 --2- 172.21.15.97:0/966044215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514105690 0x7f451410bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.954+0000 7f451aa00640 1 --2- 172.21.15.97:0/966044215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4514104cd0 0x7f4514105150 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.954+0000 7f451aa00640 1 --2- 172.21.15.97:0/966044215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514103ae0 0x7f4514103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.954+0000 7f451aa00640 1 -- 172.21.15.97:0/966044215 >> 172.21.15.97:0/966044215 conn(0x7f45140ff070 msgr2=0x7f4514101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.955+0000 7f451aa00640 1 -- 172.21.15.97:0/966044215 shutdown_connections 2024-03-26T18:55:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.955+0000 7f451aa00640 1 -- 172.21.15.97:0/966044215 wait complete. 2024-03-26T18:55:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.956+0000 7f451aa00640 1 Processor -- start 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.956+0000 7f451aa00640 1 -- start start 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.956+0000 7f451aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514103ae0 0x7f4514199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.956+0000 7f451aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514104cd0 0x7f451419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.956+0000 7f451aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4514105690 0x7f45141a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.956+0000 7f451aa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f451410f7b0 con 0x7f4514104cd0 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.956+0000 7f451aa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f451410f630 con 0x7f4514105690 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.956+0000 7f451aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f451410f930 con 0x7f4514103ae0 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.956+0000 7f4513400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514104cd0 0x7f451419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4518a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4514105690 0x7f45141a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4513e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514103ae0 0x7f4514199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4518a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4514105690 0x7f45141a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:50800/0 (socket says 172.21.15.97:50800) 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4518a00640 1 -- 172.21.15.97:0/1709592452 learned_addr learned my addr 172.21.15.97:0/1709592452 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4518a00640 1 -- 172.21.15.97:0/1709592452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514103ae0 msgr2=0x7f4514199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4518a00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514103ae0 0x7f4514199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4518a00640 1 -- 172.21.15.97:0/1709592452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514104cd0 msgr2=0x7f451419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4518a00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514104cd0 0x7f451419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4518a00640 1 -- 172.21.15.97:0/1709592452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45141a1a10 con 0x7f4514105690 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4513400640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514104cd0 0x7f451419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.957+0000 7f4518a00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4514105690 0x7f45141a13a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f450c00eb10 tx=0x7f450c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.958+0000 7f4510c00640 1 -- 172.21.15.97:0/1709592452 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c004b20 con 0x7f4514105690 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.958+0000 7f4510c00640 1 -- 172.21.15.97:0/1709592452 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f450c004cc0 con 0x7f4514105690 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.958+0000 7f451aa00640 1 -- 172.21.15.97:0/1709592452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45141a1d00 con 0x7f4514105690 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.958+0000 7f4510c00640 1 -- 172.21.15.97:0/1709592452 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c010760 con 0x7f4514105690 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.958+0000 7f451aa00640 1 -- 172.21.15.97:0/1709592452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45141a21d0 con 0x7f4514105690 2024-03-26T18:55:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.959+0000 7f451aa00640 1 -- 172.21.15.97:0/1709592452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4514068b00 con 0x7f4514105690 2024-03-26T18:55:46.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.961+0000 7f4510c00640 1 -- 172.21.15.97:0/1709592452 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f450c044b40 con 0x7f4514105690 2024-03-26T18:55:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.962+0000 7f4510c00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44f80a2f20 0x7f44f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.963+0000 7f4510c00640 1 -- 172.21.15.97:0/1709592452 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f450c0cf540 con 0x7f4514105690 2024-03-26T18:55:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.963+0000 7f4513e00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44f80a2f20 0x7f44f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.963+0000 7f4513e00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44f80a2f20 0x7f44f80a53e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4514104b30 tx=0x7f4504005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:46.964+0000 7f4510c00640 1 -- 172.21.15.97:0/1709592452 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f450c09c7f0 con 0x7f4514105690 2024-03-26T18:55:47.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.205+0000 7f451aa00640 1 -- 172.21.15.97:0/1709592452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f451410ae00 con 0x7f4514105690 2024-03-26T18:55:47.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.206+0000 7f4510c00640 1 -- 172.21.15.97:0/1709592452 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f450c09c0f0 con 0x7f4514105690 2024-03-26T18:55:47.200 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:47.200 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-26T18:55:47.200 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2024-03-26T18:55:47.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 -- 172.21.15.97:0/1709592452 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44f80a2f20 msgr2=0x7f44f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:47.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44f80a2f20 0x7f44f80a53e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4514104b30 tx=0x7f4504005f90 comp rx=0 tx=0).stop 2024-03-26T18:55:47.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 -- 172.21.15.97:0/1709592452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4514105690 msgr2=0x7f45141a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:47.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4514105690 0x7f45141a13a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f450c00eb10 tx=0x7f450c00c2d0 comp rx=0 tx=0).stop 2024-03-26T18:55:47.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 -- 172.21.15.97:0/1709592452 shutdown_connections 2024-03-26T18:55:47.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44f80a2f20 0x7f44f80a53e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:47.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4514105690 0x7f45141a13a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:47.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514104cd0 0x7f451419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:47.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 --2- 172.21.15.97:0/1709592452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514103ae0 0x7f4514199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:47.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 -- 172.21.15.97:0/1709592452 >> 172.21.15.97:0/1709592452 conn(0x7f45140ff070 msgr2=0x7f4514107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:47.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 -- 172.21.15.97:0/1709592452 shutdown_connections 2024-03-26T18:55:47.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:47.209+0000 7f451aa00640 1 -- 172.21.15.97:0/1709592452 wait complete. 2024-03-26T18:55:47.795 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 18:55:47 smithi097 podman[69830]: 2024-03-26 18:55:47.530026563 +0000 UTC m=+0.150596774 container create 6365380a10d2f9e6f36d3b9213a2bed770cbfdf31c630d2c100a9389ffbb98ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-j, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:55:47.795 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 18:55:47 smithi097 podman[69830]: 2024-03-26 18:55:47.801109153 +0000 UTC m=+0.421679416 container init 6365380a10d2f9e6f36d3b9213a2bed770cbfdf31c630d2c100a9389ffbb98ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:55:48.103 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 18:55:47 smithi097 podman[69830]: 2024-03-26 18:55:47.811303209 +0000 UTC m=+0.431873399 container start 6365380a10d2f9e6f36d3b9213a2bed770cbfdf31c630d2c100a9389ffbb98ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-j, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-26T18:55:48.103 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 18:55:47 smithi097 ceph-mds[69850]: -- 172.21.15.97:0/1347568125 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56188c5e7a40 con 0x56188c5bb000 2024-03-26T18:55:48.103 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 18:55:47 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-j[69846]: starting mds.j at 2024-03-26T18:55:48.103 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 18:55:47 smithi097 bash[69830]: 6365380a10d2f9e6f36d3b9213a2bed770cbfdf31c630d2c100a9389ffbb98ed 2024-03-26T18:55:48.104 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 18:55:47 smithi097 systemd[1]: Started Ceph mds.j for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:55:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:48 smithi105 ceph-mon[36270]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:48 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1709592452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:48 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:48 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:48.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:48 smithi097 ceph-mon[31955]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:48.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:48 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1709592452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:48.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:48 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:48.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:48 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:48.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:48 smithi144 ceph-mon[36053]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:48.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:48 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1709592452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:48.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:48 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:48.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:48 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:49.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:55:49.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.447+0000 7f034e800640 1 -- 172.21.15.97:0/931614955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0348104cf0 msgr2=0x7f0348105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:49.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.447+0000 7f034e800640 1 --2- 172.21.15.97:0/931614955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0348104cf0 0x7f0348105170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0334009900 tx=0x7f033402bc30 comp rx=0 tx=0).stop 2024-03-26T18:55:49.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.448+0000 7f034e800640 1 -- 172.21.15.97:0/931614955 shutdown_connections 2024-03-26T18:55:49.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.448+0000 7f034e800640 1 --2- 172.21.15.97:0/931614955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03481056b0 0x7f034810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.448+0000 7f034e800640 1 --2- 172.21.15.97:0/931614955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0348104cf0 0x7f0348105170 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.448+0000 7f034e800640 1 --2- 172.21.15.97:0/931614955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0348103b00 0x7f0348103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.448+0000 7f034e800640 1 -- 172.21.15.97:0/931614955 >> 172.21.15.97:0/931614955 conn(0x7f03480ff070 msgr2=0x7f03481014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:49.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.448+0000 7f034e800640 1 -- 172.21.15.97:0/931614955 shutdown_connections 2024-03-26T18:55:49.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.448+0000 7f034e800640 1 -- 172.21.15.97:0/931614955 wait complete. 2024-03-26T18:55:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f034e800640 1 Processor -- start 2024-03-26T18:55:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f034e800640 1 -- start start 2024-03-26T18:55:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f034e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0348103b00 0x7f0348199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f034e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348104cf0 0x7f034819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f034e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03481056b0 0x7f03481a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f034e800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f034810f780 con 0x7f03481056b0 2024-03-26T18:55:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f034e800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f034810f600 con 0x7f0348104cf0 2024-03-26T18:55:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f034e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f034810f900 con 0x7f0348103b00 2024-03-26T18:55:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f0347400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0348103b00 0x7f0348199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f0347400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0348103b00 0x7f0348199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44438/0 (socket says 172.21.15.97:44438) 2024-03-26T18:55:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f0347400640 1 -- 172.21.15.97:0/4254676776 learned_addr learned my addr 172.21.15.97:0/4254676776 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f0347e00640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03481056b0 0x7f03481a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.449+0000 7f0346a00640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348104cf0 0x7f034819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.450+0000 7f0347400640 1 -- 172.21.15.97:0/4254676776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348104cf0 msgr2=0x7f034819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.450+0000 7f0347400640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348104cf0 0x7f034819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.450+0000 7f0347400640 1 -- 172.21.15.97:0/4254676776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03481056b0 msgr2=0x7f03481a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.450+0000 7f0347400640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03481056b0 0x7f03481a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.450+0000 7f0347400640 1 -- 172.21.15.97:0/4254676776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03481a1a70 con 0x7f0348103b00 2024-03-26T18:55:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.450+0000 7f0347400640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0348103b00 0x7f0348199e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f033800e9c0 tx=0x7f033800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.450+0000 7f033fe00640 1 -- 172.21.15.97:0/4254676776 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033800ccc0 con 0x7f0348103b00 2024-03-26T18:55:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.451+0000 7f034e800640 1 -- 172.21.15.97:0/4254676776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03481a1d60 con 0x7f0348103b00 2024-03-26T18:55:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.451+0000 7f033fe00640 1 -- 172.21.15.97:0/4254676776 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03380035f0 con 0x7f0348103b00 2024-03-26T18:55:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.451+0000 7f033fe00640 1 -- 172.21.15.97:0/4254676776 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033800ce60 con 0x7f0348103b00 2024-03-26T18:55:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.451+0000 7f034e800640 1 -- 172.21.15.97:0/4254676776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03481a21d0 con 0x7f0348103b00 2024-03-26T18:55:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.452+0000 7f034e800640 1 -- 172.21.15.97:0/4254676776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034810aea0 con 0x7f0348103b00 2024-03-26T18:55:49.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.454+0000 7f033fe00640 1 -- 172.21.15.97:0/4254676776 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f03380037a0 con 0x7f0348103b00 2024-03-26T18:55:49.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.455+0000 7f033fe00640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f032c0a2f20 0x7f032c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:49.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.455+0000 7f033fe00640 1 -- 172.21.15.97:0/4254676776 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f03380cf340 con 0x7f0348103b00 2024-03-26T18:55:49.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.455+0000 7f0346a00640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f032c0a2f20 0x7f032c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:49.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.456+0000 7f0346a00640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f032c0a2f20 0x7f032c0a53e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f033402c7b0 tx=0x7f0334005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:49.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.459+0000 7f033fe00640 1 -- 172.21.15.97:0/4254676776 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f033809c570 con 0x7f0348103b00 2024-03-26T18:55:49.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:49.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:49.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:49.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:49 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.97:6834/1485859624,v1:172.21.15.97:6835/1485859624] up:boot 2024-03-26T18:55:49.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:49 smithi105 ceph-mon[36270]: fsmap 3 up:standby 2024-03-26T18:55:49.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T18:55:49.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:49 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:49.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:49 smithi105 ceph-mon[36270]: Deploying daemon mds.d on smithi097 2024-03-26T18:55:49.482 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:49.482 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:49.482 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:49.482 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:49 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.97:6834/1485859624,v1:172.21.15.97:6835/1485859624] up:boot 2024-03-26T18:55:49.482 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:49 smithi097 ceph-mon[31955]: fsmap 3 up:standby 2024-03-26T18:55:49.482 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T18:55:49.483 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:49 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:49.483 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:49 smithi097 ceph-mon[31955]: Deploying daemon mds.d on smithi097 2024-03-26T18:55:49.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:49.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:49.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:49.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:49 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.97:6834/1485859624,v1:172.21.15.97:6835/1485859624] up:boot 2024-03-26T18:55:49.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:49 smithi144 ceph-mon[36053]: fsmap 3 up:standby 2024-03-26T18:55:49.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T18:55:49.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:49 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:49.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:49 smithi144 ceph-mon[36053]: Deploying daemon mds.d on smithi097 2024-03-26T18:55:49.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.703+0000 7f034e800640 1 -- 172.21.15.97:0/4254676776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f034802d240 con 0x7f0348103b00 2024-03-26T18:55:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.705+0000 7f033fe00640 1 -- 172.21.15.97:0/4254676776 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f033809be70 con 0x7f0348103b00 2024-03-26T18:55:49.698 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:49.698 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-26T18:55:49.698 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 4 2024-03-26T18:55:49.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 -- 172.21.15.97:0/4254676776 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f032c0a2f20 msgr2=0x7f032c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f032c0a2f20 0x7f032c0a53e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f033402c7b0 tx=0x7f0334005fd0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 -- 172.21.15.97:0/4254676776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0348103b00 msgr2=0x7f0348199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0348103b00 0x7f0348199e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f033800e9c0 tx=0x7f033800edc0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 -- 172.21.15.97:0/4254676776 shutdown_connections 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f032c0a2f20 0x7f032c0a53e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03481056b0 0x7f03481a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348104cf0 0x7f034819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 --2- 172.21.15.97:0/4254676776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0348103b00 0x7f0348199e50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 -- 172.21.15.97:0/4254676776 >> 172.21.15.97:0/4254676776 conn(0x7f03480ff070 msgr2=0x7f0348108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 -- 172.21.15.97:0/4254676776 shutdown_connections 2024-03-26T18:55:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:49.707+0000 7f034e800640 1 -- 172.21.15.97:0/4254676776 wait complete. 2024-03-26T18:55:50.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:50 smithi097 ceph-mon[31955]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:50.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:50 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/4254676776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:50.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:50 smithi105 ceph-mon[36270]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:50.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:50 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/4254676776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:50.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:50 smithi144 ceph-mon[36053]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:50.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:50 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/4254676776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:51.204 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 18:55:51 smithi097 systemd[1]: Starting Ceph mds.d for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:55:51.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:55:51.803 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 18:55:51 smithi097 podman[70177]: 2024-03-26 18:55:51.464202275 +0000 UTC m=+0.032056104 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:55:51.803 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 18:55:51 smithi097 podman[70177]: 2024-03-26 18:55:51.595793222 +0000 UTC m=+0.163647084 container create 3c4c8e413743621cb19e1adc1e68913174b6651b8d0a07eb8b2bbdcdf55b1dab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:55:51.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.942+0000 7f235dc00640 1 -- 172.21.15.97:0/2106440763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2358102190 msgr2=0x7f2358104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.942+0000 7f235dc00640 1 --2- 172.21.15.97:0/2106440763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2358102190 0x7f2358104580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2348009900 tx=0x7f234802bc30 comp rx=0 tx=0).stop 2024-03-26T18:55:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.943+0000 7f235dc00640 1 -- 172.21.15.97:0/2106440763 shutdown_connections 2024-03-26T18:55:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.943+0000 7f235dc00640 1 --2- 172.21.15.97:0/2106440763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23581073f0 0x7f2358109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.943+0000 7f235dc00640 1 --2- 172.21.15.97:0/2106440763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2358104ac0 0x7f2358106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.943+0000 7f235dc00640 1 --2- 172.21.15.97:0/2106440763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2358102190 0x7f2358104580 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.943+0000 7f235dc00640 1 -- 172.21.15.97:0/2106440763 >> 172.21.15.97:0/2106440763 conn(0x7f23580fbba0 msgr2=0x7f23580fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.943+0000 7f235dc00640 1 -- 172.21.15.97:0/2106440763 shutdown_connections 2024-03-26T18:55:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.943+0000 7f235dc00640 1 -- 172.21.15.97:0/2106440763 wait complete. 2024-03-26T18:55:51.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f235dc00640 1 Processor -- start 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f235dc00640 1 -- start start 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f235dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2358102190 0x7f2358199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f235dc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2358104ac0 0x7f235819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f235dc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23581073f0 0x7f23581a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f235dc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f235810d400 con 0x7f2358102190 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f235dc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f235810d280 con 0x7f23581073f0 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f235dc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f235810d580 con 0x7f2358104ac0 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f2357e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2358102190 0x7f2358199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f2357e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2358102190 0x7f2358199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57852/0 (socket says 172.21.15.97:57852) 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.944+0000 7f2357e00640 1 -- 172.21.15.97:0/3754272813 learned_addr learned my addr 172.21.15.97:0/3754272813 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.945+0000 7f235d200640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23581073f0 0x7f23581a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.945+0000 7f2357400640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2358104ac0 0x7f235819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.945+0000 7f2357e00640 1 -- 172.21.15.97:0/3754272813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2358104ac0 msgr2=0x7f235819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.945+0000 7f2357e00640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2358104ac0 0x7f235819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.945+0000 7f2357e00640 1 -- 172.21.15.97:0/3754272813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23581073f0 msgr2=0x7f23581a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:51.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.945+0000 7f2357e00640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23581073f0 0x7f23581a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:51.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.945+0000 7f2357e00640 1 -- 172.21.15.97:0/3754272813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23581a1880 con 0x7f2358102190 2024-03-26T18:55:51.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.945+0000 7f2357400640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2358104ac0 0x7f235819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:55:51.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.945+0000 7f2357e00640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2358102190 0x7f2358199c70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2348009230 tx=0x7f2348005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:51.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.946+0000 7f2354c00640 1 -- 172.21.15.97:0/3754272813 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348041070 con 0x7f2358102190 2024-03-26T18:55:51.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.946+0000 7f235dc00640 1 -- 172.21.15.97:0/3754272813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23581a1b10 con 0x7f2358102190 2024-03-26T18:55:51.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.946+0000 7f2354c00640 1 -- 172.21.15.97:0/3754272813 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23480044d0 con 0x7f2358102190 2024-03-26T18:55:51.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.946+0000 7f2354c00640 1 -- 172.21.15.97:0/3754272813 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348034440 con 0x7f2358102190 2024-03-26T18:55:51.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.946+0000 7f235dc00640 1 -- 172.21.15.97:0/3754272813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23581a2000 con 0x7f2358102190 2024-03-26T18:55:51.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.947+0000 7f235dc00640 1 -- 172.21.15.97:0/3754272813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2358108ee0 con 0x7f2358102190 2024-03-26T18:55:51.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.949+0000 7f2354c00640 1 -- 172.21.15.97:0/3754272813 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f2348044070 con 0x7f2358102190 2024-03-26T18:55:51.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.951+0000 7f2354c00640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f233c0a2f20 0x7f233c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.952+0000 7f2354c00640 1 -- 172.21.15.97:0/3754272813 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f23480f0d80 con 0x7f2358102190 2024-03-26T18:55:51.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.952+0000 7f2357400640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f233c0a2f20 0x7f233c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:51.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.952+0000 7f2357400640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f233c0a2f20 0x7f233c0a53e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2358069580 tx=0x7f2344005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:51.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:51.955+0000 7f2354c00640 1 -- 172.21.15.97:0/3754272813 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f23480bdfb0 con 0x7f2358102190 2024-03-26T18:55:52.058 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 18:55:51 smithi097 podman[70177]: 2024-03-26 18:55:51.916296584 +0000 UTC m=+0.484150472 container init 3c4c8e413743621cb19e1adc1e68913174b6651b8d0a07eb8b2bbdcdf55b1dab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T18:55:52.058 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 18:55:51 smithi097 podman[70177]: 2024-03-26 18:55:51.92582023 +0000 UTC m=+0.493674125 container start 3c4c8e413743621cb19e1adc1e68913174b6651b8d0a07eb8b2bbdcdf55b1dab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-26T18:55:52.058 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 18:55:51 smithi097 ceph-mds[70216]: -- 172.21.15.97:0/1476801192 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e69858fa40 con 0x55e698562800 2024-03-26T18:55:52.058 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 18:55:51 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d[70193]: starting mds.d at 2024-03-26T18:55:52.059 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 18:55:52 smithi097 bash[70177]: 3c4c8e413743621cb19e1adc1e68913174b6651b8d0a07eb8b2bbdcdf55b1dab 2024-03-26T18:55:52.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.208+0000 7f235dc00640 1 -- 172.21.15.97:0/3754272813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f235802d240 con 0x7f2358102190 2024-03-26T18:55:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.210+0000 7f2354c00640 1 -- 172.21.15.97:0/3754272813 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f23480bd8b0 con 0x7f2358102190 2024-03-26T18:55:52.202 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:52.203 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-03-26T18:55:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 5 2024-03-26T18:55:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.212+0000 7f235dc00640 1 -- 172.21.15.97:0/3754272813 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f233c0a2f20 msgr2=0x7f233c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.212+0000 7f235dc00640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f233c0a2f20 0x7f233c0a53e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2358069580 tx=0x7f2344005f90 comp rx=0 tx=0).stop 2024-03-26T18:55:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.212+0000 7f235dc00640 1 -- 172.21.15.97:0/3754272813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2358102190 msgr2=0x7f2358199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.212+0000 7f235dc00640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2358102190 0x7f2358199c70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2348009230 tx=0x7f2348005d50 comp rx=0 tx=0).stop 2024-03-26T18:55:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.213+0000 7f235dc00640 1 -- 172.21.15.97:0/3754272813 shutdown_connections 2024-03-26T18:55:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.213+0000 7f235dc00640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f233c0a2f20 0x7f233c0a53e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.213+0000 7f235dc00640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23581073f0 0x7f23581a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.213+0000 7f235dc00640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2358104ac0 0x7f235819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.213+0000 7f235dc00640 1 --2- 172.21.15.97:0/3754272813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2358102190 0x7f2358199c70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.213+0000 7f235dc00640 1 -- 172.21.15.97:0/3754272813 >> 172.21.15.97:0/3754272813 conn(0x7f23580fbba0 msgr2=0x7f2358101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.213+0000 7f235dc00640 1 -- 172.21.15.97:0/3754272813 shutdown_connections 2024-03-26T18:55:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:52.213+0000 7f235dc00640 1 -- 172.21.15.97:0/3754272813 wait complete. 2024-03-26T18:55:52.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:52 smithi097 ceph-mon[31955]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:52.353 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 18:55:52 smithi097 systemd[1]: Started Ceph mds.d for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:55:52.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:52 smithi105 ceph-mon[36270]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:52.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:52 smithi144 ceph-mon[36053]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:53.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:53 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] up:boot 2024-03-26T18:55:53.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:53 smithi105 ceph-mon[36270]: fsmap 4 up:standby 2024-03-26T18:55:53.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T18:55:53.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:53.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:53.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:53 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3754272813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:53.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:53.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:53.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:53.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:53 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:53.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:53 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] up:boot 2024-03-26T18:55:53.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:53 smithi097 ceph-mon[31955]: fsmap 4 up:standby 2024-03-26T18:55:53.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T18:55:53.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:53.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:53.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:53 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3754272813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:53.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:53.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:53.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:53.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:53 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:53.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:53 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] up:boot 2024-03-26T18:55:53.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:53 smithi144 ceph-mon[36053]: fsmap 4 up:standby 2024-03-26T18:55:53.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T18:55:53.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:53.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:53.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:53 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3754272813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:53.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:53.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:53.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:53.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:53 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:54.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:55:54.333 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:54 smithi105 ceph-mon[36270]: Deploying daemon mds.e on smithi105 2024-03-26T18:55:54.333 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:54 smithi105 ceph-mon[36270]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:54.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.455+0000 7f0da1a00640 1 -- 172.21.15.97:0/143908789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d9c103b00 msgr2=0x7f0d9c103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:54.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.455+0000 7f0da1a00640 1 --2- 172.21.15.97:0/143908789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d9c103b00 0x7f0d9c103f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009900 tx=0x7f0d8c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:55:54.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.455+0000 7f0da1a00640 1 -- 172.21.15.97:0/143908789 shutdown_connections 2024-03-26T18:55:54.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.455+0000 7f0da1a00640 1 --2- 172.21.15.97:0/143908789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c1056b0 0x7f0d9c10bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:54.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.455+0000 7f0da1a00640 1 --2- 172.21.15.97:0/143908789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c104cf0 0x7f0d9c105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:54.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.455+0000 7f0da1a00640 1 --2- 172.21.15.97:0/143908789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d9c103b00 0x7f0d9c103f20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:54.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.455+0000 7f0da1a00640 1 -- 172.21.15.97:0/143908789 >> 172.21.15.97:0/143908789 conn(0x7f0d9c0ff070 msgr2=0x7f0d9c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:54.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.455+0000 7f0da1a00640 1 -- 172.21.15.97:0/143908789 shutdown_connections 2024-03-26T18:55:54.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.455+0000 7f0da1a00640 1 -- 172.21.15.97:0/143908789 wait complete. 2024-03-26T18:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.456+0000 7f0da1a00640 1 Processor -- start 2024-03-26T18:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.456+0000 7f0da1a00640 1 -- start start 2024-03-26T18:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0da1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d9c103b00 0x7f0d9c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0da1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c104cf0 0x7f0d9c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0da1a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c1056b0 0x7f0d9c1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0da1a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10f7c0 con 0x7f0d9c104cf0 2024-03-26T18:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0da1a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10f640 con 0x7f0d9c103b00 2024-03-26T18:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0da1a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10f940 con 0x7f0d9c1056b0 2024-03-26T18:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0d9aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d9c103b00 0x7f0d9c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0d9aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d9c103b00 0x7f0d9c199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:43986/0 (socket says 172.21.15.97:43986) 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0d93e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c104cf0 0x7f0d9c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0d9aa00640 1 -- 172.21.15.97:0/1909813670 learned_addr learned my addr 172.21.15.97:0/1909813670 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0d9b400640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c1056b0 0x7f0d9c1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0d9aa00640 1 -- 172.21.15.97:0/1909813670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c1056b0 msgr2=0x7f0d9c1a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0d9aa00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c1056b0 0x7f0d9c1a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0d9aa00640 1 -- 172.21.15.97:0/1909813670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c104cf0 msgr2=0x7f0d9c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0d9aa00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c104cf0 0x7f0d9c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.457+0000 7f0d9aa00640 1 -- 172.21.15.97:0/1909813670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c1a1a80 con 0x7f0d9c103b00 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.458+0000 7f0d9aa00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d9c103b00 0x7f0d9c199e60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c000c00 tx=0x7f0d8c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.458+0000 7f0d93400640 1 -- 172.21.15.97:0/1909813670 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c043020 con 0x7f0d9c103b00 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.458+0000 7f0d93400640 1 -- 172.21.15.97:0/1909813670 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d8c0054e0 con 0x7f0d9c103b00 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.458+0000 7f0d93400640 1 -- 172.21.15.97:0/1909813670 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c003c40 con 0x7f0d9c103b00 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.458+0000 7f0da1a00640 1 -- 172.21.15.97:0/1909813670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c1a1d10 con 0x7f0d9c103b00 2024-03-26T18:55:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.458+0000 7f0da1a00640 1 -- 172.21.15.97:0/1909813670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1a21b0 con 0x7f0d9c103b00 2024-03-26T18:55:54.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.459+0000 7f0da1a00640 1 -- 172.21.15.97:0/1909813670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c10aea0 con 0x7f0d9c103b00 2024-03-26T18:55:54.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.461+0000 7f0d93400640 1 -- 172.21.15.97:0/1909813670 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f0d8c046020 con 0x7f0d9c103b00 2024-03-26T18:55:54.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.462+0000 7f0d93400640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d800a2fa0 0x7f0d800a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.462+0000 7f0d93400640 1 -- 172.21.15.97:0/1909813670 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0d8c0f0ff0 con 0x7f0d9c103b00 2024-03-26T18:55:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.462+0000 7f0d93e00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d800a2fa0 0x7f0d800a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.463+0000 7f0d93e00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d800a2fa0 0x7f0d800a5460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c19b370 tx=0x7f0d84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:54.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.466+0000 7f0d93400640 1 -- 172.21.15.97:0/1909813670 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0d8c04d020 con 0x7f0d9c103b00 2024-03-26T18:55:54.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:54 smithi097 ceph-mon[31955]: Deploying daemon mds.e on smithi105 2024-03-26T18:55:54.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:54 smithi097 ceph-mon[31955]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:54.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:54 smithi144 ceph-mon[36053]: Deploying daemon mds.e on smithi105 2024-03-26T18:55:54.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:54 smithi144 ceph-mon[36053]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.706+0000 7f0da1a00640 1 -- 172.21.15.97:0/1909813670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d9c02d240 con 0x7f0d9c103b00 2024-03-26T18:55:54.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.707+0000 7f0d93400640 1 -- 172.21.15.97:0/1909813670 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f0d8c0be240 con 0x7f0d9c103b00 2024-03-26T18:55:54.700 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:54.700 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-03-26T18:55:54.700 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 5 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 -- 172.21.15.97:0/1909813670 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d800a2fa0 msgr2=0x7f0d800a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d800a2fa0 0x7f0d800a5460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c19b370 tx=0x7f0d84005f90 comp rx=0 tx=0).stop 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 -- 172.21.15.97:0/1909813670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d9c103b00 msgr2=0x7f0d9c199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d9c103b00 0x7f0d9c199e60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c000c00 tx=0x7f0d8c02c650 comp rx=0 tx=0).stop 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 -- 172.21.15.97:0/1909813670 shutdown_connections 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d800a2fa0 0x7f0d800a5460 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c1056b0 0x7f0d9c1a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c104cf0 0x7f0d9c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 --2- 172.21.15.97:0/1909813670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d9c103b00 0x7f0d9c199e60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 -- 172.21.15.97:0/1909813670 >> 172.21.15.97:0/1909813670 conn(0x7f0d9c0ff070 msgr2=0x7f0d9c108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:54.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 -- 172.21.15.97:0/1909813670 shutdown_connections 2024-03-26T18:55:54.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:54.710+0000 7f0da1a00640 1 -- 172.21.15.97:0/1909813670 wait complete. 2024-03-26T18:55:54.710 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 18:55:54 smithi105 systemd[1]: Starting Ceph mds.e for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:55:54.990 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 18:55:54 smithi105 podman[65665]: 2024-03-26 18:55:54.634808784 +0000 UTC m=+0.017995630 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:55:54.990 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 18:55:54 smithi105 podman[65665]: 2024-03-26 18:55:54.754179206 +0000 UTC m=+0.137366050 container create 2fff5cc0ee46be0bdac89e3ef2fea1f87560b55fe741d3fc83417e76d5538aef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-03-26T18:55:54.990 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 18:55:54 smithi105 podman[65665]: 2024-03-26 18:55:54.973154839 +0000 UTC m=+0.356341700 container init 2fff5cc0ee46be0bdac89e3ef2fea1f87560b55fe741d3fc83417e76d5538aef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:55:54.991 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 18:55:54 smithi105 podman[65665]: 2024-03-26 18:55:54.975713496 +0000 UTC m=+0.358900333 container start 2fff5cc0ee46be0bdac89e3ef2fea1f87560b55fe741d3fc83417e76d5538aef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-26T18:55:55.460 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 18:55:54 smithi105 ceph-mds[65684]: -- 172.21.15.105:0/1215368950 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55976c79ba40 con 0x55976c76e800 2024-03-26T18:55:55.460 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 18:55:54 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e[65680]: starting mds.e at 2024-03-26T18:55:55.460 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 18:55:55 smithi105 bash[65665]: 2fff5cc0ee46be0bdac89e3ef2fea1f87560b55fe741d3fc83417e76d5538aef 2024-03-26T18:55:55.461 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 18:55:55 smithi105 systemd[1]: Started Ceph mds.e for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:55:55.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:55 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1909813670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:55.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:55.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:55.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:55.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:55.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:55.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:55 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:55.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:55 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1909813670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:55.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:55.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:55.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:55.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:55.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:55.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:55 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:55.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:55 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1909813670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:55.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:55.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:55.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:55.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:55.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:55.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:55 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:56 smithi105 ceph-mon[36270]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:56 smithi105 ceph-mon[36270]: Deploying daemon mds.f on smithi144 2024-03-26T18:55:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:56 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] up:boot 2024-03-26T18:55:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:56 smithi105 ceph-mon[36270]: fsmap 5 up:standby 2024-03-26T18:55:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:56 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:55:56.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:56 smithi097 ceph-mon[31955]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:56.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:56 smithi097 ceph-mon[31955]: Deploying daemon mds.f on smithi144 2024-03-26T18:55:56.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:56 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] up:boot 2024-03-26T18:55:56.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:56 smithi097 ceph-mon[31955]: fsmap 5 up:standby 2024-03-26T18:55:56.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:56 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:55:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:56 smithi144 ceph-mon[36053]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:56 smithi144 ceph-mon[36053]: Deploying daemon mds.f on smithi144 2024-03-26T18:55:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:56 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] up:boot 2024-03-26T18:55:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:56 smithi144 ceph-mon[36053]: fsmap 5 up:standby 2024-03-26T18:55:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:56 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:55:56.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:55:56.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.954+0000 7f4ceea00640 1 -- 172.21.15.97:0/2537968884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8104cf0 msgr2=0x7f4ce8105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:56.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.954+0000 7f4ceea00640 1 --2- 172.21.15.97:0/2537968884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8104cf0 0x7f4ce8105170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009860 tx=0x7f4cd402bb70 comp rx=0 tx=0).stop 2024-03-26T18:55:56.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.954+0000 7f4ceea00640 1 -- 172.21.15.97:0/2537968884 shutdown_connections 2024-03-26T18:55:56.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.954+0000 7f4ceea00640 1 --2- 172.21.15.97:0/2537968884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce81056b0 0x7f4ce810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:56.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.954+0000 7f4ceea00640 1 --2- 172.21.15.97:0/2537968884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8104cf0 0x7f4ce8105170 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:56.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.954+0000 7f4ceea00640 1 --2- 172.21.15.97:0/2537968884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce8103b00 0x7f4ce8103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:56.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.954+0000 7f4ceea00640 1 -- 172.21.15.97:0/2537968884 >> 172.21.15.97:0/2537968884 conn(0x7f4ce80ff070 msgr2=0x7f4ce81014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:56.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.954+0000 7f4ceea00640 1 -- 172.21.15.97:0/2537968884 shutdown_connections 2024-03-26T18:55:56.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.954+0000 7f4ceea00640 1 -- 172.21.15.97:0/2537968884 wait complete. 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.955+0000 7f4ceea00640 1 Processor -- start 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.955+0000 7f4ceea00640 1 -- start start 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ceea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8103b00 0x7f4ce8199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ceea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce8104cf0 0x7f4ce819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ceea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce81056b0 0x7f4ce81a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ceea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce810f780 con 0x7f4ce81056b0 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ceea00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce810f600 con 0x7f4ce8103b00 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ceea00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce810f900 con 0x7f4ce8104cf0 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ceca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce81056b0 0x7f4ce81a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ceca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce81056b0 0x7f4ce81a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57916/0 (socket says 172.21.15.97:57916) 2024-03-26T18:55:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ceca00640 1 -- 172.21.15.97:0/228248640 learned_addr learned my addr 172.21.15.97:0/228248640 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ce7400640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce8104cf0 0x7f4ce819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ce7e00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8103b00 0x7f4ce8199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ce7400640 1 -- 172.21.15.97:0/228248640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8103b00 msgr2=0x7f4ce8199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ce7400640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8103b00 0x7f4ce8199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ce7400640 1 -- 172.21.15.97:0/228248640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce81056b0 msgr2=0x7f4ce81a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ce7400640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce81056b0 0x7f4ce81a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ce7400640 1 -- 172.21.15.97:0/228248640 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce81a1a80 con 0x7f4ce8104cf0 2024-03-26T18:55:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.956+0000 7f4ceca00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce81056b0 0x7f4ce81a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:55:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.957+0000 7f4ce7400640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce8104cf0 0x7f4ce819a3a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4cd402c0b0 tx=0x7f4cd4032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.957+0000 7f4ce4c00640 1 -- 172.21.15.97:0/228248640 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4041020 con 0x7f4ce8104cf0 2024-03-26T18:55:56.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.957+0000 7f4ce4c00640 1 -- 172.21.15.97:0/228248640 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd402c9f0 con 0x7f4ce8104cf0 2024-03-26T18:55:56.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.957+0000 7f4ceea00640 1 -- 172.21.15.97:0/228248640 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce81a1d10 con 0x7f4ce8104cf0 2024-03-26T18:55:56.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.957+0000 7f4ce4c00640 1 -- 172.21.15.97:0/228248640 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4039040 con 0x7f4ce8104cf0 2024-03-26T18:55:56.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.957+0000 7f4ceea00640 1 -- 172.21.15.97:0/228248640 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce81a21d0 con 0x7f4ce8104cf0 2024-03-26T18:55:56.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.958+0000 7f4ceea00640 1 -- 172.21.15.97:0/228248640 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce810aea0 con 0x7f4ce8104cf0 2024-03-26T18:55:56.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.961+0000 7f4ce4c00640 1 -- 172.21.15.97:0/228248640 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f4cd4044020 con 0x7f4ce8104cf0 2024-03-26T18:55:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.962+0000 7f4ce4c00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4ccc0a2f20 0x7f4ccc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.962+0000 7f4ce4c00640 1 -- 172.21.15.97:0/228248640 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4cd40f06e0 con 0x7f4ce8104cf0 2024-03-26T18:55:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.962+0000 7f4ce7e00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4ccc0a2f20 0x7f4ccc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.963+0000 7f4ce7e00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4ccc0a2f20 0x7f4ccc0a53e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8104b50 tx=0x7f4cd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:56.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:56.966+0000 7f4ce4c00640 1 -- 172.21.15.97:0/228248640 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4cd404b020 con 0x7f4ce8104cf0 2024-03-26T18:55:57.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.205+0000 7f4ceea00640 1 -- 172.21.15.97:0/228248640 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ce802d450 con 0x7f4ce8104cf0 2024-03-26T18:55:57.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.206+0000 7f4ce4c00640 1 -- 172.21.15.97:0/228248640 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f4cd40b7e20 con 0x7f4ce8104cf0 2024-03-26T18:55:57.199 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:57.199 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-26T18:55:57.199 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-03-26T18:55:57.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.208+0000 7f4ceea00640 1 -- 172.21.15.97:0/228248640 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4ccc0a2f20 msgr2=0x7f4ccc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:57.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.208+0000 7f4ceea00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4ccc0a2f20 0x7f4ccc0a53e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8104b50 tx=0x7f4cd8005f90 comp rx=0 tx=0).stop 2024-03-26T18:55:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.208+0000 7f4ceea00640 1 -- 172.21.15.97:0/228248640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce8104cf0 msgr2=0x7f4ce819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.208+0000 7f4ceea00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce8104cf0 0x7f4ce819a3a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4cd402c0b0 tx=0x7f4cd4032430 comp rx=0 tx=0).stop 2024-03-26T18:55:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.209+0000 7f4ceea00640 1 -- 172.21.15.97:0/228248640 shutdown_connections 2024-03-26T18:55:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.209+0000 7f4ceea00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4ccc0a2f20 0x7f4ccc0a53e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.209+0000 7f4ceea00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce81056b0 0x7f4ce81a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.209+0000 7f4ceea00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce8104cf0 0x7f4ce819a3a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.209+0000 7f4ceea00640 1 --2- 172.21.15.97:0/228248640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8103b00 0x7f4ce8199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.209+0000 7f4ceea00640 1 -- 172.21.15.97:0/228248640 >> 172.21.15.97:0/228248640 conn(0x7f4ce80ff070 msgr2=0x7f4ce8108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.209+0000 7f4ceea00640 1 -- 172.21.15.97:0/228248640 shutdown_connections 2024-03-26T18:55:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:57.209+0000 7f4ceea00640 1 -- 172.21.15.97:0/228248640 wait complete. 2024-03-26T18:55:57.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:57 smithi105 ceph-mon[36270]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:57.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:57 smithi097 ceph-mon[31955]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:57.606 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:57 smithi144 ceph-mon[36053]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:57.607 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:55:57 smithi144 systemd[1]: Starting Ceph mds.f for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:55:57.867 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:55:57 smithi144 podman[66345]: 2024-03-26 18:55:57.513231559 +0000 UTC m=+0.017632603 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:55:57.867 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:55:57 smithi144 podman[66345]: 2024-03-26 18:55:57.638280358 +0000 UTC m=+0.142681392 container create 81527c8e835fe449f644c5561d81b36ffc45561ee6e6b745fb060ffa2f2ed3bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-26T18:55:57.867 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:55:57 smithi144 podman[66345]: 2024-03-26 18:55:57.849480532 +0000 UTC m=+0.353881576 container init 81527c8e835fe449f644c5561d81b36ffc45561ee6e6b745fb060ffa2f2ed3bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T18:55:57.868 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:55:57 smithi144 podman[66345]: 2024-03-26 18:55:57.852198051 +0000 UTC m=+0.356599088 container start 81527c8e835fe449f644c5561d81b36ffc45561ee6e6b745fb060ffa2f2ed3bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:55:57.868 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:55:57 smithi144 ceph-mds[66364]: -- 172.21.15.144:0/82758538 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e31bd3a40 con 0x563e31ba7000 2024-03-26T18:55:58.126 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:55:57 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f[66360]: starting mds.f at 2024-03-26T18:55:58.126 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:55:57 smithi144 bash[66345]: 81527c8e835fe449f644c5561d81b36ffc45561ee6e6b745fb060ffa2f2ed3bc 2024-03-26T18:55:58.126 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:55:57 smithi144 systemd[1]: Started Ceph mds.f for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:55:58.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:58 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/228248640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:58 smithi105 ceph-mon[36270]: Deploying daemon mds.c on smithi144 2024-03-26T18:55:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:58 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/228248640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:58 smithi097 ceph-mon[31955]: Deploying daemon mds.c on smithi144 2024-03-26T18:55:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:58 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/228248640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:55:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:58.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:55:58.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:55:58.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:55:58.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:55:58.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:58 smithi144 ceph-mon[36053]: Deploying daemon mds.c on smithi144 2024-03-26T18:55:59.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:55:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:59 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.144:6834/411039946,v1:172.21.15.144:6835/411039946] up:boot 2024-03-26T18:55:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:59 smithi144 ceph-mon[36053]: fsmap 6 up:standby 2024-03-26T18:55:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:59 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:55:59.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:55:59 smithi144 ceph-mon[36053]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.451+0000 7f20f6800640 1 -- 172.21.15.97:0/1844756530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f0107450 msgr2=0x7f20f0109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.451+0000 7f20f6800640 1 --2- 172.21.15.97:0/1844756530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f0107450 0x7f20f0109840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009ca0 tx=0x7f20e802c100 comp rx=0 tx=0).stop 2024-03-26T18:55:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.451+0000 7f20f6800640 1 -- 172.21.15.97:0/1844756530 shutdown_connections 2024-03-26T18:55:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.451+0000 7f20f6800640 1 --2- 172.21.15.97:0/1844756530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f0107450 0x7f20f0109840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.451+0000 7f20f6800640 1 --2- 172.21.15.97:0/1844756530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f0104b20 0x7f20f0106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.451+0000 7f20f6800640 1 --2- 172.21.15.97:0/1844756530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f01021f0 0x7f20f01045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.451+0000 7f20f6800640 1 -- 172.21.15.97:0/1844756530 >> 172.21.15.97:0/1844756530 conn(0x7f20f00fbba0 msgr2=0x7f20f00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.451+0000 7f20f6800640 1 -- 172.21.15.97:0/1844756530 shutdown_connections 2024-03-26T18:55:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.451+0000 7f20f6800640 1 -- 172.21.15.97:0/1844756530 wait complete. 2024-03-26T18:55:59.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.452+0000 7f20f6800640 1 Processor -- start 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f6800640 1 -- start start 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f01021f0 0x7f20f0101b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f0104b20 0x7f20f01001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f6800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f0107450 0x7f20f01006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f6800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20f010d390 con 0x7f20f0104b20 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f6800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f20f010d210 con 0x7f20f0107450 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20f010d510 con 0x7f20f01021f0 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f5400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f01021f0 0x7f20f0101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f0104b20 0x7f20f01001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f0107450 0x7f20f01006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f5400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f01021f0 0x7f20f0101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:49870/0 (socket says 172.21.15.97:49870) 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f0104b20 0x7f20f01001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57922/0 (socket says 172.21.15.97:57922) 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.453+0000 7f20f5400640 1 -- 172.21.15.97:0/3577682554 learned_addr learned my addr 172.21.15.97:0/3577682554 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:55:59.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20f5400640 1 -- 172.21.15.97:0/3577682554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f0107450 msgr2=0x7f20f01006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20f5400640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f0107450 0x7f20f01006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20f5400640 1 -- 172.21.15.97:0/3577682554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f0104b20 msgr2=0x7f20f01001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20f5400640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f0104b20 0x7f20f01001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20f5400640 1 -- 172.21.15.97:0/3577682554 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f0100f70 con 0x7f20f01021f0 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20f4a00640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f0104b20 0x7f20f01001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20f5400640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f01021f0 0x7f20f0101b60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f20e400e9f0 tx=0x7f20e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20ee000640 1 -- 172.21.15.97:0/3577682554 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e400cd80 con 0x7f20f01021f0 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20f6800640 1 -- 172.21.15.97:0/3577682554 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f01a1ac0 con 0x7f20f01021f0 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20ee000640 1 -- 172.21.15.97:0/3577682554 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e4010910 con 0x7f20f01021f0 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20ee000640 1 -- 172.21.15.97:0/3577682554 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4018950 con 0x7f20f01021f0 2024-03-26T18:55:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.454+0000 7f20f6800640 1 -- 172.21.15.97:0/3577682554 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f01a2010 con 0x7f20f01021f0 2024-03-26T18:55:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.455+0000 7f20f6800640 1 -- 172.21.15.97:0/3577682554 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f00fd690 con 0x7f20f01021f0 2024-03-26T18:55:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.457+0000 7f20ee000640 1 -- 172.21.15.97:0/3577682554 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f20e4010ac0 con 0x7f20f01021f0 2024-03-26T18:55:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.458+0000 7f20ee000640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20d80a2f20 0x7f20d80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:55:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.459+0000 7f20ee000640 1 -- 172.21.15.97:0/3577682554 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f20e40cf380 con 0x7f20f01021f0 2024-03-26T18:55:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.459+0000 7f20f4a00640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20d80a2f20 0x7f20d80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:55:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.459+0000 7f20f4a00640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20d80a2f20 0x7f20d80a53e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f20f01018d0 tx=0x7f20dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:55:59.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.463+0000 7f20ee000640 1 -- 172.21.15.97:0/3577682554 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f20e409bf60 con 0x7f20f01021f0 2024-03-26T18:55:59.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:59 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.144:6834/411039946,v1:172.21.15.144:6835/411039946] up:boot 2024-03-26T18:55:59.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:59 smithi105 ceph-mon[36270]: fsmap 6 up:standby 2024-03-26T18:55:59.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:59 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:55:59.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:55:59 smithi105 ceph-mon[36270]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:59.602 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:59 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.144:6834/411039946,v1:172.21.15.144:6835/411039946] up:boot 2024-03-26T18:55:59.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:59 smithi097 ceph-mon[31955]: fsmap 6 up:standby 2024-03-26T18:55:59.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:59 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:55:59.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:55:59 smithi097 ceph-mon[31955]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:55:59.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.702+0000 7f20f6800640 1 -- 172.21.15.97:0/3577682554 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20f00624c0 con 0x7f20f01021f0 2024-03-26T18:55:59.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.703+0000 7f20ee000640 1 -- 172.21.15.97:0/3577682554 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f20e40a0000 con 0x7f20f01021f0 2024-03-26T18:55:59.695 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:55:59.695 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-26T18:55:59.696 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2024-03-26T18:55:59.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 -- 172.21.15.97:0/3577682554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20d80a2f20 msgr2=0x7f20d80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:59.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20d80a2f20 0x7f20d80a53e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f20f01018d0 tx=0x7f20dc005f90 comp rx=0 tx=0).stop 2024-03-26T18:55:59.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 -- 172.21.15.97:0/3577682554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f01021f0 msgr2=0x7f20f0101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:55:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f01021f0 0x7f20f0101b60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f20e400e9f0 tx=0x7f20e400eeb0 comp rx=0 tx=0).stop 2024-03-26T18:55:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 -- 172.21.15.97:0/3577682554 shutdown_connections 2024-03-26T18:55:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20d80a2f20 0x7f20d80a53e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20f0107450 0x7f20f01006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f0104b20 0x7f20f01001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 --2- 172.21.15.97:0/3577682554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f01021f0 0x7f20f0101b60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:55:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 -- 172.21.15.97:0/3577682554 >> 172.21.15.97:0/3577682554 conn(0x7f20f00fbba0 msgr2=0x7f20f0102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:55:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 -- 172.21.15.97:0/3577682554 shutdown_connections 2024-03-26T18:55:59.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:55:59.706+0000 7f20f6800640 1 -- 172.21.15.97:0/3577682554 wait complete. 2024-03-26T18:56:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:00 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3577682554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:00.376 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:56:00 smithi144 systemd[1]: Starting Ceph mds.c for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:56:00.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:00 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3577682554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:00.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:00 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3577682554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:00.739 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:56:00 smithi144 podman[66654]: 2024-03-26 18:56:00.392474927 +0000 UTC m=+0.018384282 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:56:00.740 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:56:00 smithi144 podman[66654]: 2024-03-26 18:56:00.511445448 +0000 UTC m=+0.137354846 container create a57bdb37035e9fdb40cf5a94ac8658839d25d98d3ef76755e72032ad1d79ff5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-26T18:56:00.740 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:56:00 smithi144 podman[66654]: 2024-03-26 18:56:00.747841869 +0000 UTC m=+0.373751224 container init a57bdb37035e9fdb40cf5a94ac8658839d25d98d3ef76755e72032ad1d79ff5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-26T18:56:01.126 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:56:00 smithi144 podman[66654]: 2024-03-26 18:56:00.750465726 +0000 UTC m=+0.376375082 container start a57bdb37035e9fdb40cf5a94ac8658839d25d98d3ef76755e72032ad1d79ff5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-03-26T18:56:01.126 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:56:00 smithi144 ceph-mds[66673]: -- 172.21.15.144:0/2042298447 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56354980fa40 con 0x5635497e2c00 2024-03-26T18:56:01.126 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:56:00 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c[66669]: starting mds.c at 2024-03-26T18:56:01.126 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:56:00 smithi144 bash[66654]: a57bdb37035e9fdb40cf5a94ac8658839d25d98d3ef76755e72032ad1d79ff5f 2024-03-26T18:56:01.126 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:56:00 smithi144 systemd[1]: Started Ceph mds.c for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:56:01.720 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:56:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.931+0000 7ff38a000640 1 -- 172.21.15.97:0/2071301728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3840758a0 msgr2=0x7ff384073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.931+0000 7ff38a000640 1 --2- 172.21.15.97:0/2071301728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3840758a0 0x7ff384073cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff378009900 tx=0x7ff37802bcd0 comp rx=0 tx=0).stop 2024-03-26T18:56:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.931+0000 7ff38a000640 1 -- 172.21.15.97:0/2071301728 shutdown_connections 2024-03-26T18:56:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.931+0000 7ff38a000640 1 --2- 172.21.15.97:0/2071301728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3841095c0 0x7ff38410ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.931+0000 7ff38a000640 1 --2- 172.21.15.97:0/2071301728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3840742c0 0x7ff384074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.931+0000 7ff38a000640 1 --2- 172.21.15.97:0/2071301728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3840758a0 0x7ff384073cf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.931+0000 7ff38a000640 1 -- 172.21.15.97:0/2071301728 >> 172.21.15.97:0/2071301728 conn(0x7ff3840fcca0 msgr2=0x7ff3840ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.931+0000 7ff38a000640 1 -- 172.21.15.97:0/2071301728 shutdown_connections 2024-03-26T18:56:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.931+0000 7ff38a000640 1 -- 172.21.15.97:0/2071301728 wait complete. 2024-03-26T18:56:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff38a000640 1 Processor -- start 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff38a000640 1 -- start start 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff38a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3840742c0 0x7ff384199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff38a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3840758a0 0x7ff38419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff38a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3841095c0 0x7ff3841a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff38a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff38410f5a0 con 0x7ff3841095c0 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff38a000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff38410f420 con 0x7ff3840742c0 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff38a000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff38410f720 con 0x7ff3840758a0 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff388c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3840742c0 0x7ff384199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff388c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3840742c0 0x7ff384199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34544/0 (socket says 172.21.15.97:34544) 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.932+0000 7ff388c00640 1 -- 172.21.15.97:0/2656245147 learned_addr learned my addr 172.21.15.97:0/2656245147 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff383e00640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3840758a0 0x7ff38419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff389600640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3841095c0 0x7ff3841a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff388c00640 1 -- 172.21.15.97:0/2656245147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3840758a0 msgr2=0x7ff38419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff388c00640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3840758a0 0x7ff38419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff388c00640 1 -- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3841095c0 msgr2=0x7ff3841a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff388c00640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3841095c0 0x7ff3841a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff388c00640 1 -- 172.21.15.97:0/2656245147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3841a18b0 con 0x7ff3840742c0 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff389600640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3841095c0 0x7ff3841a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff383e00640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3840758a0 0x7ff38419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff388c00640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3840742c0 0x7ff384199c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff378005ab0 tx=0x7ff37802c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff381600640 1 -- 172.21.15.97:0/2656245147 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378041070 con 0x7ff3840742c0 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff381600640 1 -- 172.21.15.97:0/2656245147 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff378004450 con 0x7ff3840742c0 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff38a000640 1 -- 172.21.15.97:0/2656245147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3841a1b40 con 0x7ff3840742c0 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.933+0000 7ff381600640 1 -- 172.21.15.97:0/2656245147 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378034440 con 0x7ff3840742c0 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.934+0000 7ff38a000640 1 -- 172.21.15.97:0/2656245147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3841a2030 con 0x7ff3840742c0 2024-03-26T18:56:01.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.934+0000 7ff38a000640 1 -- 172.21.15.97:0/2656245147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff384108380 con 0x7ff3840742c0 2024-03-26T18:56:01.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.936+0000 7ff381600640 1 -- 172.21.15.97:0/2656245147 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7ff378044070 con 0x7ff3840742c0 2024-03-26T18:56:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.937+0000 7ff381600640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff36c0a2f20 0x7ff36c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.938+0000 7ff381600640 1 -- 172.21.15.97:0/2656245147 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff3780f0dc0 con 0x7ff3840742c0 2024-03-26T18:56:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.938+0000 7ff383e00640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff36c0a2f20 0x7ff36c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.938+0000 7ff383e00640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff36c0a2f20 0x7ff36c0a53e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff38419b170 tx=0x7ff370005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:01.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:01.940+0000 7ff381600640 1 -- 172.21.15.97:0/2656245147 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff3780be080 con 0x7ff3840742c0 2024-03-26T18:56:02.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:02.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:02.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:02.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:02.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:02.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:02.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:02.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: Deploying daemon mds.h on smithi105 2024-03-26T18:56:02.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.144:6836/1138636767,v1:172.21.15.144:6837/1138636767] up:boot 2024-03-26T18:56:02.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: fsmap 7 up:standby 2024-03-26T18:56:02.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:56:02.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:56:02.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.177+0000 7ff38a000640 1 -- 172.21.15.97:0/2656245147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff38402d240 con 0x7ff3840742c0 2024-03-26T18:56:02.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.178+0000 7ff381600640 1 -- 172.21.15.97:0/2656245147 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7ff3780bd980 con 0x7ff3840742c0 2024-03-26T18:56:02.170 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:02.170 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-26T18:56:02.171 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-03-26T18:56:02.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.180+0000 7ff38a000640 1 -- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff36c0a2f20 msgr2=0x7ff36c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:02.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.180+0000 7ff38a000640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff36c0a2f20 0x7ff36c0a53e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff38419b170 tx=0x7ff370005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:02.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.180+0000 7ff38a000640 1 -- 172.21.15.97:0/2656245147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3840742c0 msgr2=0x7ff384199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:02.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.180+0000 7ff38a000640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3840742c0 0x7ff384199c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff378005ab0 tx=0x7ff37802c6e0 comp rx=0 tx=0).stop 2024-03-26T18:56:02.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.181+0000 7ff38a000640 1 -- 172.21.15.97:0/2656245147 shutdown_connections 2024-03-26T18:56:02.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.181+0000 7ff38a000640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff36c0a2f20 0x7ff36c0a53e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:02.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.181+0000 7ff38a000640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3841095c0 0x7ff3841a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:02.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.181+0000 7ff38a000640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3840758a0 0x7ff38419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:02.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.181+0000 7ff38a000640 1 --2- 172.21.15.97:0/2656245147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3840742c0 0x7ff384199c70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:02.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.181+0000 7ff38a000640 1 -- 172.21.15.97:0/2656245147 >> 172.21.15.97:0/2656245147 conn(0x7ff3840fcca0 msgr2=0x7ff384105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:02.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.181+0000 7ff38a000640 1 -- 172.21.15.97:0/2656245147 shutdown_connections 2024-03-26T18:56:02.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:02.181+0000 7ff38a000640 1 -- 172.21.15.97:0/2656245147 wait complete. 2024-03-26T18:56:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: Deploying daemon mds.h on smithi105 2024-03-26T18:56:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.144:6836/1138636767,v1:172.21.15.144:6837/1138636767] up:boot 2024-03-26T18:56:02.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: fsmap 7 up:standby 2024-03-26T18:56:02.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:56:02.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:56:02.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:02.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:02.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:02.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:02.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:02.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:02.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:02.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: Deploying daemon mds.h on smithi105 2024-03-26T18:56:02.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.144:6836/1138636767,v1:172.21.15.144:6837/1138636767] up:boot 2024-03-26T18:56:02.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: fsmap 7 up:standby 2024-03-26T18:56:02.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:56:02.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:56:03.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:02 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2656245147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:03.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:02 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2656245147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:03.210 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 18:56:03 smithi105 systemd[1]: Starting Ceph mds.h for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:56:03.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:02 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2656245147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:03.605 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 18:56:03 smithi105 podman[65974]: 2024-03-26 18:56:03.193785882 +0000 UTC m=+0.016981780 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:56:03.605 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 18:56:03 smithi105 podman[65974]: 2024-03-26 18:56:03.350744703 +0000 UTC m=+0.173940596 container create b609505c9761d26eac2f5ea3bd7e6f2c6878db7773a9e8d9748e793c6b7f0db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:56:03.920 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 18:56:03 smithi105 podman[65974]: 2024-03-26 18:56:03.612172898 +0000 UTC m=+0.435368798 container init b609505c9761d26eac2f5ea3bd7e6f2c6878db7773a9e8d9748e793c6b7f0db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-26T18:56:03.920 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 18:56:03 smithi105 podman[65974]: 2024-03-26 18:56:03.614809041 +0000 UTC m=+0.438004950 container start b609505c9761d26eac2f5ea3bd7e6f2c6878db7773a9e8d9748e793c6b7f0db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux ) 2024-03-26T18:56:03.920 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 18:56:03 smithi105 ceph-mds[65993]: -- 172.21.15.105:0/1682191324 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fdd7327a40 con 0x55fdd72fac00 2024-03-26T18:56:03.920 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 18:56:03 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h[65989]: starting mds.h at 2024-03-26T18:56:03.920 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 18:56:03 smithi105 bash[65974]: b609505c9761d26eac2f5ea3bd7e6f2c6878db7773a9e8d9748e793c6b7f0db5 2024-03-26T18:56:03.920 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 18:56:03 smithi105 systemd[1]: Started Ceph mds.h for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:56:04.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:56:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:03 smithi105 ceph-mon[36270]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:04.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:03 smithi097 ceph-mon[31955]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:04.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:04.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:04.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:04.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:04.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:04.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:04.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:03 smithi144 ceph-mon[36053]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:04.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:04.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:04.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:04.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:04.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:04.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.414+0000 7f2cf3200640 1 -- 172.21.15.97:0/1053067996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cec109090 msgr2=0x7f2cec109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.414+0000 7f2cf3200640 1 --2- 172.21.15.97:0/1053067996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cec109090 0x7f2cec109510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009900 tx=0x7f2cd402bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.414+0000 7f2cf3200640 1 -- 172.21.15.97:0/1053067996 shutdown_connections 2024-03-26T18:56:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.414+0000 7f2cf3200640 1 --2- 172.21.15.97:0/1053067996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec109a50 0x7f2cec110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.414+0000 7f2cf3200640 1 --2- 172.21.15.97:0/1053067996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cec109090 0x7f2cec109510 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.414+0000 7f2cf3200640 1 --2- 172.21.15.97:0/1053067996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec107eb0 0x7f2cec1082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.414+0000 7f2cf3200640 1 -- 172.21.15.97:0/1053067996 >> 172.21.15.97:0/1053067996 conn(0x7f2cec076ed0 msgr2=0x7f2cec0792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.414+0000 7f2cf3200640 1 -- 172.21.15.97:0/1053067996 shutdown_connections 2024-03-26T18:56:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.414+0000 7f2cf3200640 1 -- 172.21.15.97:0/1053067996 wait complete. 2024-03-26T18:56:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.415+0000 7f2cf3200640 1 Processor -- start 2024-03-26T18:56:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.415+0000 7f2cf3200640 1 -- start start 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cf3200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cec107eb0 0x7f2cec19e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cf3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec109090 0x7f2cec19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cf3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec109a50 0x7f2cec1a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cf3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec113bf0 con 0x7f2cec109a50 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cf3200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec113a70 con 0x7f2cec107eb0 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cf3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec113d70 con 0x7f2cec109090 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cf1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec109a50 0x7f2cec1a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cf1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec109a50 0x7f2cec1a57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55684/0 (socket says 172.21.15.97:55684) 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cf1200640 1 -- 172.21.15.97:0/1358994991 learned_addr learned my addr 172.21.15.97:0/1358994991 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cebe00640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cec107eb0 0x7f2cec19e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2ce3400640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec109090 0x7f2cec19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.416+0000 7f2cf1200640 1 -- 172.21.15.97:0/1358994991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec109090 msgr2=0x7f2cec19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2cf1200640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec109090 0x7f2cec19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2cf1200640 1 -- 172.21.15.97:0/1358994991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cec107eb0 msgr2=0x7f2cec19e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2cf1200640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cec107eb0 0x7f2cec19e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2cf1200640 1 -- 172.21.15.97:0/1358994991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec1a5e20 con 0x7f2cec109a50 2024-03-26T18:56:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2cebe00640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cec107eb0 0x7f2cec19e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2cf1200640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec109a50 0x7f2cec1a57b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400eb10 tx=0x7f2ce400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2ce9600640 1 -- 172.21.15.97:0/1358994991 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4004b20 con 0x7f2cec109a50 2024-03-26T18:56:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2cf3200640 1 -- 172.21.15.97:0/1358994991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cec1a6110 con 0x7f2cec109a50 2024-03-26T18:56:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2ce9600640 1 -- 172.21.15.97:0/1358994991 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce4004cc0 con 0x7f2cec109a50 2024-03-26T18:56:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2ce9600640 1 -- 172.21.15.97:0/1358994991 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4010760 con 0x7f2cec109a50 2024-03-26T18:56:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.417+0000 7f2cf3200640 1 -- 172.21.15.97:0/1358994991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cec1a65e0 con 0x7f2cec109a50 2024-03-26T18:56:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.418+0000 7f2cf3200640 1 -- 172.21.15.97:0/1358994991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cec10f250 con 0x7f2cec109a50 2024-03-26T18:56:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.420+0000 7f2ce9600640 1 -- 172.21.15.97:0/1358994991 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f2ce4005020 con 0x7f2cec109a50 2024-03-26T18:56:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.422+0000 7f2ce9600640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2cd00a2f20 0x7f2cd00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.422+0000 7f2ce9600640 1 -- 172.21.15.97:0/1358994991 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2ce40cf540 con 0x7f2cec109a50 2024-03-26T18:56:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.422+0000 7f2cebe00640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2cd00a2f20 0x7f2cd00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.422+0000 7f2cebe00640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2cd00a2f20 0x7f2cd00a53e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2cec108c00 tx=0x7f2cdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.426+0000 7f2ce9600640 1 -- 172.21.15.97:0/1358994991 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2ce409c770 con 0x7f2cec109a50 2024-03-26T18:56:04.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.665+0000 7f2cf3200640 1 -- 172.21.15.97:0/1358994991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cec0624c0 con 0x7f2cec109a50 2024-03-26T18:56:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.666+0000 7f2ce9600640 1 -- 172.21.15.97:0/1358994991 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f2ce409c070 con 0x7f2cec109a50 2024-03-26T18:56:04.659 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:04.659 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-26T18:56:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-03-26T18:56:04.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 -- 172.21.15.97:0/1358994991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2cd00a2f20 msgr2=0x7f2cd00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2cd00a2f20 0x7f2cd00a53e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2cec108c00 tx=0x7f2cdc005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 -- 172.21.15.97:0/1358994991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec109a50 msgr2=0x7f2cec1a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec109a50 0x7f2cec1a57b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400eb10 tx=0x7f2ce400c2d0 comp rx=0 tx=0).stop 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 -- 172.21.15.97:0/1358994991 shutdown_connections 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2cd00a2f20 0x7f2cd00a53e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec109a50 0x7f2cec1a57b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec109090 0x7f2cec19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 --2- 172.21.15.97:0/1358994991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cec107eb0 0x7f2cec19e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 -- 172.21.15.97:0/1358994991 >> 172.21.15.97:0/1358994991 conn(0x7f2cec076ed0 msgr2=0x7f2cec10c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 -- 172.21.15.97:0/1358994991 shutdown_connections 2024-03-26T18:56:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:04.669+0000 7f2cf3200640 1 -- 172.21.15.97:0/1358994991 wait complete. 2024-03-26T18:56:05.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:04 smithi097 ceph-mon[31955]: Deploying daemon mds.i on smithi144 2024-03-26T18:56:05.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:04 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6836/1343808654,v1:172.21.15.105:6837/1343808654] up:boot 2024-03-26T18:56:05.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:04 smithi097 ceph-mon[31955]: fsmap 8 up:standby 2024-03-26T18:56:05.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:04 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:56:05.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:04 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1358994991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:05.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:04 smithi144 ceph-mon[36053]: Deploying daemon mds.i on smithi144 2024-03-26T18:56:05.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:04 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6836/1343808654,v1:172.21.15.105:6837/1343808654] up:boot 2024-03-26T18:56:05.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:04 smithi144 ceph-mon[36053]: fsmap 8 up:standby 2024-03-26T18:56:05.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:04 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:56:05.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:04 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1358994991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:05.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:04 smithi105 ceph-mon[36270]: Deploying daemon mds.i on smithi144 2024-03-26T18:56:05.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:04 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6836/1343808654,v1:172.21.15.105:6837/1343808654] up:boot 2024-03-26T18:56:05.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:04 smithi105 ceph-mon[36270]: fsmap 8 up:standby 2024-03-26T18:56:05.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:04 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:56:05.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:04 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1358994991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:06.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:05 smithi144 ceph-mon[36053]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:06.127 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:56:05 smithi144 systemd[1]: Starting Ceph mds.i for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:56:06.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:05 smithi105 ceph-mon[36270]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:06.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:05 smithi097 ceph-mon[31955]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:06.471 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:56:06 smithi144 podman[66963]: 2024-03-26 18:56:06.118612229 +0000 UTC m=+0.018397645 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:56:06.471 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:56:06 smithi144 podman[66963]: 2024-03-26 18:56:06.234551752 +0000 UTC m=+0.134337154 container create a28535586c94ea935e3b1c8ff7d2dc7eaa5d237a526b22398cf97ef34215217d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T18:56:06.471 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:56:06 smithi144 podman[66963]: 2024-03-26 18:56:06.479116314 +0000 UTC m=+0.378901726 container init a28535586c94ea935e3b1c8ff7d2dc7eaa5d237a526b22398cf97ef34215217d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:56:06.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:56:06.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.881+0000 7f2523600640 1 -- 172.21.15.97:0/2161205515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f251c102ad0 msgr2=0x7f251c102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.881+0000 7f2523600640 1 --2- 172.21.15.97:0/2161205515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f251c102ad0 0x7f251c102f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2508009900 tx=0x7f250802bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.882+0000 7f2523600640 1 -- 172.21.15.97:0/2161205515 shutdown_connections 2024-03-26T18:56:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.882+0000 7f2523600640 1 --2- 172.21.15.97:0/2161205515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251c103490 0x7f251c109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.882+0000 7f2523600640 1 --2- 172.21.15.97:0/2161205515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f251c102ad0 0x7f251c102f50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.882+0000 7f2523600640 1 --2- 172.21.15.97:0/2161205515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f251c1018e0 0x7f251c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.882+0000 7f2523600640 1 -- 172.21.15.97:0/2161205515 >> 172.21.15.97:0/2161205515 conn(0x7f251c0fce70 msgr2=0x7f251c0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.882+0000 7f2523600640 1 -- 172.21.15.97:0/2161205515 shutdown_connections 2024-03-26T18:56:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.882+0000 7f2523600640 1 -- 172.21.15.97:0/2161205515 wait complete. 2024-03-26T18:56:06.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2523600640 1 Processor -- start 2024-03-26T18:56:06.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2523600640 1 -- start start 2024-03-26T18:56:06.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2523600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f251c1018e0 0x7f251c199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:06.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2523600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251c102ad0 0x7f251c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:06.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2523600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f251c103490 0x7f251c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2523600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f251c10d5b0 con 0x7f251c102ad0 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2523600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f251c10d430 con 0x7f251c103490 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2523600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f251c10d730 con 0x7f251c1018e0 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f251be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251c102ad0 0x7f251c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2520c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f251c1018e0 0x7f251c199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2521600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f251c103490 0x7f251c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2520c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f251c1018e0 0x7f251c199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51410/0 (socket says 172.21.15.97:51410) 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f2520c00640 1 -- 172.21.15.97:0/1960187022 learned_addr learned my addr 172.21.15.97:0/1960187022 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.883+0000 7f251be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251c102ad0 0x7f251c19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55692/0 (socket says 172.21.15.97:55692) 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2521600640 1 -- 172.21.15.97:0/1960187022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f251c1018e0 msgr2=0x7f251c199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2521600640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f251c1018e0 0x7f251c199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:06.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2521600640 1 -- 172.21.15.97:0/1960187022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251c102ad0 msgr2=0x7f251c19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:06.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2521600640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251c102ad0 0x7f251c19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:06.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2521600640 1 -- 172.21.15.97:0/1960187022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c1a1a30 con 0x7f251c103490 2024-03-26T18:56:06.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2520c00640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f251c1018e0 0x7f251c199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:06.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2521600640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f251c103490 0x7f251c1a13c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2514009fd0 tx=0x7f251400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:06.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2519600640 1 -- 172.21.15.97:0/1960187022 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251400ce20 con 0x7f251c103490 2024-03-26T18:56:06.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2523600640 1 -- 172.21.15.97:0/1960187022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c1a1d20 con 0x7f251c103490 2024-03-26T18:56:06.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2519600640 1 -- 172.21.15.97:0/1960187022 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2514005020 con 0x7f251c103490 2024-03-26T18:56:06.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.884+0000 7f2519600640 1 -- 172.21.15.97:0/1960187022 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514010720 con 0x7f251c103490 2024-03-26T18:56:06.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.885+0000 7f2523600640 1 -- 172.21.15.97:0/1960187022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1a2190 con 0x7f251c103490 2024-03-26T18:56:06.880 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:56:06 smithi144 podman[66963]: 2024-03-26 18:56:06.482020475 +0000 UTC m=+0.381805886 container start a28535586c94ea935e3b1c8ff7d2dc7eaa5d237a526b22398cf97ef34215217d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:56:06.880 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:56:06 smithi144 ceph-mds[66982]: -- 172.21.15.144:0/3694972258 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570bcb7ba40 con 0x5570bcb4e800 2024-03-26T18:56:06.880 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:56:06 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i[66978]: starting mds.i at 2024-03-26T18:56:06.880 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:56:06 smithi144 bash[66963]: a28535586c94ea935e3b1c8ff7d2dc7eaa5d237a526b22398cf97ef34215217d 2024-03-26T18:56:06.880 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:56:06 smithi144 systemd[1]: Started Ceph mds.i for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:56:06.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.886+0000 7f2523600640 1 -- 172.21.15.97:0/1960187022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f251c1a2420 con 0x7f251c103490 2024-03-26T18:56:06.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.887+0000 7f2519600640 1 -- 172.21.15.97:0/1960187022 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f2514005540 con 0x7f251c103490 2024-03-26T18:56:06.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.889+0000 7f2519600640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f25040a2f20 0x7f25040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:06.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.889+0000 7f2519600640 1 -- 172.21.15.97:0/1960187022 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f25140cf6b0 con 0x7f251c103490 2024-03-26T18:56:06.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.889+0000 7f2520c00640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f25040a2f20 0x7f25040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:06.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.890+0000 7f2520c00640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f25040a2f20 0x7f25040a53e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25100058c0 tx=0x7f2510005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:06.893+0000 7f2519600640 1 -- 172.21.15.97:0/1960187022 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f251409c8e0 con 0x7f251c103490 2024-03-26T18:56:07.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.133+0000 7f2523600640 1 -- 172.21.15.97:0/1960187022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f251c1a25f0 con 0x7f251c103490 2024-03-26T18:56:07.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.134+0000 7f2519600640 1 -- 172.21.15.97:0/1960187022 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f251409c1e0 con 0x7f251c103490 2024-03-26T18:56:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:07.127 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-26T18:56:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 10 2024-03-26T18:56:07.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.137+0000 7f2523600640 1 -- 172.21.15.97:0/1960187022 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f25040a2f20 msgr2=0x7f25040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:07.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.137+0000 7f2523600640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f25040a2f20 0x7f25040a53e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25100058c0 tx=0x7f2510005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:07.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.137+0000 7f2523600640 1 -- 172.21.15.97:0/1960187022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f251c103490 msgr2=0x7f251c1a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.137+0000 7f2523600640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f251c103490 0x7f251c1a13c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2514009fd0 tx=0x7f251400ef10 comp rx=0 tx=0).stop 2024-03-26T18:56:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.138+0000 7f2523600640 1 -- 172.21.15.97:0/1960187022 shutdown_connections 2024-03-26T18:56:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.138+0000 7f2523600640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f25040a2f20 0x7f25040a53e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.138+0000 7f2523600640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f251c103490 0x7f251c1a13c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.138+0000 7f2523600640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f251c102ad0 0x7f251c19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.138+0000 7f2523600640 1 --2- 172.21.15.97:0/1960187022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f251c1018e0 0x7f251c199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.138+0000 7f2523600640 1 -- 172.21.15.97:0/1960187022 >> 172.21.15.97:0/1960187022 conn(0x7f251c0fce70 msgr2=0x7f251c073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.138+0000 7f2523600640 1 -- 172.21.15.97:0/1960187022 shutdown_connections 2024-03-26T18:56:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:07.138+0000 7f2523600640 1 -- 172.21.15.97:0/1960187022 wait complete. 2024-03-26T18:56:07.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:07.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:07.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:07.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:07.846 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:07.847 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:07.847 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: Deploying daemon mds.g on smithi097 2024-03-26T18:56:07.847 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:07.847 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] up:boot 2024-03-26T18:56:07.847 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: fsmap 9 up:standby 2024-03-26T18:56:07.847 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:56:07.847 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:07 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1960187022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:07.889 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:07.889 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:07.889 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:07.889 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:07.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:07.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:07.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: Deploying daemon mds.g on smithi097 2024-03-26T18:56:07.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:07.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] up:boot 2024-03-26T18:56:07.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: fsmap 9 up:standby 2024-03-26T18:56:07.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:56:07.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:07 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1960187022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:07.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:07.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:07.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:07.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:07.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:07.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: Deploying daemon mds.g on smithi097 2024-03-26T18:56:07.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:07.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] up:boot 2024-03-26T18:56:07.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: fsmap 9 up:standby 2024-03-26T18:56:07.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:56:07.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:07 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1960187022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:09.149 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:56:09.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.535+0000 7f8463200640 1 -- 172.21.15.97:0/3061651662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84641095c0 msgr2=0x7f846410ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:09.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.535+0000 7f8463200640 1 --2- 172.21.15.97:0/3061651662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84641095c0 0x7f846410ba60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f845c009be0 tx=0x7f845c02bf80 comp rx=0 tx=0).stop 2024-03-26T18:56:09.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.535+0000 7f8463200640 1 -- 172.21.15.97:0/3061651662 shutdown_connections 2024-03-26T18:56:09.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.535+0000 7f8463200640 1 --2- 172.21.15.97:0/3061651662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84641095c0 0x7f846410ba60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:09.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.535+0000 7f8463200640 1 --2- 172.21.15.97:0/3061651662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84640742c0 0x7f8464074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:09.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.535+0000 7f8463200640 1 --2- 172.21.15.97:0/3061651662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84640758a0 0x7f8464073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:09.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.535+0000 7f8463200640 1 -- 172.21.15.97:0/3061651662 >> 172.21.15.97:0/3061651662 conn(0x7f84640fcca0 msgr2=0x7f84640ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:09.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.535+0000 7f8463200640 1 -- 172.21.15.97:0/3061651662 shutdown_connections 2024-03-26T18:56:09.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.535+0000 7f8463200640 1 -- 172.21.15.97:0/3061651662 wait complete. 2024-03-26T18:56:09.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.537+0000 7f8463200640 1 Processor -- start 2024-03-26T18:56:09.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.537+0000 7f8463200640 1 -- start start 2024-03-26T18:56:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.537+0000 7f8463200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84640742c0 0x7f8464199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.537+0000 7f8463200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84640758a0 0x7f846419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.537+0000 7f8463200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84641095c0 0x7f84641a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.537+0000 7f8463200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f846410f5b0 con 0x7f84640758a0 2024-03-26T18:56:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.537+0000 7f8463200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f846410f430 con 0x7f84640742c0 2024-03-26T18:56:09.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.537+0000 7f8463200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f846410f730 con 0x7f84641095c0 2024-03-26T18:56:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8461e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84640742c0 0x7f8464199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8461e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84640742c0 0x7f8464199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34606/0 (socket says 172.21.15.97:34606) 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8461e00640 1 -- 172.21.15.97:0/1008515009 learned_addr learned my addr 172.21.15.97:0/1008515009 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8459400640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84640758a0 0x7f846419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8462800640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84641095c0 0x7f84641a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8461e00640 1 -- 172.21.15.97:0/1008515009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84641095c0 msgr2=0x7f84641a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8461e00640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84641095c0 0x7f84641a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8461e00640 1 -- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84640758a0 msgr2=0x7f846419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8461e00640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84640758a0 0x7f846419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8461e00640 1 -- 172.21.15.97:0/1008515009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84641a17e0 con 0x7f84640742c0 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8459400640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84640758a0 0x7f846419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f8461e00640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84640742c0 0x7f8464199bc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f845400e9c0 tx=0x7f845400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.538+0000 7f845b400640 1 -- 172.21.15.97:0/1008515009 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845400ccc0 con 0x7f84640742c0 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.539+0000 7f8463200640 1 -- 172.21.15.97:0/1008515009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84641a1ad0 con 0x7f84640742c0 2024-03-26T18:56:09.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.539+0000 7f845b400640 1 -- 172.21.15.97:0/1008515009 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8454005500 con 0x7f84640742c0 2024-03-26T18:56:09.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.539+0000 7f845b400640 1 -- 172.21.15.97:0/1008515009 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454010430 con 0x7f84640742c0 2024-03-26T18:56:09.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.539+0000 7f8463200640 1 -- 172.21.15.97:0/1008515009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84641a2070 con 0x7f84640742c0 2024-03-26T18:56:09.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.540+0000 7f8463200640 1 -- 172.21.15.97:0/1008515009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8464108380 con 0x7f84640742c0 2024-03-26T18:56:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.542+0000 7f845b400640 1 -- 172.21.15.97:0/1008515009 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f84540046c0 con 0x7f84640742c0 2024-03-26T18:56:09.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.543+0000 7f845b400640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f84480a2f20 0x7f84480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:09.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.543+0000 7f845b400640 1 -- 172.21.15.97:0/1008515009 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f84540cf360 con 0x7f84640742c0 2024-03-26T18:56:09.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.543+0000 7f8459400640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f84480a2f20 0x7f84480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:09.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.544+0000 7f8459400640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f84480a2f20 0x7f84480a53e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f846419b0d0 tx=0x7f844c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.547+0000 7f845b400640 1 -- 172.21.15.97:0/1008515009 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f845409c590 con 0x7f84640742c0 2024-03-26T18:56:09.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.790+0000 7f8463200640 1 -- 172.21.15.97:0/1008515009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f846402d240 con 0x7f84640742c0 2024-03-26T18:56:09.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.791+0000 7f845b400640 1 -- 172.21.15.97:0/1008515009 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f846402d240 con 0x7f84640742c0 2024-03-26T18:56:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:09.783 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-26T18:56:09.784 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 10 2024-03-26T18:56:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.793+0000 7f8463200640 1 -- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f84480a2f20 msgr2=0x7f84480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.793+0000 7f8463200640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f84480a2f20 0x7f84480a53e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f846419b0d0 tx=0x7f844c005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.793+0000 7f8463200640 1 -- 172.21.15.97:0/1008515009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84640742c0 msgr2=0x7f8464199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.793+0000 7f8463200640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84640742c0 0x7f8464199bc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f845400e9c0 tx=0x7f845400ee80 comp rx=0 tx=0).stop 2024-03-26T18:56:09.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.794+0000 7f8463200640 1 -- 172.21.15.97:0/1008515009 shutdown_connections 2024-03-26T18:56:09.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.794+0000 7f8463200640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f84480a2f20 0x7f84480a53e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:09.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.794+0000 7f8463200640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84641095c0 0x7f84641a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:09.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.794+0000 7f8463200640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84640758a0 0x7f846419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:09.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.794+0000 7f8463200640 1 --2- 172.21.15.97:0/1008515009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84640742c0 0x7f8464199bc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:09.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.794+0000 7f8463200640 1 -- 172.21.15.97:0/1008515009 >> 172.21.15.97:0/1008515009 conn(0x7f84640fcca0 msgr2=0x7f8464105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:09.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.794+0000 7f8463200640 1 -- 172.21.15.97:0/1008515009 shutdown_connections 2024-03-26T18:56:09.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:09.794+0000 7f8463200640 1 -- 172.21.15.97:0/1008515009 wait complete. 2024-03-26T18:56:10.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:09 smithi097 ceph-mon[31955]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:10.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:09 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1008515009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:09 smithi105 ceph-mon[36270]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:10.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:09 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1008515009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:10.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:09 smithi144 ceph-mon[36053]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:10.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:09 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1008515009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:10.853 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 26 18:56:10 smithi097 systemd[1]: Started Ceph mds.g for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:56:11.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:56:11.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:11.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:11.960 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:11.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:11.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:11.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:11.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: Deploying daemon mds.a on smithi097 2024-03-26T18:56:11.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:11.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.97:6838/940320803,v1:172.21.15.97:6839/940320803] up:boot 2024-03-26T18:56:11.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: fsmap 10 up:standby 2024-03-26T18:56:11.961 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:56:12.013 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:12.013 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:12.013 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:12.013 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:12.014 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:12.014 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:12.014 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: Deploying daemon mds.a on smithi097 2024-03-26T18:56:12.014 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:12.014 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.97:6838/940320803,v1:172.21.15.97:6839/940320803] up:boot 2024-03-26T18:56:12.014 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: fsmap 10 up:standby 2024-03-26T18:56:12.014 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:56:12.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.025+0000 7f542b800640 1 -- 172.21.15.97:0/2226503765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54241034b0 msgr2=0x7f5424109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:12.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.025+0000 7f542b800640 1 --2- 172.21.15.97:0/2226503765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54241034b0 0x7f5424109d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f541c009ca0 tx=0x7f541c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.025+0000 7f542b800640 1 -- 172.21.15.97:0/2226503765 shutdown_connections 2024-03-26T18:56:12.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.025+0000 7f542b800640 1 --2- 172.21.15.97:0/2226503765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54241034b0 0x7f5424109d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.025+0000 7f542b800640 1 --2- 172.21.15.97:0/2226503765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5424102af0 0x7f5424102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.025+0000 7f542b800640 1 --2- 172.21.15.97:0/2226503765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424101900 0x7f5424101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.025+0000 7f542b800640 1 -- 172.21.15.97:0/2226503765 >> 172.21.15.97:0/2226503765 conn(0x7f54240fce70 msgr2=0x7f54240ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:12.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.027+0000 7f542b800640 1 -- 172.21.15.97:0/2226503765 shutdown_connections 2024-03-26T18:56:12.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.028+0000 7f542b800640 1 -- 172.21.15.97:0/2226503765 wait complete. 2024-03-26T18:56:12.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.029+0000 7f542b800640 1 Processor -- start 2024-03-26T18:56:12.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.029+0000 7f542b800640 1 -- start start 2024-03-26T18:56:12.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.029+0000 7f542b800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5424101900 0x7f5424197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.029+0000 7f542b800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424102af0 0x7f54241980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.029+0000 7f542b800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54241034b0 0x7f542419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.029+0000 7f542b800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f542410d580 con 0x7f54241034b0 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.029+0000 7f542b800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f542410d400 con 0x7f5424101900 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.029+0000 7f542b800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f542410d700 con 0x7f5424102af0 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5429800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54241034b0 0x7f542419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5429800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54241034b0 0x7f542419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39440/0 (socket says 172.21.15.97:39440) 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5429800640 1 -- 172.21.15.97:0/975953108 learned_addr learned my addr 172.21.15.97:0/975953108 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5428e00640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5424101900 0x7f5424197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5423e00640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424102af0 0x7f54241980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5429800640 1 -- 172.21.15.97:0/975953108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424102af0 msgr2=0x7f54241980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:12.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5429800640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424102af0 0x7f54241980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5429800640 1 -- 172.21.15.97:0/975953108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5424101900 msgr2=0x7f5424197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:12.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5429800640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5424101900 0x7f5424197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5429800640 1 -- 172.21.15.97:0/975953108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5424073b70 con 0x7f54241034b0 2024-03-26T18:56:12.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.030+0000 7f5428e00640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5424101900 0x7f5424197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:12.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.031+0000 7f5429800640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54241034b0 0x7f542419f140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f541c02c630 tx=0x7f541c032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:12.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.031+0000 7f5421600640 1 -- 172.21.15.97:0/975953108 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c041020 con 0x7f54241034b0 2024-03-26T18:56:12.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.031+0000 7f5421600640 1 -- 172.21.15.97:0/975953108 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541c034cd0 con 0x7f54241034b0 2024-03-26T18:56:12.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.031+0000 7f5421600640 1 -- 172.21.15.97:0/975953108 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c039040 con 0x7f54241034b0 2024-03-26T18:56:12.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.031+0000 7f542b800640 1 -- 172.21.15.97:0/975953108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5424073e00 con 0x7f54241034b0 2024-03-26T18:56:12.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.031+0000 7f542b800640 1 -- 172.21.15.97:0/975953108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54240742a0 con 0x7f54241034b0 2024-03-26T18:56:12.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.032+0000 7f542b800640 1 -- 172.21.15.97:0/975953108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f542402d240 con 0x7f54241034b0 2024-03-26T18:56:12.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.036+0000 7f5421600640 1 -- 172.21.15.97:0/975953108 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f541c044020 con 0x7f54241034b0 2024-03-26T18:56:12.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.037+0000 7f5421600640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54080a2fa0 0x7f54080a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:12.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.038+0000 7f5421600640 1 -- 172.21.15.97:0/975953108 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f541c0f0d00 con 0x7f54241034b0 2024-03-26T18:56:12.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.038+0000 7f5428e00640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54080a2fa0 0x7f54080a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:12.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.038+0000 7f5428e00640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54080a2fa0 0x7f54080a5460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5424102950 tx=0x7f5414005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:12.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.042+0000 7f5421600640 1 -- 172.21.15.97:0/975953108 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f541c04b020 con 0x7f54241034b0 2024-03-26T18:56:12.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:12.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:12.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:12.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:12.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:12.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:12.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: Deploying daemon mds.a on smithi097 2024-03-26T18:56:12.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:12.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.97:6838/940320803,v1:172.21.15.97:6839/940320803] up:boot 2024-03-26T18:56:12.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: fsmap 10 up:standby 2024-03-26T18:56:12.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:56:12.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.273+0000 7f542b800640 1 -- 172.21.15.97:0/975953108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54241087d0 con 0x7f54241034b0 2024-03-26T18:56:12.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.275+0000 7f5421600640 1 -- 172.21.15.97:0/975953108 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f541c0be060 con 0x7f54241034b0 2024-03-26T18:56:12.267 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-26T18:56:12.268 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 11 2024-03-26T18:56:12.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.277+0000 7f542b800640 1 -- 172.21.15.97:0/975953108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54080a2fa0 msgr2=0x7f54080a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:12.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.277+0000 7f542b800640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54080a2fa0 0x7f54080a5460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5424102950 tx=0x7f5414005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:12.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.277+0000 7f542b800640 1 -- 172.21.15.97:0/975953108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54241034b0 msgr2=0x7f542419f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:12.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.277+0000 7f542b800640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54241034b0 0x7f542419f140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f541c02c630 tx=0x7f541c032ea0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.277+0000 7f542b800640 1 -- 172.21.15.97:0/975953108 shutdown_connections 2024-03-26T18:56:12.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.277+0000 7f542b800640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54080a2fa0 0x7f54080a5460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.277+0000 7f542b800640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54241034b0 0x7f542419f140 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.277+0000 7f542b800640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424102af0 0x7f54241980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.277+0000 7f542b800640 1 --2- 172.21.15.97:0/975953108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5424101900 0x7f5424197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:12.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.277+0000 7f542b800640 1 -- 172.21.15.97:0/975953108 >> 172.21.15.97:0/975953108 conn(0x7f54240fce70 msgr2=0x7f5424105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:12.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.278+0000 7f542b800640 1 -- 172.21.15.97:0/975953108 shutdown_connections 2024-03-26T18:56:12.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:12.278+0000 7f542b800640 1 -- 172.21.15.97:0/975953108 wait complete. 2024-03-26T18:56:13.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:12 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/975953108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:13.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:12 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/975953108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:13.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:12 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/975953108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:13.818 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:13 smithi097 ceph-mon[31955]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:14.103 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 18:56:13 smithi097 systemd[1]: Starting Ceph mds.a for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:56:14.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:13 smithi144 ceph-mon[36053]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:14.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:13 smithi105 ceph-mon[36270]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:14.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:56:14.467 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 18:56:14 smithi097 podman[71162]: 2024-03-26 18:56:14.084707743 +0000 UTC m=+0.032436246 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:56:14.468 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 18:56:14 smithi097 podman[71162]: 2024-03-26 18:56:14.22034123 +0000 UTC m=+0.168069738 container create 99cab503a0a8cdb3cf9f756634304f28eaab1ee49139c04d5598dd7b03c40965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:56:14.468 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 18:56:14 smithi097 podman[71162]: 2024-03-26 18:56:14.474434199 +0000 UTC m=+0.422162771 container init 99cab503a0a8cdb3cf9f756634304f28eaab1ee49139c04d5598dd7b03c40965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux ) 2024-03-26T18:56:14.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.508+0000 7f07d4600640 1 -- 172.21.15.97:0/1543344207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc1056b0 msgr2=0x7f07cc10bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:14.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.508+0000 7f07d4600640 1 --2- 172.21.15.97:0/1543344207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc1056b0 0x7f07cc10bf40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009ca0 tx=0x7f07c402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.508+0000 7f07d4600640 1 -- 172.21.15.97:0/1543344207 shutdown_connections 2024-03-26T18:56:14.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.508+0000 7f07d4600640 1 --2- 172.21.15.97:0/1543344207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc1056b0 0x7f07cc10bf40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.508+0000 7f07d4600640 1 --2- 172.21.15.97:0/1543344207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07cc104cf0 0x7f07cc105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.508+0000 7f07d4600640 1 --2- 172.21.15.97:0/1543344207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc103b00 0x7f07cc103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.508+0000 7f07d4600640 1 -- 172.21.15.97:0/1543344207 >> 172.21.15.97:0/1543344207 conn(0x7f07cc0ff0b0 msgr2=0x7f07cc1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:14.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.508+0000 7f07d4600640 1 -- 172.21.15.97:0/1543344207 shutdown_connections 2024-03-26T18:56:14.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.508+0000 7f07d4600640 1 -- 172.21.15.97:0/1543344207 wait complete. 2024-03-26T18:56:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d4600640 1 Processor -- start 2024-03-26T18:56:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d4600640 1 -- start start 2024-03-26T18:56:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d4600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc103b00 0x7f07cc199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d4600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07cc104cf0 0x7f07cc19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d4600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc1056b0 0x7f07cc1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d4600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc10f760 con 0x7f07cc103b00 2024-03-26T18:56:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d4600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc10f5e0 con 0x7f07cc104cf0 2024-03-26T18:56:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d4600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc10f8e0 con 0x7f07cc1056b0 2024-03-26T18:56:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d1c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc103b00 0x7f07cc199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d2600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc1056b0 0x7f07cc1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d1c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc103b00 0x7f07cc199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39460/0 (socket says 172.21.15.97:39460) 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d1c00640 1 -- 172.21.15.97:0/2169058233 learned_addr learned my addr 172.21.15.97:0/2169058233 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.510+0000 7f07d1200640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07cc104cf0 0x7f07cc19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.511+0000 7f07d2600640 1 -- 172.21.15.97:0/2169058233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07cc104cf0 msgr2=0x7f07cc19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.511+0000 7f07d2600640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07cc104cf0 0x7f07cc19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.511+0000 7f07d2600640 1 -- 172.21.15.97:0/2169058233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc103b00 msgr2=0x7f07cc199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.511+0000 7f07d2600640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc103b00 0x7f07cc199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.511+0000 7f07d2600640 1 -- 172.21.15.97:0/2169058233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc1a19d0 con 0x7f07cc1056b0 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.511+0000 7f07d2600640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc1056b0 0x7f07cc1a1330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07c4037040 tx=0x7f07c4032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.511+0000 7f07ca000640 1 -- 172.21.15.97:0/2169058233 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4041020 con 0x7f07cc1056b0 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.511+0000 7f07ca000640 1 -- 172.21.15.97:0/2169058233 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c4034cd0 con 0x7f07cc1056b0 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.511+0000 7f07d4600640 1 -- 172.21.15.97:0/2169058233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc1a1c60 con 0x7f07cc1056b0 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.511+0000 7f07ca000640 1 -- 172.21.15.97:0/2169058233 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4039040 con 0x7f07cc1056b0 2024-03-26T18:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.512+0000 7f07d4600640 1 -- 172.21.15.97:0/2169058233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc1a2170 con 0x7f07cc1056b0 2024-03-26T18:56:14.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.513+0000 7f07d4600640 1 -- 172.21.15.97:0/2169058233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07cc10ae20 con 0x7f07cc1056b0 2024-03-26T18:56:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.515+0000 7f07ca000640 1 -- 172.21.15.97:0/2169058233 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f07c4044020 con 0x7f07cc1056b0 2024-03-26T18:56:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.516+0000 7f07ca000640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07b40a2fa0 0x7f07b40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.516+0000 7f07ca000640 1 -- 172.21.15.97:0/2169058233 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f07c40f0f30 con 0x7f07cc1056b0 2024-03-26T18:56:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.516+0000 7f07d1c00640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07b40a2fa0 0x7f07b40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.517+0000 7f07d1c00640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07b40a2fa0 0x7f07b40a5460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07cc19b290 tx=0x7f07c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:14.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.519+0000 7f07ca000640 1 -- 172.21.15.97:0/2169058233 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f07c411f740 con 0x7f07cc1056b0 2024-03-26T18:56:14.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.788+0000 7f07d4600640 1 -- 172.21.15.97:0/2169058233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07cc02d400 con 0x7f07cc1056b0 2024-03-26T18:56:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.789+0000 7f07ca000640 1 -- 172.21.15.97:0/2169058233 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f07c404b020 con 0x7f07cc1056b0 2024-03-26T18:56:14.782 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:14.782 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-03-26T18:56:14.782 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 12 2024-03-26T18:56:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.791+0000 7f07d4600640 1 -- 172.21.15.97:0/2169058233 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07b40a2fa0 msgr2=0x7f07b40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.791+0000 7f07d4600640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07b40a2fa0 0x7f07b40a5460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07cc19b290 tx=0x7f07c0005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.791+0000 7f07d4600640 1 -- 172.21.15.97:0/2169058233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc1056b0 msgr2=0x7f07cc1a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.791+0000 7f07d4600640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc1056b0 0x7f07cc1a1330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07c4037040 tx=0x7f07c4032ea0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.792+0000 7f07d4600640 1 -- 172.21.15.97:0/2169058233 shutdown_connections 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.792+0000 7f07d4600640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07b40a2fa0 0x7f07b40a5460 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.792+0000 7f07d4600640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc1056b0 0x7f07cc1a1330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.792+0000 7f07d4600640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07cc104cf0 0x7f07cc19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.792+0000 7f07d4600640 1 --2- 172.21.15.97:0/2169058233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc103b00 0x7f07cc199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.792+0000 7f07d4600640 1 -- 172.21.15.97:0/2169058233 >> 172.21.15.97:0/2169058233 conn(0x7f07cc0ff0b0 msgr2=0x7f07cc107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.792+0000 7f07d4600640 1 -- 172.21.15.97:0/2169058233 shutdown_connections 2024-03-26T18:56:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:14.792+0000 7f07d4600640 1 -- 172.21.15.97:0/2169058233 wait complete. 2024-03-26T18:56:14.801 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 18:56:14 smithi097 podman[71162]: 2024-03-26 18:56:14.485308245 +0000 UTC m=+0.433036752 container start 99cab503a0a8cdb3cf9f756634304f28eaab1ee49139c04d5598dd7b03c40965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-26T18:56:14.801 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 18:56:14 smithi097 ceph-mds[71201]: -- 172.21.15.97:0/1043837275 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a83aa7a40 con 0x556a83a7ac00 2024-03-26T18:56:14.801 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 18:56:14 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a[71193]: starting mds.a at 2024-03-26T18:56:14.801 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 18:56:14 smithi097 bash[71162]: 99cab503a0a8cdb3cf9f756634304f28eaab1ee49139c04d5598dd7b03c40965 2024-03-26T18:56:14.801 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 18:56:14 smithi097 systemd[1]: Started Ceph mds.a for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:56:16.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:16.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] up:boot 2024-03-26T18:56:16.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: fsmap 11 up:standby 2024-03-26T18:56:16.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:56:16.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:16.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:16.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:16.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:16.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:16.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: Deploying daemon mds.b on smithi105 2024-03-26T18:56:16.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2169058233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:16.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:15 smithi097 ceph-mon[31955]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:16.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:16.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] up:boot 2024-03-26T18:56:16.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: fsmap 11 up:standby 2024-03-26T18:56:16.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:56:16.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:16.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:16.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:16.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:16.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:16.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: Deploying daemon mds.b on smithi105 2024-03-26T18:56:16.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2169058233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:16.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:15 smithi144 ceph-mon[36053]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:16.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:16.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] up:boot 2024-03-26T18:56:16.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: fsmap 11 up:standby 2024-03-26T18:56:16.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:56:16.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:16.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:16.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T18:56:16.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T18:56:16.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:16.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: Deploying daemon mds.b on smithi105 2024-03-26T18:56:16.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2169058233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:16.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:15 smithi105 ceph-mon[36270]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:16.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:56:16.960 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:16 smithi105 systemd[1]: Starting Ceph mds.b for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T18:56:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.038+0000 7f8091200640 1 -- 172.21.15.97:0/2362235602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808c104cf0 msgr2=0x7f808c105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.038+0000 7f8091200640 1 --2- 172.21.15.97:0/2362235602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808c104cf0 0x7f808c105170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8074009900 tx=0x7f807402bc50 comp rx=0 tx=0).stop 2024-03-26T18:56:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.038+0000 7f8091200640 1 -- 172.21.15.97:0/2362235602 shutdown_connections 2024-03-26T18:56:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.038+0000 7f8091200640 1 --2- 172.21.15.97:0/2362235602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f808c1056b0 0x7f808c10bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.038+0000 7f8091200640 1 --2- 172.21.15.97:0/2362235602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808c104cf0 0x7f808c105170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.038+0000 7f8091200640 1 --2- 172.21.15.97:0/2362235602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f808c103b00 0x7f808c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.038+0000 7f8091200640 1 -- 172.21.15.97:0/2362235602 >> 172.21.15.97:0/2362235602 conn(0x7f808c0ff070 msgr2=0x7f808c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.038+0000 7f8091200640 1 -- 172.21.15.97:0/2362235602 shutdown_connections 2024-03-26T18:56:17.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.038+0000 7f8091200640 1 -- 172.21.15.97:0/2362235602 wait complete. 2024-03-26T18:56:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.039+0000 7f8091200640 1 Processor -- start 2024-03-26T18:56:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.039+0000 7f8091200640 1 -- start start 2024-03-26T18:56:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f8091200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f808c103b00 0x7f808c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f8091200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f808c104cf0 0x7f808c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f8091200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808c1056b0 0x7f808c1a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f8091200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f808c10f780 con 0x7f808c1056b0 2024-03-26T18:56:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f8091200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f808c10f600 con 0x7f808c104cf0 2024-03-26T18:56:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f8091200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f808c10f900 con 0x7f808c103b00 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f808a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808c1056b0 0x7f808c1a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f8081400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f808c104cf0 0x7f808c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f808a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808c1056b0 0x7f808c1a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39498/0 (socket says 172.21.15.97:39498) 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f808a800640 1 -- 172.21.15.97:0/139246747 learned_addr learned my addr 172.21.15.97:0/139246747 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f8089e00640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f808c103b00 0x7f808c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f808a800640 1 -- 172.21.15.97:0/139246747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f808c103b00 msgr2=0x7f808c199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f808a800640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f808c103b00 0x7f808c199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f808a800640 1 -- 172.21.15.97:0/139246747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f808c104cf0 msgr2=0x7f808c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f808a800640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f808c104cf0 0x7f808c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.040+0000 7f808a800640 1 -- 172.21.15.97:0/139246747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c1a1ac0 con 0x7f808c1056b0 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.041+0000 7f808a800640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808c1056b0 0x7f808c1a1450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f808400eb10 tx=0x7f808400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.041+0000 7f8083400640 1 -- 172.21.15.97:0/139246747 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084004b20 con 0x7f808c1056b0 2024-03-26T18:56:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.041+0000 7f8083400640 1 -- 172.21.15.97:0/139246747 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8084004cc0 con 0x7f808c1056b0 2024-03-26T18:56:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.041+0000 7f8083400640 1 -- 172.21.15.97:0/139246747 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084010760 con 0x7f808c1056b0 2024-03-26T18:56:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.041+0000 7f8091200640 1 -- 172.21.15.97:0/139246747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c1a1db0 con 0x7f808c1056b0 2024-03-26T18:56:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.041+0000 7f8091200640 1 -- 172.21.15.97:0/139246747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c1a2270 con 0x7f808c1056b0 2024-03-26T18:56:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.042+0000 7f8091200640 1 -- 172.21.15.97:0/139246747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c10aea0 con 0x7f808c1056b0 2024-03-26T18:56:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.044+0000 7f8083400640 1 -- 172.21.15.97:0/139246747 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f8084005020 con 0x7f808c1056b0 2024-03-26T18:56:17.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.045+0000 7f8083400640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80700a2f20 0x7f80700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:17.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.046+0000 7f8083400640 1 -- 172.21.15.97:0/139246747 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f80840cf690 con 0x7f808c1056b0 2024-03-26T18:56:17.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.046+0000 7f8089e00640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80700a2f20 0x7f80700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:17.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.046+0000 7f8089e00640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80700a2f20 0x7f80700a53e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f808c104b50 tx=0x7f807c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:17.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.050+0000 7f8083400640 1 -- 172.21.15.97:0/139246747 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f808409c8c0 con 0x7f808c1056b0 2024-03-26T18:56:17.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.289+0000 7f8091200640 1 -- 172.21.15.97:0/139246747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f808c02d240 con 0x7f808c1056b0 2024-03-26T18:56:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.290+0000 7f8083400640 1 -- 172.21.15.97:0/139246747 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f808409c1c0 con 0x7f808c1056b0 2024-03-26T18:56:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:17.283 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-03-26T18:56:17.284 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 12 2024-03-26T18:56:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 -- 172.21.15.97:0/139246747 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80700a2f20 msgr2=0x7f80700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80700a2f20 0x7f80700a53e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f808c104b50 tx=0x7f807c005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 -- 172.21.15.97:0/139246747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808c1056b0 msgr2=0x7f808c1a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808c1056b0 0x7f808c1a1450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f808400eb10 tx=0x7f808400c2d0 comp rx=0 tx=0).stop 2024-03-26T18:56:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 -- 172.21.15.97:0/139246747 shutdown_connections 2024-03-26T18:56:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80700a2f20 0x7f80700a53e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f808c1056b0 0x7f808c1a1450 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f808c104cf0 0x7f808c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 --2- 172.21.15.97:0/139246747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f808c103b00 0x7f808c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 -- 172.21.15.97:0/139246747 >> 172.21.15.97:0/139246747 conn(0x7f808c0ff070 msgr2=0x7f808c108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:17.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 -- 172.21.15.97:0/139246747 shutdown_connections 2024-03-26T18:56:17.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:17.293+0000 7f8091200640 1 -- 172.21.15.97:0/139246747 wait complete. 2024-03-26T18:56:17.334 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:17 smithi105 podman[66284]: 2024-03-26 18:56:16.973043566 +0000 UTC m=+0.019088140 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:56:17.334 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:17 smithi105 podman[66284]: 2024-03-26 18:56:17.089723797 +0000 UTC m=+0.135768357 container create ccc6801e72c38410dde60e5f5420a0f1b59ec2d9953d6e069fab6a49e6615f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:56:17.334 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:17 smithi105 podman[66284]: 2024-03-26 18:56:17.342349637 +0000 UTC m=+0.388394185 container init ccc6801e72c38410dde60e5f5420a0f1b59ec2d9953d6e069fab6a49e6615f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T18:56:17.710 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:17 smithi105 podman[66284]: 2024-03-26 18:56:17.344987236 +0000 UTC m=+0.391031796 container start ccc6801e72c38410dde60e5f5420a0f1b59ec2d9953d6e069fab6a49e6615f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-26T18:56:17.710 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:17 smithi105 ceph-mds[66303]: -- 172.21.15.105:0/1313805191 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56044759da40 con 0x560447570c00 2024-03-26T18:56:17.710 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:17 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b[66299]: starting mds.b at 2024-03-26T18:56:17.710 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:17 smithi105 bash[66284]: ccc6801e72c38410dde60e5f5420a0f1b59ec2d9953d6e069fab6a49e6615f6d 2024-03-26T18:56:17.710 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:17 smithi105 systemd[1]: Started Ceph mds.b for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T18:56:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:17 smithi105 ceph-mon[36270]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:17 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/139246747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:56:18.315 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:17 smithi144 ceph-mon[36053]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:18.316 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:17 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/139246747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:18.316 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.316 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.316 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.316 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.316 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.316 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:56:18.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:17 smithi097 ceph-mon[31955]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:18.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:17 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/139246747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:18.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:18.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:56:19.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:18 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] up:boot 2024-03-26T18:56:19.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:18 smithi105 ceph-mon[36270]: fsmap 12 up:standby 2024-03-26T18:56:19.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:18 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:56:19.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:18 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:19.221 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:18 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] up:boot 2024-03-26T18:56:19.221 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:18 smithi097 ceph-mon[31955]: fsmap 12 up:standby 2024-03-26T18:56:19.221 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:18 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:56:19.221 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:18 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:19.305 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:56:19.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:18 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] up:boot 2024-03-26T18:56:19.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:18 smithi144 ceph-mon[36053]: fsmap 12 up:standby 2024-03-26T18:56:19.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:18 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:56:19.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:18 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:19.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.515+0000 7f9fbe600640 1 -- 172.21.15.97:0/2224026484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb8102ad0 msgr2=0x7f9fb8102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:19.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.515+0000 7f9fbe600640 1 --2- 172.21.15.97:0/2224026484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb8102ad0 0x7f9fb8102f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009900 tx=0x7f9fa002bc40 comp rx=0 tx=0).stop 2024-03-26T18:56:19.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.515+0000 7f9fbe600640 1 -- 172.21.15.97:0/2224026484 shutdown_connections 2024-03-26T18:56:19.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.515+0000 7f9fbe600640 1 --2- 172.21.15.97:0/2224026484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb8103490 0x7f9fb8109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.515+0000 7f9fbe600640 1 --2- 172.21.15.97:0/2224026484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb8102ad0 0x7f9fb8102f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.515+0000 7f9fbe600640 1 --2- 172.21.15.97:0/2224026484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb81018e0 0x7f9fb8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.515+0000 7f9fbe600640 1 -- 172.21.15.97:0/2224026484 >> 172.21.15.97:0/2224026484 conn(0x7f9fb80fce70 msgr2=0x7f9fb80ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:19.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.515+0000 7f9fbe600640 1 -- 172.21.15.97:0/2224026484 shutdown_connections 2024-03-26T18:56:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.515+0000 7f9fbe600640 1 -- 172.21.15.97:0/2224026484 wait complete. 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.516+0000 7f9fbe600640 1 Processor -- start 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fbe600640 1 -- start start 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fbe600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb81018e0 0x7f9fb8199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fbe600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb8102ad0 0x7f9fb819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fbe600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb8103490 0x7f9fb81a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fbe600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb810d560 con 0x7f9fb81018e0 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fbe600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb810d3e0 con 0x7f9fb8102ad0 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fbe600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb810d6e0 con 0x7f9fb8103490 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb8102ad0 0x7f9fb819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fb7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb81018e0 0x7f9fb8199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb8102ad0 0x7f9fb819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:39726/0 (socket says 172.21.15.97:39726) 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fafe00640 1 -- 172.21.15.97:0/1506348898 learned_addr learned my addr 172.21.15.97:0/1506348898 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.517+0000 7f9fb7e00640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb8103490 0x7f9fb81a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.518+0000 7f9fafe00640 1 -- 172.21.15.97:0/1506348898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb8103490 msgr2=0x7f9fb81a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.518+0000 7f9fafe00640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb8103490 0x7f9fb81a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.518+0000 7f9fafe00640 1 -- 172.21.15.97:0/1506348898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb81018e0 msgr2=0x7f9fb8199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.518+0000 7f9fafe00640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb81018e0 0x7f9fb8199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.518+0000 7f9fafe00640 1 -- 172.21.15.97:0/1506348898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb81a19e0 con 0x7f9fb8102ad0 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.518+0000 7f9fafe00640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb8102ad0 0x7f9fb819a300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009230 tx=0x7f9fa0005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.518+0000 7f9fb4c00640 1 -- 172.21.15.97:0/1506348898 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0041070 con 0x7f9fb8102ad0 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.518+0000 7f9fb4c00640 1 -- 172.21.15.97:0/1506348898 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fa00044d0 con 0x7f9fb8102ad0 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.518+0000 7f9fb4c00640 1 -- 172.21.15.97:0/1506348898 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0034440 con 0x7f9fb8102ad0 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.518+0000 7f9fbe600640 1 -- 172.21.15.97:0/1506348898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb81a1c70 con 0x7f9fb8102ad0 2024-03-26T18:56:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.519+0000 7f9fbe600640 1 -- 172.21.15.97:0/1506348898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb81a2110 con 0x7f9fb8102ad0 2024-03-26T18:56:19.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.520+0000 7f9fbe600640 1 -- 172.21.15.97:0/1506348898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb81a22e0 con 0x7f9fb8102ad0 2024-03-26T18:56:19.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.521+0000 7f9fb4c00640 1 -- 172.21.15.97:0/1506348898 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f9fa0044070 con 0x7f9fb8102ad0 2024-03-26T18:56:19.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.523+0000 7f9fb4c00640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f9c0a2f20 0x7f9f9c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:19.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.523+0000 7f9fb4c00640 1 -- 172.21.15.97:0/1506348898 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9fa00f0c10 con 0x7f9fb8102ad0 2024-03-26T18:56:19.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.523+0000 7f9fb7400640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f9c0a2f20 0x7f9f9c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:19.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.524+0000 7f9fb7400640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f9c0a2f20 0x7f9f9c0a53e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8102930 tx=0x7f9fa8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.527+0000 7f9fb4c00640 1 -- 172.21.15.97:0/1506348898 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9fa00bde80 con 0x7f9fb8102ad0 2024-03-26T18:56:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.769+0000 7f9fbe600640 1 -- 172.21.15.97:0/1506348898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fb81a2670 con 0x7f9fb8102ad0 2024-03-26T18:56:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.770+0000 7f9fb4c00640 1 -- 172.21.15.97:0/1506348898 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7f9fa00bd780 con 0x7f9fb8102ad0 2024-03-26T18:56:19.763 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:19.763 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/2391073232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2391073232},{"type":"v1","addr":"172.21.15.105:6839","nonce":2391073232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-26T18:56:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 13 2024-03-26T18:56:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 -- 172.21.15.97:0/1506348898 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f9c0a2f20 msgr2=0x7f9f9c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f9c0a2f20 0x7f9f9c0a53e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8102930 tx=0x7f9fa8005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 -- 172.21.15.97:0/1506348898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb8102ad0 msgr2=0x7f9fb819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb8102ad0 0x7f9fb819a300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009230 tx=0x7f9fa0005d50 comp rx=0 tx=0).stop 2024-03-26T18:56:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 -- 172.21.15.97:0/1506348898 shutdown_connections 2024-03-26T18:56:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f9c0a2f20 0x7f9f9c0a53e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb8103490 0x7f9fb81a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb8102ad0 0x7f9fb819a300 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009230 tx=0x7f9fa0005d50 comp rx=0 tx=0).stop 2024-03-26T18:56:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 --2- 172.21.15.97:0/1506348898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb81018e0 0x7f9fb8199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 -- 172.21.15.97:0/1506348898 >> 172.21.15.97:0/1506348898 conn(0x7f9fb80fce70 msgr2=0x7f9fb8073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 -- 172.21.15.97:0/1506348898 shutdown_connections 2024-03-26T18:56:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.773+0000 7f9fbe600640 1 -- 172.21.15.97:0/1506348898 wait complete. 2024-03-26T18:56:19.782 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:56:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.978+0000 7f5207200640 1 -- 172.21.15.97:0/399712382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5208101730 msgr2=0x7f5208101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.978+0000 7f5207200640 1 --2- 172.21.15.97:0/399712382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5208101730 0x7f5208101b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009900 tx=0x7f51f802bc50 comp rx=0 tx=0).stop 2024-03-26T18:56:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.978+0000 7f5207200640 1 -- 172.21.15.97:0/399712382 shutdown_connections 2024-03-26T18:56:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.978+0000 7f5207200640 1 --2- 172.21.15.97:0/399712382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52081032e0 0x7f5208109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.978+0000 7f5207200640 1 --2- 172.21.15.97:0/399712382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5208102920 0x7f5208102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.978+0000 7f5207200640 1 --2- 172.21.15.97:0/399712382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5208101730 0x7f5208101b50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.978+0000 7f5207200640 1 -- 172.21.15.97:0/399712382 >> 172.21.15.97:0/399712382 conn(0x7f52080fcca0 msgr2=0x7f52080ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.978+0000 7f5207200640 1 -- 172.21.15.97:0/399712382 shutdown_connections 2024-03-26T18:56:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.978+0000 7f5207200640 1 -- 172.21.15.97:0/399712382 wait complete. 2024-03-26T18:56:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.979+0000 7f5207200640 1 Processor -- start 2024-03-26T18:56:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.979+0000 7f5207200640 1 -- start start 2024-03-26T18:56:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5207200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5208101730 0x7f5208075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5207200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5208102920 0x7f5208073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5207200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52081032e0 0x7f52080741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5207200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f520810d3d0 con 0x7f52081032e0 2024-03-26T18:56:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5207200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f520810d250 con 0x7f5208101730 2024-03-26T18:56:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5207200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f520810d550 con 0x7f5208102920 2024-03-26T18:56:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5205e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5208101730 0x7f5208075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5205e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5208101730 0x7f5208075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:39754/0 (socket says 172.21.15.97:39754) 2024-03-26T18:56:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5205e00640 1 -- 172.21.15.97:0/3024575218 learned_addr learned my addr 172.21.15.97:0/3024575218 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5206800640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52081032e0 0x7f52080741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5205400640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5208102920 0x7f5208073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5205e00640 1 -- 172.21.15.97:0/3024575218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5208102920 msgr2=0x7f5208073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5205e00640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5208102920 0x7f5208073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5205e00640 1 -- 172.21.15.97:0/3024575218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52081032e0 msgr2=0x7f52080741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5205e00640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52081032e0 0x7f52080741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.980+0000 7f5205e00640 1 -- 172.21.15.97:0/3024575218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5208074a80 con 0x7f5208101730 2024-03-26T18:56:19.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.981+0000 7f5205e00640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5208101730 0x7f5208075670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f51f802c260 tx=0x7f51f802c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:19.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.981+0000 7f51fea00640 1 -- 172.21.15.97:0/3024575218 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8041070 con 0x7f5208101730 2024-03-26T18:56:19.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.981+0000 7f51fea00640 1 -- 172.21.15.97:0/3024575218 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f8004450 con 0x7f5208101730 2024-03-26T18:56:19.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.981+0000 7f51fea00640 1 -- 172.21.15.97:0/3024575218 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8003e20 con 0x7f5208101730 2024-03-26T18:56:19.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.981+0000 7f5207200640 1 -- 172.21.15.97:0/3024575218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52081a1b60 con 0x7f5208101730 2024-03-26T18:56:19.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.981+0000 7f5207200640 1 -- 172.21.15.97:0/3024575218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52081a2060 con 0x7f5208101730 2024-03-26T18:56:19.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.983+0000 7f5207200640 1 -- 172.21.15.97:0/3024575218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5208108a50 con 0x7f5208101730 2024-03-26T18:56:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.984+0000 7f51fea00640 1 -- 172.21.15.97:0/3024575218 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f51f8044070 con 0x7f5208101730 2024-03-26T18:56:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.985+0000 7f51fea00640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f51ec0a2fa0 0x7f51ec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.986+0000 7f51fea00640 1 -- 172.21.15.97:0/3024575218 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f51f80f0960 con 0x7f5208101730 2024-03-26T18:56:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.986+0000 7f5205400640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f51ec0a2fa0 0x7f51ec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.986+0000 7f5205400640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f51ec0a2fa0 0x7f51ec0a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51f40095f0 tx=0x7f51f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:19.990+0000 7f51fea00640 1 -- 172.21.15.97:0/3024575218 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f51f80bdc30 con 0x7f5208101730 2024-03-26T18:56:20.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:19 smithi097 ceph-mon[31955]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:20.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:19 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1506348898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.197+0000 7f5207200640 1 -- 172.21.15.97:0/3024575218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f520802d3f0 con 0x7f5208101730 2024-03-26T18:56:20.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.198+0000 7f51fea00640 1 -- 172.21.15.97:0/3024575218 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18431 (secure 0 0 0) 0x7f51f80bd530 con 0x7f5208101730 2024-03-26T18:56:20.191 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:20.191 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":81,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:55:27.022168+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:56:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 -- 172.21.15.97:0/3024575218 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f51ec0a2fa0 msgr2=0x7f51ec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f51ec0a2fa0 0x7f51ec0a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51f40095f0 tx=0x7f51f4005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 -- 172.21.15.97:0/3024575218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5208101730 msgr2=0x7f5208075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:20.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5208101730 0x7f5208075670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f51f802c260 tx=0x7f51f802c670 comp rx=0 tx=0).stop 2024-03-26T18:56:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 -- 172.21.15.97:0/3024575218 shutdown_connections 2024-03-26T18:56:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f51ec0a2fa0 0x7f51ec0a5460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52081032e0 0x7f52080741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5208102920 0x7f5208073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 --2- 172.21.15.97:0/3024575218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5208101730 0x7f5208075670 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 -- 172.21.15.97:0/3024575218 >> 172.21.15.97:0/3024575218 conn(0x7f52080fcca0 msgr2=0x7f5208105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 -- 172.21.15.97:0/3024575218 shutdown_connections 2024-03-26T18:56:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.201+0000 7f5207200640 1 -- 172.21.15.97:0/3024575218 wait complete. 2024-03-26T18:56:20.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T18:51:11.459500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T18:56:20.210 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:56:20.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:19 smithi144 ceph-mon[36053]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:20.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:19 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1506348898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.427+0000 7ff1ebc00640 1 -- 172.21.15.97:0/2321753414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e4074320 msgr2=0x7ff1e40747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.427+0000 7ff1ebc00640 1 --2- 172.21.15.97:0/2321753414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e4074320 0x7ff1e40747a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009900 tx=0x7ff1d002bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.427+0000 7ff1ebc00640 1 -- 172.21.15.97:0/2321753414 shutdown_connections 2024-03-26T18:56:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.427+0000 7ff1ebc00640 1 --2- 172.21.15.97:0/2321753414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e4109790 0x7ff1e410bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.427+0000 7ff1ebc00640 1 --2- 172.21.15.97:0/2321753414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e4074320 0x7ff1e40747a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.427+0000 7ff1ebc00640 1 --2- 172.21.15.97:0/2321753414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4075900 0x7ff1e4073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.427+0000 7ff1ebc00640 1 -- 172.21.15.97:0/2321753414 >> 172.21.15.97:0/2321753414 conn(0x7ff1e40fce40 msgr2=0x7ff1e40ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.427+0000 7ff1ebc00640 1 -- 172.21.15.97:0/2321753414 shutdown_connections 2024-03-26T18:56:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.427+0000 7ff1ebc00640 1 -- 172.21.15.97:0/2321753414 wait complete. 2024-03-26T18:56:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.428+0000 7ff1ebc00640 1 Processor -- start 2024-03-26T18:56:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.428+0000 7ff1ebc00640 1 -- start start 2024-03-26T18:56:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.428+0000 7ff1ebc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4074320 0x7ff1e4199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.428+0000 7ff1ebc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e4075900 0x7ff1e419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.428+0000 7ff1ebc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e4109790 0x7ff1e41a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.428+0000 7ff1ebc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410f730 con 0x7ff1e4075900 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.428+0000 7ff1ebc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410f5b0 con 0x7ff1e4074320 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.428+0000 7ff1ebc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410f8b0 con 0x7ff1e4109790 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e4075900 0x7ff1e419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e4075900 0x7ff1e419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39562/0 (socket says 172.21.15.97:39562) 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e3e00640 1 -- 172.21.15.97:0/4100096976 learned_addr learned my addr 172.21.15.97:0/4100096976 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e9200640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4074320 0x7ff1e4199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e9c00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e4109790 0x7ff1e41a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e3e00640 1 -- 172.21.15.97:0/4100096976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e4109790 msgr2=0x7ff1e41a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e3e00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e4109790 0x7ff1e41a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e3e00640 1 -- 172.21.15.97:0/4100096976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4074320 msgr2=0x7ff1e4199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e3e00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4074320 0x7ff1e4199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e3e00640 1 -- 172.21.15.97:0/4100096976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e41a1a00 con 0x7ff1e4075900 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e9c00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e4109790 0x7ff1e41a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.429+0000 7ff1e9200640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4074320 0x7ff1e4199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.430+0000 7ff1e3e00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e4075900 0x7ff1e419a370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009230 tx=0x7ff1d002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.430+0000 7ff1e1600640 1 -- 172.21.15.97:0/4100096976 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0041070 con 0x7ff1e4075900 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.430+0000 7ff1e1600640 1 -- 172.21.15.97:0/4100096976 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d0004450 con 0x7ff1e4075900 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.430+0000 7ff1e1600640 1 -- 172.21.15.97:0/4100096976 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0003e20 con 0x7ff1e4075900 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.430+0000 7ff1ebc00640 1 -- 172.21.15.97:0/4100096976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e41a1c90 con 0x7ff1e4075900 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.430+0000 7ff1ebc00640 1 -- 172.21.15.97:0/4100096976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e41a2130 con 0x7ff1e4075900 2024-03-26T18:56:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.432+0000 7ff1ebc00640 1 -- 172.21.15.97:0/4100096976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e41084c0 con 0x7ff1e4075900 2024-03-26T18:56:20.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.433+0000 7ff1e1600640 1 -- 172.21.15.97:0/4100096976 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7ff1d0044070 con 0x7ff1e4075900 2024-03-26T18:56:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.436+0000 7ff1e1600640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff1cc0a2fa0 0x7ff1cc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.437+0000 7ff1e1600640 1 -- 172.21.15.97:0/4100096976 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff1d00f0a90 con 0x7ff1e4075900 2024-03-26T18:56:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.437+0000 7ff1e9200640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff1cc0a2fa0 0x7ff1cc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.437+0000 7ff1e9200640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff1cc0a2fa0 0x7ff1cc0a5460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009fc0 tx=0x7ff1d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:20.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.440+0000 7ff1e1600640 1 -- 172.21.15.97:0/4100096976 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff1d00bd670 con 0x7ff1e4075900 2024-03-26T18:56:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:19 smithi105 ceph-mon[36270]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:19 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1506348898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.660+0000 7ff1ebc00640 1 -- 172.21.15.97:0/4100096976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff1e4062560 con 0x7ff1e4075900 2024-03-26T18:56:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.661+0000 7ff1e1600640 1 -- 172.21.15.97:0/4100096976 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7ff1d00c1710 con 0x7ff1e4075900 2024-03-26T18:56:20.654 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-26T18:56:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 -- 172.21.15.97:0/4100096976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff1cc0a2fa0 msgr2=0x7ff1cc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff1cc0a2fa0 0x7ff1cc0a5460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009fc0 tx=0x7ff1d8005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 -- 172.21.15.97:0/4100096976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e4075900 msgr2=0x7ff1e419a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e4075900 0x7ff1e419a370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009230 tx=0x7ff1d002c650 comp rx=0 tx=0).stop 2024-03-26T18:56:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 -- 172.21.15.97:0/4100096976 shutdown_connections 2024-03-26T18:56:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff1cc0a2fa0 0x7ff1cc0a5460 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e4109790 0x7ff1e41a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e4075900 0x7ff1e419a370 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 --2- 172.21.15.97:0/4100096976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4074320 0x7ff1e4199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 -- 172.21.15.97:0/4100096976 >> 172.21.15.97:0/4100096976 conn(0x7ff1e40fce40 msgr2=0x7ff1e4105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 -- 172.21.15.97:0/4100096976 shutdown_connections 2024-03-26T18:56:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.664+0000 7ff1ebc00640 1 -- 172.21.15.97:0/4100096976 wait complete. 2024-03-26T18:56:20.676 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-26T18:56:20.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.865+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2933727228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab0103b00 msgr2=0x7f1ab0103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:20.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.865+0000 7f1ab5e00640 1 --2- 172.21.15.97:0/2933727228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab0103b00 0x7f1ab0103f20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009900 tx=0x7f1aa002bc40 comp rx=0 tx=0).stop 2024-03-26T18:56:20.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.865+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2933727228 shutdown_connections 2024-03-26T18:56:20.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.865+0000 7f1ab5e00640 1 --2- 172.21.15.97:0/2933727228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ab01056b0 0x7f1ab010bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.865+0000 7f1ab5e00640 1 --2- 172.21.15.97:0/2933727228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0104cf0 0x7f1ab0105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.865+0000 7f1ab5e00640 1 --2- 172.21.15.97:0/2933727228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab0103b00 0x7f1ab0103f20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.865+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2933727228 >> 172.21.15.97:0/2933727228 conn(0x7f1ab00ff070 msgr2=0x7f1ab01014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:20.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.865+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2933727228 shutdown_connections 2024-03-26T18:56:20.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.865+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2933727228 wait complete. 2024-03-26T18:56:20.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.866+0000 7f1ab5e00640 1 Processor -- start 2024-03-26T18:56:20.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.866+0000 7f1ab5e00640 1 -- start start 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.866+0000 7f1ab5e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab0103b00 0x7f1ab0199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.866+0000 7f1ab5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ab0104cf0 0x7f1ab019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.866+0000 7f1ab5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01056b0 0x7f1ab01a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.866+0000 7f1ab5e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab010f7a0 con 0x7f1ab01056b0 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.866+0000 7f1ab5e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab010f620 con 0x7f1ab0104cf0 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.866+0000 7f1ab5e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab010f920 con 0x7f1ab0103b00 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.866+0000 7f1aaea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab0103b00 0x7f1ab0199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1aaea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab0103b00 0x7f1ab0199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34580/0 (socket says 172.21.15.97:34580) 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1aaea00640 1 -- 172.21.15.97:0/2840821113 learned_addr learned my addr 172.21.15.97:0/2840821113 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1aaf400640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01056b0 0x7f1ab01a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1aae000640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ab0104cf0 0x7f1ab019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1aaea00640 1 -- 172.21.15.97:0/2840821113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ab0104cf0 msgr2=0x7f1ab019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1aaea00640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ab0104cf0 0x7f1ab019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1aaea00640 1 -- 172.21.15.97:0/2840821113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01056b0 msgr2=0x7f1ab01a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1aaea00640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01056b0 0x7f1ab01a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1aaea00640 1 -- 172.21.15.97:0/2840821113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab01a1a00 con 0x7f1ab0103b00 2024-03-26T18:56:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1aaea00640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab0103b00 0x7f1ab0199de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1aa002efd0 tx=0x7f1aa002c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.867+0000 7f1ab4a00640 1 -- 172.21.15.97:0/2840821113 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0041070 con 0x7f1ab0103b00 2024-03-26T18:56:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.868+0000 7f1ab4a00640 1 -- 172.21.15.97:0/2840821113 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa0004450 con 0x7f1ab0103b00 2024-03-26T18:56:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.868+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2840821113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab01a1c90 con 0x7f1ab0103b00 2024-03-26T18:56:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.868+0000 7f1ab4a00640 1 -- 172.21.15.97:0/2840821113 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0003d90 con 0x7f1ab0103b00 2024-03-26T18:56:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.868+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2840821113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab01a2150 con 0x7f1ab0103b00 2024-03-26T18:56:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.869+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2840821113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab010ae20 con 0x7f1ab0103b00 2024-03-26T18:56:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.871+0000 7f1ab4a00640 1 -- 172.21.15.97:0/2840821113 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f1aa0044070 con 0x7f1ab0103b00 2024-03-26T18:56:20.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.872+0000 7f1ab4a00640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1a940a2f20 0x7f1a940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:20.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.872+0000 7f1ab4a00640 1 -- 172.21.15.97:0/2840821113 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1aa00f0bd0 con 0x7f1ab0103b00 2024-03-26T18:56:20.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.872+0000 7f1aae000640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1a940a2f20 0x7f1a940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:20.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.873+0000 7f1aae000640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1a940a2f20 0x7f1a940a53e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ab019b2f0 tx=0x7f1a9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:20.876+0000 7f1ab4a00640 1 -- 172.21.15.97:0/2840821113 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1aa00bde00 con 0x7f1ab0103b00 2024-03-26T18:56:20.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:20 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3024575218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:20.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:20.966 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.083+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2840821113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1ab00624c0 con 0x7f1ab0103b00 2024-03-26T18:56:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.083+0000 7f1ab4a00640 1 -- 172.21.15.97:0/2840821113 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f1aa00bd700 con 0x7f1ab0103b00 2024-03-26T18:56:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:21.076 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-03-26T18:56:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T18:56:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-03-26T18:56:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-03-26T18:56:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T18:56:21.076 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-03-26T18:56:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2840821113 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1a940a2f20 msgr2=0x7f1a940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1a940a2f20 0x7f1a940a53e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ab019b2f0 tx=0x7f1a9c005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2840821113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab0103b00 msgr2=0x7f1ab0199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab0103b00 0x7f1ab0199de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1aa002efd0 tx=0x7f1aa002c660 comp rx=0 tx=0).stop 2024-03-26T18:56:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2840821113 shutdown_connections 2024-03-26T18:56:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1a940a2f20 0x7f1a940a53e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01056b0 0x7f1ab01a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ab0104cf0 0x7f1ab019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 --2- 172.21.15.97:0/2840821113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab0103b00 0x7f1ab0199de0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2840821113 >> 172.21.15.97:0/2840821113 conn(0x7f1ab00ff070 msgr2=0x7f1ab0107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2840821113 shutdown_connections 2024-03-26T18:56:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.086+0000 7f1ab5e00640 1 -- 172.21.15.97:0/2840821113 wait complete. 2024-03-26T18:56:21.095 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-26T18:56:21.095 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-26T18:56:21.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:20 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3024575218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:21.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:21.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:21.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:21.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:56:21.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:20 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/4100096976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:56:21.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:21.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:20 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:56:21.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.307+0000 7f9a93400640 1 -- 172.21.15.97:0/2516768713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c102350 msgr2=0x7f9a8c104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:21.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.307+0000 7f9a93400640 1 --2- 172.21.15.97:0/2516768713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c102350 0x7f9a8c104740 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009900 tx=0x7f9a8002bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:21.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.307+0000 7f9a93400640 1 -- 172.21.15.97:0/2516768713 shutdown_connections 2024-03-26T18:56:21.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.307+0000 7f9a93400640 1 --2- 172.21.15.97:0/2516768713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a8c1075b0 0x7f9a8c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:21.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.307+0000 7f9a93400640 1 --2- 172.21.15.97:0/2516768713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a8c104c80 0x7f9a8c107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:21.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.307+0000 7f9a93400640 1 --2- 172.21.15.97:0/2516768713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c102350 0x7f9a8c104740 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:21.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.307+0000 7f9a93400640 1 -- 172.21.15.97:0/2516768713 >> 172.21.15.97:0/2516768713 conn(0x7f9a8c0fbd60 msgr2=0x7f9a8c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:21.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.307+0000 7f9a93400640 1 -- 172.21.15.97:0/2516768713 shutdown_connections 2024-03-26T18:56:21.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.307+0000 7f9a93400640 1 -- 172.21.15.97:0/2516768713 wait complete. 2024-03-26T18:56:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.308+0000 7f9a93400640 1 Processor -- start 2024-03-26T18:56:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a93400640 1 -- start start 2024-03-26T18:56:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a93400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a8c102350 0x7f9a8c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a93400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c104c80 0x7f9a8c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a93400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a8c1075b0 0x7f9a8c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a93400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c10d5c0 con 0x7f9a8c102350 2024-03-26T18:56:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a93400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c10d440 con 0x7f9a8c104c80 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a93400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c10d740 con 0x7f9a8c1075b0 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a90a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a8c102350 0x7f9a8c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a8be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c104c80 0x7f9a8c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a91400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a8c1075b0 0x7f9a8c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a8be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c104c80 0x7f9a8c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60164/0 (socket says 172.21.15.97:60164) 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a90a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a8c102350 0x7f9a8c197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54814/0 (socket says 172.21.15.97:54814) 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a91400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a8c1075b0 0x7f9a8c19f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34598/0 (socket says 172.21.15.97:34598) 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.309+0000 7f9a8be00640 1 -- 172.21.15.97:0/2168804176 learned_addr learned my addr 172.21.15.97:0/2168804176 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a8be00640 1 -- 172.21.15.97:0/2168804176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a8c1075b0 msgr2=0x7f9a8c19f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a8be00640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a8c1075b0 0x7f9a8c19f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a8be00640 1 -- 172.21.15.97:0/2168804176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a8c102350 msgr2=0x7f9a8c197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a8be00640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a8c102350 0x7f9a8c197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a8be00640 1 -- 172.21.15.97:0/2168804176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c19f870 con 0x7f9a8c104c80 2024-03-26T18:56:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a8be00640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c104c80 0x7f9a8c1981a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800e9a0 tx=0x7f9a7800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a89600640 1 -- 172.21.15.97:0/2168804176 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7800cd80 con 0x7f9a8c104c80 2024-03-26T18:56:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a89600640 1 -- 172.21.15.97:0/2168804176 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a780035f0 con 0x7f9a8c104c80 2024-03-26T18:56:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a89600640 1 -- 172.21.15.97:0/2168804176 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78010670 con 0x7f9a8c104c80 2024-03-26T18:56:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a93400640 1 -- 172.21.15.97:0/2168804176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c1001a0 con 0x7f9a8c104c80 2024-03-26T18:56:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.310+0000 7f9a93400640 1 -- 172.21.15.97:0/2168804176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c1006c0 con 0x7f9a8c104c80 2024-03-26T18:56:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.311+0000 7f9a93400640 1 -- 172.21.15.97:0/2168804176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c0fd850 con 0x7f9a8c104c80 2024-03-26T18:56:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.314+0000 7f9a89600640 1 -- 172.21.15.97:0/2168804176 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f9a780037a0 con 0x7f9a8c104c80 2024-03-26T18:56:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.315+0000 7f9a89600640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9a740a2f20 0x7f9a740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.315+0000 7f9a89600640 1 -- 172.21.15.97:0/2168804176 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9a780cf2d0 con 0x7f9a8c104c80 2024-03-26T18:56:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.315+0000 7f9a90a00640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9a740a2f20 0x7f9a740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.316+0000 7f9a90a00640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9a740a2f20 0x7f9a740a53e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9a8002c240 tx=0x7f9a8002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.319+0000 7f9a89600640 1 -- 172.21.15.97:0/2168804176 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9a7809c500 con 0x7f9a8c104c80 2024-03-26T18:56:21.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:21.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:56:21.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:20 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/4100096976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:56:21.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:21.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:20 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:56:21.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:20 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3024575218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:21.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:21.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:21.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:21.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:56:21.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:20 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/4100096976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:56:21.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:21.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:20 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:56:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:21.514+0000 7f9a93400640 1 -- 172.21.15.97:0/2168804176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9a8c02d240 con 0x7f9a8c104c80 2024-03-26T18:56:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.125+0000 7f9a89600640 1 -- 172.21.15.97:0/2168804176 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f9a7809be00 con 0x7f9a8c104c80 2024-03-26T18:56:22.118 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-03-26T18:56:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.130+0000 7f9a93400640 1 -- 172.21.15.97:0/2168804176 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9a740a2f20 msgr2=0x7f9a740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.130+0000 7f9a93400640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9a740a2f20 0x7f9a740a53e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9a8002c240 tx=0x7f9a8002df90 comp rx=0 tx=0).stop 2024-03-26T18:56:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.130+0000 7f9a93400640 1 -- 172.21.15.97:0/2168804176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c104c80 msgr2=0x7f9a8c1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.130+0000 7f9a93400640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c104c80 0x7f9a8c1981a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800e9a0 tx=0x7f9a7800ee60 comp rx=0 tx=0).stop 2024-03-26T18:56:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.130+0000 7f9a93400640 1 -- 172.21.15.97:0/2168804176 shutdown_connections 2024-03-26T18:56:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.130+0000 7f9a93400640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9a740a2f20 0x7f9a740a53e0 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9a8002c240 tx=0x7f9a8002df90 comp rx=0 tx=0).stop 2024-03-26T18:56:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.130+0000 7f9a93400640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a8c1075b0 0x7f9a8c19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.130+0000 7f9a93400640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c104c80 0x7f9a8c1981a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.130+0000 7f9a93400640 1 --2- 172.21.15.97:0/2168804176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a8c102350 0x7f9a8c197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.130+0000 7f9a93400640 1 -- 172.21.15.97:0/2168804176 >> 172.21.15.97:0/2168804176 conn(0x7f9a8c0fbd60 msgr2=0x7f9a8c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:22.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.131+0000 7f9a93400640 1 -- 172.21.15.97:0/2168804176 shutdown_connections 2024-03-26T18:56:22.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.131+0000 7f9a93400640 1 -- 172.21.15.97:0/2168804176 wait complete. 2024-03-26T18:56:22.142 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-26T18:56:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.343+0000 7f9c12a00640 1 -- 172.21.15.97:0/872110989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c0c108ed0 msgr2=0x7f9c0c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.343+0000 7f9c12a00640 1 --2- 172.21.15.97:0/872110989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c0c108ed0 0x7f9c0c109350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009900 tx=0x7f9bfc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.343+0000 7f9c12a00640 1 -- 172.21.15.97:0/872110989 shutdown_connections 2024-03-26T18:56:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.343+0000 7f9c12a00640 1 --2- 172.21.15.97:0/872110989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c0c109890 0x7f9c0c110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.343+0000 7f9c12a00640 1 --2- 172.21.15.97:0/872110989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c0c108ed0 0x7f9c0c109350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.343+0000 7f9c12a00640 1 --2- 172.21.15.97:0/872110989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c0c107ce0 0x7f9c0c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.343+0000 7f9c12a00640 1 -- 172.21.15.97:0/872110989 >> 172.21.15.97:0/872110989 conn(0x7f9c0c076c80 msgr2=0x7f9c0c0790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.343+0000 7f9c12a00640 1 -- 172.21.15.97:0/872110989 shutdown_connections 2024-03-26T18:56:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.343+0000 7f9c12a00640 1 -- 172.21.15.97:0/872110989 wait complete. 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.344+0000 7f9c12a00640 1 Processor -- start 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.344+0000 7f9c12a00640 1 -- start start 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.344+0000 7f9c12a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c0c107ce0 0x7f9c0c19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.344+0000 7f9c12a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c0c108ed0 0x7f9c0c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.344+0000 7f9c12a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c0c109890 0x7f9c0c1a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.344+0000 7f9c12a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c113990 con 0x7f9c0c107ce0 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.344+0000 7f9c12a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c113810 con 0x7f9c0c108ed0 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c12a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c113b10 con 0x7f9c0c109890 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c11600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c0c107ce0 0x7f9c0c19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c10c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c0c108ed0 0x7f9c0c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c11600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c0c107ce0 0x7f9c0c19dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54838/0 (socket says 172.21.15.97:54838) 2024-03-26T18:56:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c11600640 1 -- 172.21.15.97:0/1168095446 learned_addr learned my addr 172.21.15.97:0/1168095446 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c12000640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c0c109890 0x7f9c0c1a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c10c00640 1 -- 172.21.15.97:0/1168095446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c0c109890 msgr2=0x7f9c0c1a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c10c00640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c0c109890 0x7f9c0c1a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c10c00640 1 -- 172.21.15.97:0/1168095446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c0c107ce0 msgr2=0x7f9c0c19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c10c00640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c0c107ce0 0x7f9c0c19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c10c00640 1 -- 172.21.15.97:0/1168095446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c1a5bc0 con 0x7f9c0c108ed0 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c12000640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c0c109890 0x7f9c0c1a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c11600640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c0c107ce0 0x7f9c0c19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.345+0000 7f9c10c00640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c0c108ed0 0x7f9c0c19e4e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc036040 tx=0x7f9bfc02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.346+0000 7f9c0a000640 1 -- 172.21.15.97:0/1168095446 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc043020 con 0x7f9c0c108ed0 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.346+0000 7f9c0a000640 1 -- 172.21.15.97:0/1168095446 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bfc0054e0 con 0x7f9c0c108ed0 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.346+0000 7f9c12a00640 1 -- 172.21.15.97:0/1168095446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c1a5e50 con 0x7f9c0c108ed0 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.346+0000 7f9c0a000640 1 -- 172.21.15.97:0/1168095446 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc003ce0 con 0x7f9c0c108ed0 2024-03-26T18:56:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.346+0000 7f9c12a00640 1 -- 172.21.15.97:0/1168095446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c1a6310 con 0x7f9c0c108ed0 2024-03-26T18:56:22.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.347+0000 7f9c12a00640 1 -- 172.21.15.97:0/1168095446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c10f000 con 0x7f9c0c108ed0 2024-03-26T18:56:22.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.349+0000 7f9c0a000640 1 -- 172.21.15.97:0/1168095446 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f9bfc046020 con 0x7f9c0c108ed0 2024-03-26T18:56:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.350+0000 7f9c0a000640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf40a2f20 0x7f9bf40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.350+0000 7f9c0a000640 1 -- 172.21.15.97:0/1168095446 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f9bfc0f11f0 con 0x7f9c0c108ed0 2024-03-26T18:56:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.350+0000 7f9c11600640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf40a2f20 0x7f9bf40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.350+0000 7f9c11600640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf40a2f20 0x7f9bf40a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9c000104e0 tx=0x7f9c0000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:22.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.354+0000 7f9c0a000640 1 -- 172.21.15.97:0/1168095446 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9bfc04d020 con 0x7f9c0c108ed0 2024-03-26T18:56:22.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:22 smithi097 ceph-mon[31955]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:22.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:22 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2840821113' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T18:56:22.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:22 smithi144 ceph-mon[36053]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:22.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:22 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2840821113' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T18:56:22.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:22 smithi105 ceph-mon[36270]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:22.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:22 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2840821113' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T18:56:22.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:22.562+0000 7f9c12a00640 1 -- 172.21.15.97:0/1168095446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9c0c062560 con 0x7f9c0c108ed0 2024-03-26T18:56:23.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.136+0000 7f9c0a000640 1 -- 172.21.15.97:0/1168095446 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f9bfc0be2b0 con 0x7f9c0c108ed0 2024-03-26T18:56:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-03-26T18:56:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.145+0000 7f9c12a00640 1 -- 172.21.15.97:0/1168095446 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf40a2f20 msgr2=0x7f9bf40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.145+0000 7f9c12a00640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf40a2f20 0x7f9bf40a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9c000104e0 tx=0x7f9c0000b410 comp rx=0 tx=0).stop 2024-03-26T18:56:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.145+0000 7f9c12a00640 1 -- 172.21.15.97:0/1168095446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c0c108ed0 msgr2=0x7f9c0c19e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.145+0000 7f9c12a00640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c0c108ed0 0x7f9c0c19e4e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc036040 tx=0x7f9bfc02c6c0 comp rx=0 tx=0).stop 2024-03-26T18:56:23.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.146+0000 7f9c12a00640 1 -- 172.21.15.97:0/1168095446 shutdown_connections 2024-03-26T18:56:23.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.146+0000 7f9c12a00640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9bf40a2f20 0x7f9bf40a53e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:23.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.146+0000 7f9c12a00640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c0c109890 0x7f9c0c1a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:23.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.146+0000 7f9c12a00640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c0c108ed0 0x7f9c0c19e4e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:23.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.146+0000 7f9c12a00640 1 --2- 172.21.15.97:0/1168095446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c0c107ce0 0x7f9c0c19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:23.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.146+0000 7f9c12a00640 1 -- 172.21.15.97:0/1168095446 >> 172.21.15.97:0/1168095446 conn(0x7f9c0c076c80 msgr2=0x7f9c0c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.148+0000 7f9c12a00640 1 -- 172.21.15.97:0/1168095446 shutdown_connections 2024-03-26T18:56:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.148+0000 7f9c12a00640 1 -- 172.21.15.97:0/1168095446 wait complete. 2024-03-26T18:56:23.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-26T18:56:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.367+0000 7fb55f800640 1 -- 172.21.15.97:0/2087160442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558104cf0 msgr2=0x7fb558105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.367+0000 7fb55f800640 1 --2- 172.21.15.97:0/2087160442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558104cf0 0x7fb558105170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb544009860 tx=0x7fb54402bb70 comp rx=0 tx=0).stop 2024-03-26T18:56:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.368+0000 7fb55f800640 1 -- 172.21.15.97:0/2087160442 shutdown_connections 2024-03-26T18:56:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.368+0000 7fb55f800640 1 --2- 172.21.15.97:0/2087160442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5581056b0 0x7fb55810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.368+0000 7fb55f800640 1 --2- 172.21.15.97:0/2087160442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb558104cf0 0x7fb558105170 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.368+0000 7fb55f800640 1 --2- 172.21.15.97:0/2087160442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb558103b00 0x7fb558103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.368+0000 7fb55f800640 1 -- 172.21.15.97:0/2087160442 >> 172.21.15.97:0/2087160442 conn(0x7fb5580ff070 msgr2=0x7fb5581014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.368+0000 7fb55f800640 1 -- 172.21.15.97:0/2087160442 shutdown_connections 2024-03-26T18:56:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.368+0000 7fb55f800640 1 -- 172.21.15.97:0/2087160442 wait complete. 2024-03-26T18:56:23.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb55f800640 1 Processor -- start 2024-03-26T18:56:23.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb55f800640 1 -- start start 2024-03-26T18:56:23.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb55f800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb558103b00 0x7fb558199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:23.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb55f800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb558104cf0 0x7fb55819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:23.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb55f800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5581056b0 0x7fb5581a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:23.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb55f800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb55810f780 con 0x7fb558104cf0 2024-03-26T18:56:23.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb55f800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb55810f600 con 0x7fb5581056b0 2024-03-26T18:56:23.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb55f800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb55810f900 con 0x7fb558103b00 2024-03-26T18:56:23.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb557e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb558104cf0 0x7fb55819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb557e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb558104cf0 0x7fb55819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54848/0 (socket says 172.21.15.97:54848) 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb55ce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb558103b00 0x7fb558199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb557e00640 1 -- 172.21.15.97:0/3527531631 learned_addr learned my addr 172.21.15.97:0/3527531631 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.369+0000 7fb55d800640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5581056b0 0x7fb5581a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb55ce00640 1 -- 172.21.15.97:0/3527531631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5581056b0 msgr2=0x7fb5581a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb55ce00640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5581056b0 0x7fb5581a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb55ce00640 1 -- 172.21.15.97:0/3527531631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb558104cf0 msgr2=0x7fb55819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb55ce00640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb558104cf0 0x7fb55819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb55ce00640 1 -- 172.21.15.97:0/3527531631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5581a1a50 con 0x7fb558103b00 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb557e00640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb558104cf0 0x7fb55819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb55ce00640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb558103b00 0x7fb558199e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00e9c0 tx=0x7fb54c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb555600640 1 -- 172.21.15.97:0/3527531631 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00ccc0 con 0x7fb558103b00 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb55f800640 1 -- 172.21.15.97:0/3527531631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5581a1d40 con 0x7fb558103b00 2024-03-26T18:56:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb555600640 1 -- 172.21.15.97:0/3527531631 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb54c0035f0 con 0x7fb558103b00 2024-03-26T18:56:23.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb555600640 1 -- 172.21.15.97:0/3527531631 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00ce60 con 0x7fb558103b00 2024-03-26T18:56:23.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.370+0000 7fb55f800640 1 -- 172.21.15.97:0/3527531631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5581a2180 con 0x7fb558103b00 2024-03-26T18:56:23.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.371+0000 7fb55f800640 1 -- 172.21.15.97:0/3527531631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55810ae90 con 0x7fb558103b00 2024-03-26T18:56:23.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.373+0000 7fb555600640 1 -- 172.21.15.97:0/3527531631 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fb54c0037a0 con 0x7fb558103b00 2024-03-26T18:56:23.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.374+0000 7fb555600640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5400a2f20 0x7fb5400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:23.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.375+0000 7fb555600640 1 -- 172.21.15.97:0/3527531631 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fb54c0cf7f0 con 0x7fb558103b00 2024-03-26T18:56:23.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.375+0000 7fb557e00640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5400a2f20 0x7fb5400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:23.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.375+0000 7fb557e00640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5400a2f20 0x7fb5400a53e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb55819b340 tx=0x7fb54402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:23.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.379+0000 7fb555600640 1 -- 172.21.15.97:0/3527531631 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb54c09c730 con 0x7fb558103b00 2024-03-26T18:56:23.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:23 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2168804176' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T18:56:23.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:23 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T18:56:23.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:23 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T18:56:23.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:23 smithi144 ceph-mon[36053]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T18:56:23.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:23 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1168095446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T18:56:23.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:23 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2168804176' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T18:56:23.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:23 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T18:56:23.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:23 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T18:56:23.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:23 smithi105 ceph-mon[36270]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T18:56:23.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:23 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1168095446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T18:56:23.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:23 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2168804176' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T18:56:23.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:23 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T18:56:23.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:23 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T18:56:23.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:23 smithi097 ceph-mon[31955]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T18:56:23.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:23 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1168095446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T18:56:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:23.639+0000 7fb55f800640 1 -- 172.21.15.97:0/3527531631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb55802d240 con 0x7fb558103b00 2024-03-26T18:56:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T18:56:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: pgmap v309: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T18:56:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3527531631' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T18:56:24.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:24 smithi144 ceph-mon[36053]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T18:56:24.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T18:56:24.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: pgmap v309: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:24.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T18:56:24.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3527531631' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T18:56:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:24 smithi105 ceph-mon[36270]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T18:56:24.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.542+0000 7fb555600640 1 -- 172.21.15.97:0/3527531631 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb54c09c030 con 0x7fb558103b00 2024-03-26T18:56:24.536 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 2 and data pool 3 2024-03-26T18:56:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.546+0000 7fb55f800640 1 -- 172.21.15.97:0/3527531631 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5400a2f20 msgr2=0x7fb5400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.546+0000 7fb55f800640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5400a2f20 0x7fb5400a53e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb55819b340 tx=0x7fb54402e040 comp rx=0 tx=0).stop 2024-03-26T18:56:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.546+0000 7fb55f800640 1 -- 172.21.15.97:0/3527531631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb558103b00 msgr2=0x7fb558199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.546+0000 7fb55f800640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb558103b00 0x7fb558199e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00e9c0 tx=0x7fb54c00ee80 comp rx=0 tx=0).stop 2024-03-26T18:56:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.547+0000 7fb55f800640 1 -- 172.21.15.97:0/3527531631 shutdown_connections 2024-03-26T18:56:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.547+0000 7fb55f800640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5400a2f20 0x7fb5400a53e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.547+0000 7fb55f800640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5581056b0 0x7fb5581a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.547+0000 7fb55f800640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb558104cf0 0x7fb55819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.547+0000 7fb55f800640 1 --2- 172.21.15.97:0/3527531631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb558103b00 0x7fb558199e30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.547+0000 7fb55f800640 1 -- 172.21.15.97:0/3527531631 >> 172.21.15.97:0/3527531631 conn(0x7fb5580ff070 msgr2=0x7fb558108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:24.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.549+0000 7fb55f800640 1 -- 172.21.15.97:0/3527531631 shutdown_connections 2024-03-26T18:56:24.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.549+0000 7fb55f800640 1 -- 172.21.15.97:0/3527531631 wait complete. 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: pgmap v309: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:56:24.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:56:24.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:56:24.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3527531631' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T18:56:24.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T18:56:24.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:24 smithi097 ceph-mon[31955]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T18:56:24.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:56:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.775+0000 7fac03400640 1 -- 172.21.15.97:0/2759713637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabfc103b00 msgr2=0x7fabfc103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.775+0000 7fac03400640 1 --2- 172.21.15.97:0/2759713637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabfc103b00 0x7fabfc103f20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fabec009900 tx=0x7fabec02bc50 comp rx=0 tx=0).stop 2024-03-26T18:56:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.775+0000 7fac03400640 1 -- 172.21.15.97:0/2759713637 shutdown_connections 2024-03-26T18:56:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.775+0000 7fac03400640 1 --2- 172.21.15.97:0/2759713637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabfc1056b0 0x7fabfc10bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.775+0000 7fac03400640 1 --2- 172.21.15.97:0/2759713637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabfc104cf0 0x7fabfc105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.775+0000 7fac03400640 1 --2- 172.21.15.97:0/2759713637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabfc103b00 0x7fabfc103f20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.775+0000 7fac03400640 1 -- 172.21.15.97:0/2759713637 >> 172.21.15.97:0/2759713637 conn(0x7fabfc0ff0b0 msgr2=0x7fabfc1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.775+0000 7fac03400640 1 -- 172.21.15.97:0/2759713637 shutdown_connections 2024-03-26T18:56:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.775+0000 7fac03400640 1 -- 172.21.15.97:0/2759713637 wait complete. 2024-03-26T18:56:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.776+0000 7fac03400640 1 Processor -- start 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.776+0000 7fac03400640 1 -- start start 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.776+0000 7fac03400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabfc103b00 0x7fabfc199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.776+0000 7fac03400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabfc104cf0 0x7fabfc19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.776+0000 7fac03400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabfc1056b0 0x7fabfc1a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.776+0000 7fac03400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc10f7a0 con 0x7fabfc103b00 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.776+0000 7fac03400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc10f620 con 0x7fabfc1056b0 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.776+0000 7fac03400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc10f920 con 0x7fabfc104cf0 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fac00a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabfc103b00 0x7fabfc199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fac00a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabfc103b00 0x7fabfc199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54864/0 (socket says 172.21.15.97:54864) 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fac00a00640 1 -- 172.21.15.97:0/2829911083 learned_addr learned my addr 172.21.15.97:0/2829911083 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fabf3e00640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabfc104cf0 0x7fabfc19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fac01400640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabfc1056b0 0x7fabfc1a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fac00a00640 1 -- 172.21.15.97:0/2829911083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabfc104cf0 msgr2=0x7fabfc19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fac00a00640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabfc104cf0 0x7fabfc19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fac00a00640 1 -- 172.21.15.97:0/2829911083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabfc1056b0 msgr2=0x7fabfc1a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fac00a00640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabfc1056b0 0x7fabfc1a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fac00a00640 1 -- 172.21.15.97:0/2829911083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc1a1a00 con 0x7fabfc103b00 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.777+0000 7fac00a00640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabfc103b00 0x7fabfc199de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabec02c260 tx=0x7fabec02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.778+0000 7fabfa000640 1 -- 172.21.15.97:0/2829911083 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec041070 con 0x7fabfc103b00 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.778+0000 7fabfa000640 1 -- 172.21.15.97:0/2829911083 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabec004450 con 0x7fabfc103b00 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.778+0000 7fac03400640 1 -- 172.21.15.97:0/2829911083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabfc1a1c90 con 0x7fabfc103b00 2024-03-26T18:56:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.778+0000 7fabfa000640 1 -- 172.21.15.97:0/2829911083 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec034440 con 0x7fabfc103b00 2024-03-26T18:56:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.778+0000 7fac03400640 1 -- 172.21.15.97:0/2829911083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabfc1a2180 con 0x7fabfc103b00 2024-03-26T18:56:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.779+0000 7fac03400640 1 -- 172.21.15.97:0/2829911083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabfc10adc0 con 0x7fabfc103b00 2024-03-26T18:56:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.781+0000 7fabfa000640 1 -- 172.21.15.97:0/2829911083 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fabec044070 con 0x7fabfc103b00 2024-03-26T18:56:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.782+0000 7fabfa000640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fabe00a2fa0 0x7fabe00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.783+0000 7fabfa000640 1 -- 172.21.15.97:0/2829911083 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fabec0f0d20 con 0x7fabfc103b00 2024-03-26T18:56:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.783+0000 7fabf3e00640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fabe00a2fa0 0x7fabe00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.783+0000 7fabf3e00640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fabe00a2fa0 0x7fabe00a5460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabfc19b2f0 tx=0x7fabe4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:24.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.787+0000 7fabfa000640 1 -- 172.21.15.97:0/2829911083 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fabec0bd5b0 con 0x7fabfc103b00 2024-03-26T18:56:24.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.990+0000 7fac03400640 1 -- 172.21.15.97:0/2829911083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabfc062560 con 0x7fabfc103b00 2024-03-26T18:56:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.993+0000 7fabfa000640 1 -- 172.21.15.97:0/2829911083 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22207 (secure 0 0 0) 0x7fabec0c1650 con 0x7fabfc103b00 2024-03-26T18:56:24.987 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:24.987 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:56:23.642012+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:56:24.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 -- 172.21.15.97:0/2829911083 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fabe00a2fa0 msgr2=0x7fabe00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:24.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fabe00a2fa0 0x7fabe00a5460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabfc19b2f0 tx=0x7fabe4005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:24.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 -- 172.21.15.97:0/2829911083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabfc103b00 msgr2=0x7fabfc199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:24.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabfc103b00 0x7fabfc199de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabec02c260 tx=0x7fabec02c670 comp rx=0 tx=0).stop 2024-03-26T18:56:24.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 -- 172.21.15.97:0/2829911083 shutdown_connections 2024-03-26T18:56:24.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fabe00a2fa0 0x7fabe00a5460 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabfc1056b0 0x7fabfc1a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabfc104cf0 0x7fabfc19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 --2- 172.21.15.97:0/2829911083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabfc103b00 0x7fabfc199de0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:24.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 -- 172.21.15.97:0/2829911083 >> 172.21.15.97:0/2829911083 conn(0x7fabfc0ff0b0 msgr2=0x7fabfc107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:24.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 -- 172.21.15.97:0/2829911083 shutdown_connections 2024-03-26T18:56:24.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:24.996+0000 7fac03400640 1 -- 172.21.15.97:0/2829911083 wait complete. 2024-03-26T18:56:25.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:56:25.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.235+0000 7f0fdb000640 1 -- 172.21.15.97:0/3066714922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fd410b2d0 msgr2=0x7f0fd410b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:25.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.235+0000 7f0fdb000640 1 --2- 172.21.15.97:0/3066714922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fd410b2d0 0x7f0fd410b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009900 tx=0x7f0fbc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:25.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.235+0000 7f0fdb000640 1 -- 172.21.15.97:0/3066714922 shutdown_connections 2024-03-26T18:56:25.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.235+0000 7f0fdb000640 1 --2- 172.21.15.97:0/3066714922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd410bc90 0x7f0fd4112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.235+0000 7f0fdb000640 1 --2- 172.21.15.97:0/3066714922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fd410b2d0 0x7f0fd410b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.235+0000 7f0fdb000640 1 --2- 172.21.15.97:0/3066714922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd410a0e0 0x7f0fd410a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.235+0000 7f0fdb000640 1 -- 172.21.15.97:0/3066714922 >> 172.21.15.97:0/3066714922 conn(0x7f0fd4070870 msgr2=0x7f0fd4072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:25.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.235+0000 7f0fdb000640 1 -- 172.21.15.97:0/3066714922 shutdown_connections 2024-03-26T18:56:25.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.235+0000 7f0fdb000640 1 -- 172.21.15.97:0/3066714922 wait complete. 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.236+0000 7f0fdb000640 1 Processor -- start 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.236+0000 7f0fdb000640 1 -- start start 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fdb000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd410a0e0 0x7f0fd41a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fdb000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fd410b2d0 0x7f0fd41a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fdb000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd410bc90 0x7f0fd41a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fdb000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4115dc0 con 0x7f0fd410a0e0 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fdb000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4115c40 con 0x7f0fd410b2d0 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fdb000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4115f40 con 0x7f0fd410bc90 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fd3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd410a0e0 0x7f0fd41a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fcb400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fd410b2d0 0x7f0fd41a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fcb400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fd410b2d0 0x7f0fd41a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60240/0 (socket says 172.21.15.97:60240) 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fcb400640 1 -- 172.21.15.97:0/348049055 learned_addr learned my addr 172.21.15.97:0/348049055 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:25.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fd9000640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd410bc90 0x7f0fd41a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fcb400640 1 -- 172.21.15.97:0/348049055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd410bc90 msgr2=0x7f0fd41a9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fcb400640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd410bc90 0x7f0fd41a9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fcb400640 1 -- 172.21.15.97:0/348049055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd410a0e0 msgr2=0x7f0fd41a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fcb400640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd410a0e0 0x7f0fd41a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.237+0000 7f0fcb400640 1 -- 172.21.15.97:0/348049055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd41aa200 con 0x7f0fd410b2d0 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.238+0000 7f0fcb400640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fd410b2d0 0x7f0fd41a2b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc005ab0 tx=0x7f0fbc02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.238+0000 7f0fd1600640 1 -- 172.21.15.97:0/348049055 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc043020 con 0x7f0fd410b2d0 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.238+0000 7f0fd1600640 1 -- 172.21.15.97:0/348049055 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fbc0054e0 con 0x7f0fd410b2d0 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.238+0000 7f0fdb000640 1 -- 172.21.15.97:0/348049055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd41aa490 con 0x7f0fd410b2d0 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.238+0000 7f0fd1600640 1 -- 172.21.15.97:0/348049055 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc003ce0 con 0x7f0fd410b2d0 2024-03-26T18:56:25.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.238+0000 7f0fdb000640 1 -- 172.21.15.97:0/348049055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd41aa9e0 con 0x7f0fd410b2d0 2024-03-26T18:56:25.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.239+0000 7f0fdb000640 1 -- 172.21.15.97:0/348049055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd41aabe0 con 0x7f0fd410b2d0 2024-03-26T18:56:25.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.241+0000 7f0fd1600640 1 -- 172.21.15.97:0/348049055 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f0fbc046020 con 0x7f0fd410b2d0 2024-03-26T18:56:25.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.242+0000 7f0fd1600640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb80a2f20 0x7f0fb80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:25.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.242+0000 7f0fd1600640 1 -- 172.21.15.97:0/348049055 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f0fbc0f13b0 con 0x7f0fd410b2d0 2024-03-26T18:56:25.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.242+0000 7f0fd3e00640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb80a2f20 0x7f0fb80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:25.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.243+0000 7f0fd3e00640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb80a2f20 0x7f0fb80a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0fd410b130 tx=0x7f0fc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:25.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.246+0000 7f0fd1600640 1 -- 172.21.15.97:0/348049055 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0fbc04d020 con 0x7f0fd410b2d0 2024-03-26T18:56:25.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:25 smithi097 ceph-mon[31955]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T18:56:25.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:25 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T18:56:25.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:25 smithi097 ceph-mon[31955]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T18:56:25.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:25 smithi097 ceph-mon[31955]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T18:56:25.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:25 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] up:active 2024-03-26T18:56:25.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:25 smithi097 ceph-mon[31955]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:56:25.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:25 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2829911083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:25.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.446+0000 7f0fdb000640 1 -- 172.21.15.97:0/348049055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fd41aae50 con 0x7f0fd410b2d0 2024-03-26T18:56:25.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.448+0000 7f0fd1600640 1 -- 172.21.15.97:0/348049055 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22194 (secure 0 0 0) 0x7f0fbc0be330 con 0x7f0fd410b2d0 2024-03-26T18:56:25.442 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:25.442 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:56:25.123057+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:56:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.450+0000 7f0fdb000640 1 -- 172.21.15.97:0/348049055 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb80a2f20 msgr2=0x7f0fb80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.450+0000 7f0fdb000640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb80a2f20 0x7f0fb80a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0fd410b130 tx=0x7f0fc4005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.450+0000 7f0fdb000640 1 -- 172.21.15.97:0/348049055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fd410b2d0 msgr2=0x7f0fd41a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.450+0000 7f0fdb000640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fd410b2d0 0x7f0fd41a2b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc005ab0 tx=0x7f0fbc02c6c0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.451+0000 7f0fdb000640 1 -- 172.21.15.97:0/348049055 shutdown_connections 2024-03-26T18:56:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.451+0000 7f0fdb000640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb80a2f20 0x7f0fb80a53e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.451+0000 7f0fdb000640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd410bc90 0x7f0fd41a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.451+0000 7f0fdb000640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fd410b2d0 0x7f0fd41a2b20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.451+0000 7f0fdb000640 1 --2- 172.21.15.97:0/348049055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fd410a0e0 0x7f0fd41a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.451+0000 7f0fdb000640 1 -- 172.21.15.97:0/348049055 >> 172.21.15.97:0/348049055 conn(0x7f0fd4070870 msgr2=0x7f0fd406b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.451+0000 7f0fdb000640 1 -- 172.21.15.97:0/348049055 shutdown_connections 2024-03-26T18:56:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.451+0000 7f0fdb000640 1 -- 172.21.15.97:0/348049055 wait complete. 2024-03-26T18:56:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:25 smithi105 ceph-mon[36270]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T18:56:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:25 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T18:56:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:25 smithi105 ceph-mon[36270]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T18:56:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:25 smithi105 ceph-mon[36270]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T18:56:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:25 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] up:active 2024-03-26T18:56:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:25 smithi105 ceph-mon[36270]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:56:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:25 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2829911083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:25.463 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-26T18:56:25.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:25 smithi144 ceph-mon[36053]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T18:56:25.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:25 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T18:56:25.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:25 smithi144 ceph-mon[36053]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T18:56:25.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:25 smithi144 ceph-mon[36053]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T18:56:25.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:25 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] up:active 2024-03-26T18:56:25.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:25 smithi144 ceph-mon[36053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:56:25.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:25 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2829911083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:25.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.683+0000 7f36f1000640 1 -- 172.21.15.97:0/3798873556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36ec1056b0 msgr2=0x7f36ec10bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.683+0000 7f36f1000640 1 --2- 172.21.15.97:0/3798873556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36ec1056b0 0x7f36ec10bf40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009900 tx=0x7f36e402bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.683+0000 7f36f1000640 1 -- 172.21.15.97:0/3798873556 shutdown_connections 2024-03-26T18:56:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.683+0000 7f36f1000640 1 --2- 172.21.15.97:0/3798873556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36ec1056b0 0x7f36ec10bf40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.683+0000 7f36f1000640 1 --2- 172.21.15.97:0/3798873556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ec104cf0 0x7f36ec105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.683+0000 7f36f1000640 1 --2- 172.21.15.97:0/3798873556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ec103b00 0x7f36ec103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.683+0000 7f36f1000640 1 -- 172.21.15.97:0/3798873556 >> 172.21.15.97:0/3798873556 conn(0x7f36ec0ff070 msgr2=0x7f36ec1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.683+0000 7f36f1000640 1 -- 172.21.15.97:0/3798873556 shutdown_connections 2024-03-26T18:56:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.683+0000 7f36f1000640 1 -- 172.21.15.97:0/3798873556 wait complete. 2024-03-26T18:56:25.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.684+0000 7f36f1000640 1 Processor -- start 2024-03-26T18:56:25.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.684+0000 7f36f1000640 1 -- start start 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.684+0000 7f36f1000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36ec103b00 0x7f36ec199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.684+0000 7f36f1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ec104cf0 0x7f36ec19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.684+0000 7f36f1000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ec1056b0 0x7f36ec1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.684+0000 7f36f1000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec10f7d0 con 0x7f36ec104cf0 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.684+0000 7f36f1000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec10f650 con 0x7f36ec103b00 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.684+0000 7f36f1000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec10f950 con 0x7f36ec1056b0 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.684+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36ec103b00 0x7f36ec199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.684+0000 7f36e1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ec104cf0 0x7f36ec19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36ec103b00 0x7f36ec199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60256/0 (socket says 172.21.15.97:60256) 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36e9e00640 1 -- 172.21.15.97:0/509276433 learned_addr learned my addr 172.21.15.97:0/509276433 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36ea800640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ec1056b0 0x7f36ec1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36e1400640 1 -- 172.21.15.97:0/509276433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ec1056b0 msgr2=0x7f36ec1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36e1400640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ec1056b0 0x7f36ec1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36e1400640 1 -- 172.21.15.97:0/509276433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36ec103b00 msgr2=0x7f36ec199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36e1400640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36ec103b00 0x7f36ec199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36e1400640 1 -- 172.21.15.97:0/509276433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec1a1980 con 0x7f36ec104cf0 2024-03-26T18:56:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36e9e00640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36ec103b00 0x7f36ec199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36e1400640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ec104cf0 0x7f36ec19a2a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f36d400e9f0 tx=0x7f36d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.685+0000 7f36e3400640 1 -- 172.21.15.97:0/509276433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d400cd80 con 0x7f36ec104cf0 2024-03-26T18:56:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.686+0000 7f36e3400640 1 -- 172.21.15.97:0/509276433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d40035f0 con 0x7f36ec104cf0 2024-03-26T18:56:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.686+0000 7f36f1000640 1 -- 172.21.15.97:0/509276433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec1a1c70 con 0x7f36ec104cf0 2024-03-26T18:56:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.686+0000 7f36e3400640 1 -- 172.21.15.97:0/509276433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4010690 con 0x7f36ec104cf0 2024-03-26T18:56:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.686+0000 7f36f1000640 1 -- 172.21.15.97:0/509276433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec1a21c0 con 0x7f36ec104cf0 2024-03-26T18:56:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.687+0000 7f36f1000640 1 -- 172.21.15.97:0/509276433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ec10ad50 con 0x7f36ec104cf0 2024-03-26T18:56:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.689+0000 7f36e3400640 1 -- 172.21.15.97:0/509276433 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f36d4003b30 con 0x7f36ec104cf0 2024-03-26T18:56:25.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.691+0000 7f36e3400640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36d00a2f20 0x7f36d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:25.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.691+0000 7f36e3400640 1 -- 172.21.15.97:0/509276433 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f36d40cf690 con 0x7f36ec104cf0 2024-03-26T18:56:25.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.691+0000 7f36e9e00640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36d00a2f20 0x7f36d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:25.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.692+0000 7f36e9e00640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36d00a2f20 0x7f36d00a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005f40 tx=0x7f36dc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:25.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.693+0000 7f36e3400640 1 -- 172.21.15.97:0/509276433 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f36d409c600 con 0x7f36ec104cf0 2024-03-26T18:56:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:25.935+0000 7f36f1000640 1 -- 172.21.15.97:0/509276433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f36ec02d240 con 0x7f36ec104cf0 2024-03-26T18:56:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:26 smithi105 ceph-mon[36270]: pgmap v312: 97 pgs: 26 active+clean, 36 creating+peering, 35 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:26 smithi105 ceph-mon[36270]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T18:56:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:26 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/348049055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:26 smithi105 ceph-mon[36270]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T18:56:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:26 smithi105 ceph-mon[36270]: Cluster is now healthy 2024-03-26T18:56:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:26 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/509276433' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T18:56:26.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:26 smithi097 ceph-mon[31955]: pgmap v312: 97 pgs: 26 active+clean, 36 creating+peering, 35 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:26.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:26 smithi097 ceph-mon[31955]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T18:56:26.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:26 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/348049055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:26.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:26 smithi097 ceph-mon[31955]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T18:56:26.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:26 smithi097 ceph-mon[31955]: Cluster is now healthy 2024-03-26T18:56:26.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:26 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/509276433' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T18:56:26.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:26 smithi144 ceph-mon[36053]: pgmap v312: 97 pgs: 26 active+clean, 36 creating+peering, 35 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:56:26.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:26 smithi144 ceph-mon[36053]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T18:56:26.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:26 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/348049055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:26.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:26 smithi144 ceph-mon[36053]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T18:56:26.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:26 smithi144 ceph-mon[36053]: Cluster is now healthy 2024-03-26T18:56:26.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:26 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/509276433' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T18:56:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.811+0000 7f36e3400640 1 -- 172.21.15.97:0/509276433 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f36d409bf00 con 0x7f36ec104cf0 2024-03-26T18:56:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 -- 172.21.15.97:0/509276433 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36d00a2f20 msgr2=0x7f36d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36d00a2f20 0x7f36d00a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005f40 tx=0x7f36dc009500 comp rx=0 tx=0).stop 2024-03-26T18:56:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 -- 172.21.15.97:0/509276433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ec104cf0 msgr2=0x7f36ec19a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ec104cf0 0x7f36ec19a2a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f36d400e9f0 tx=0x7f36d400eeb0 comp rx=0 tx=0).stop 2024-03-26T18:56:26.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 -- 172.21.15.97:0/509276433 shutdown_connections 2024-03-26T18:56:26.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36d00a2f20 0x7f36d00a53e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:26.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ec1056b0 0x7f36ec1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:26.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36ec104cf0 0x7f36ec19a2a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:26.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 --2- 172.21.15.97:0/509276433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36ec103b00 0x7f36ec199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:26.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 -- 172.21.15.97:0/509276433 >> 172.21.15.97:0/509276433 conn(0x7f36ec0ff070 msgr2=0x7f36ec107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:26.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 -- 172.21.15.97:0/509276433 shutdown_connections 2024-03-26T18:56:26.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:26.815+0000 7f36f1000640 1 -- 172.21.15.97:0/509276433 wait complete. 2024-03-26T18:56:26.830 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-26T18:56:26.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-26T18:56:27.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.054+0000 7fe677a00640 1 -- 172.21.15.97:0/3524352314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe670105690 msgr2=0x7fe67010bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:27.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.054+0000 7fe677a00640 1 --2- 172.21.15.97:0/3524352314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe670105690 0x7fe67010bf20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe668009ca0 tx=0x7fe66802c100 comp rx=0 tx=0).stop 2024-03-26T18:56:27.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.054+0000 7fe677a00640 1 -- 172.21.15.97:0/3524352314 shutdown_connections 2024-03-26T18:56:27.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.054+0000 7fe677a00640 1 --2- 172.21.15.97:0/3524352314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe670105690 0x7fe67010bf20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:27.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.054+0000 7fe677a00640 1 --2- 172.21.15.97:0/3524352314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670104cd0 0x7fe670105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:27.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.054+0000 7fe677a00640 1 --2- 172.21.15.97:0/3524352314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670103ae0 0x7fe670103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:27.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.054+0000 7fe677a00640 1 -- 172.21.15.97:0/3524352314 >> 172.21.15.97:0/3524352314 conn(0x7fe6700ff070 msgr2=0x7fe670101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:27.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.054+0000 7fe677a00640 1 -- 172.21.15.97:0/3524352314 shutdown_connections 2024-03-26T18:56:27.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.054+0000 7fe677a00640 1 -- 172.21.15.97:0/3524352314 wait complete. 2024-03-26T18:56:27.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.055+0000 7fe677a00640 1 Processor -- start 2024-03-26T18:56:27.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.055+0000 7fe677a00640 1 -- start start 2024-03-26T18:56:27.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe677a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670103ae0 0x7fe670199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:27.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe677a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670104cd0 0x7fe67019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:27.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe677a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe670105690 0x7fe6701a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:27.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe677a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe67010f740 con 0x7fe670104cd0 2024-03-26T18:56:27.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe677a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe67010f5c0 con 0x7fe670105690 2024-03-26T18:56:27.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe677a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe67010f8c0 con 0x7fe670103ae0 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe66fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670104cd0 0x7fe67019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe675a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe670105690 0x7fe6701a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe66fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670104cd0 0x7fe67019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54916/0 (socket says 172.21.15.97:54916) 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe66fe00640 1 -- 172.21.15.97:0/3182495500 learned_addr learned my addr 172.21.15.97:0/3182495500 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe675000640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670103ae0 0x7fe670199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe675a00640 1 -- 172.21.15.97:0/3182495500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670103ae0 msgr2=0x7fe670199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe675a00640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670103ae0 0x7fe670199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe675a00640 1 -- 172.21.15.97:0/3182495500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670104cd0 msgr2=0x7fe67019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe675a00640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670104cd0 0x7fe67019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe675a00640 1 -- 172.21.15.97:0/3182495500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6701a1a00 con 0x7fe670105690 2024-03-26T18:56:27.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.056+0000 7fe675000640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670103ae0 0x7fe670199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:56:27.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.057+0000 7fe675a00640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe670105690 0x7fe6701a1390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe668009bf0 tx=0x7fe668035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:27.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.057+0000 7fe66d600640 1 -- 172.21.15.97:0/3182495500 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668039040 con 0x7fe670105690 2024-03-26T18:56:27.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.057+0000 7fe66d600640 1 -- 172.21.15.97:0/3182495500 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe668044070 con 0x7fe670105690 2024-03-26T18:56:27.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.057+0000 7fe66d600640 1 -- 172.21.15.97:0/3182495500 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668004430 con 0x7fe670105690 2024-03-26T18:56:27.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.057+0000 7fe677a00640 1 -- 172.21.15.97:0/3182495500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6701a1c90 con 0x7fe670105690 2024-03-26T18:56:27.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.057+0000 7fe677a00640 1 -- 172.21.15.97:0/3182495500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6701a21a0 con 0x7fe670105690 2024-03-26T18:56:27.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.058+0000 7fe677a00640 1 -- 172.21.15.97:0/3182495500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67010ada0 con 0x7fe670105690 2024-03-26T18:56:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.060+0000 7fe66d600640 1 -- 172.21.15.97:0/3182495500 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fe6680045d0 con 0x7fe670105690 2024-03-26T18:56:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.061+0000 7fe66d600640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe6580a2fa0 0x7fe6580a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.061+0000 7fe66d600640 1 -- 172.21.15.97:0/3182495500 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe6680f1920 con 0x7fe670105690 2024-03-26T18:56:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.061+0000 7fe675000640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe6580a2fa0 0x7fe6580a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.062+0000 7fe675000640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe6580a2fa0 0x7fe6580a5460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6640058c0 tx=0x7fe664005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:27.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.065+0000 7fe66d600640 1 -- 172.21.15.97:0/3182495500 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe6680be800 con 0x7fe670105690 2024-03-26T18:56:27.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.305+0000 7fe677a00640 1 -- 172.21.15.97:0/3182495500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fe67002d240 con 0x7fe670105690 2024-03-26T18:56:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.828+0000 7fe66d600640 1 -- 172.21.15.97:0/3182495500 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7fe6680be100 con 0x7fe670105690 2024-03-26T18:56:27.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 -- 172.21.15.97:0/3182495500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe6580a2fa0 msgr2=0x7fe6580a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:27.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe6580a2fa0 0x7fe6580a5460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6640058c0 tx=0x7fe664005850 comp rx=0 tx=0).stop 2024-03-26T18:56:27.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 -- 172.21.15.97:0/3182495500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe670105690 msgr2=0x7fe6701a1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe670105690 0x7fe6701a1390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe668009bf0 tx=0x7fe668035d10 comp rx=0 tx=0).stop 2024-03-26T18:56:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 -- 172.21.15.97:0/3182495500 shutdown_connections 2024-03-26T18:56:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe6580a2fa0 0x7fe6580a5460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe670105690 0x7fe6701a1390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670104cd0 0x7fe67019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 --2- 172.21.15.97:0/3182495500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670103ae0 0x7fe670199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 -- 172.21.15.97:0/3182495500 >> 172.21.15.97:0/3182495500 conn(0x7fe6700ff070 msgr2=0x7fe670107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 -- 172.21.15.97:0/3182495500 shutdown_connections 2024-03-26T18:56:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:27.832+0000 7fe677a00640 1 -- 172.21.15.97:0/3182495500 wait complete. 2024-03-26T18:56:27.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-26T18:56:28.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.053+0000 7f859f400640 1 -- 172.21.15.97:0/2140851534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8598104cf0 msgr2=0x7f8598105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:28.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.053+0000 7f859f400640 1 --2- 172.21.15.97:0/2140851534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8598104cf0 0x7f8598105170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8584009900 tx=0x7f858402bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:28.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.053+0000 7f859f400640 1 -- 172.21.15.97:0/2140851534 shutdown_connections 2024-03-26T18:56:28.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.053+0000 7f859f400640 1 --2- 172.21.15.97:0/2140851534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85981056b0 0x7f859810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:28.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.053+0000 7f859f400640 1 --2- 172.21.15.97:0/2140851534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8598104cf0 0x7f8598105170 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:28.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.053+0000 7f859f400640 1 --2- 172.21.15.97:0/2140851534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598103b00 0x7f8598103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:28.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.053+0000 7f859f400640 1 -- 172.21.15.97:0/2140851534 >> 172.21.15.97:0/2140851534 conn(0x7f85980ff070 msgr2=0x7f85981014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:28.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.053+0000 7f859f400640 1 -- 172.21.15.97:0/2140851534 shutdown_connections 2024-03-26T18:56:28.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.053+0000 7f859f400640 1 -- 172.21.15.97:0/2140851534 wait complete. 2024-03-26T18:56:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.054+0000 7f859f400640 1 Processor -- start 2024-03-26T18:56:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.054+0000 7f859f400640 1 -- start start 2024-03-26T18:56:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.054+0000 7f859f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598103b00 0x7f8598199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.054+0000 7f859f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598104cf0 0x7f859819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85981056b0 0x7f85981a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859f400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f859810f7d0 con 0x7f8598104cf0 2024-03-26T18:56:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859f400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f859810f650 con 0x7f85981056b0 2024-03-26T18:56:28.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859f400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f859810f950 con 0x7f8598103b00 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f8597e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598104cf0 0x7f859819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598103b00 0x7f8598199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85981056b0 0x7f85981a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f8597e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598104cf0 0x7f859819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54938/0 (socket says 172.21.15.97:54938) 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85981056b0 0x7f85981a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60272/0 (socket says 172.21.15.97:60272) 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f8597e00640 1 -- 172.21.15.97:0/3406272419 learned_addr learned my addr 172.21.15.97:0/3406272419 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598103b00 0x7f8598199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34754/0 (socket says 172.21.15.97:34754) 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859d400640 1 -- 172.21.15.97:0/3406272419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598103b00 msgr2=0x7f8598199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859d400640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598103b00 0x7f8598199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859d400640 1 -- 172.21.15.97:0/3406272419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598104cf0 msgr2=0x7f859819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859d400640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598104cf0 0x7f859819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859d400640 1 -- 172.21.15.97:0/3406272419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85981a1a10 con 0x7f85981056b0 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.055+0000 7f859ca00640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598103b00 0x7f8598199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:28.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.056+0000 7f859d400640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85981056b0 0x7f85981a13a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f859000e940 tx=0x7f859000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:28.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.056+0000 7f8595600640 1 -- 172.21.15.97:0/3406272419 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590004d20 con 0x7f85981056b0 2024-03-26T18:56:28.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.056+0000 7f8595600640 1 -- 172.21.15.97:0/3406272419 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8590005020 con 0x7f85981056b0 2024-03-26T18:56:28.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.056+0000 7f859f400640 1 -- 172.21.15.97:0/3406272419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85981a1d00 con 0x7f85981056b0 2024-03-26T18:56:28.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.056+0000 7f8595600640 1 -- 172.21.15.97:0/3406272419 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85900116b0 con 0x7f85981056b0 2024-03-26T18:56:28.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.056+0000 7f859f400640 1 -- 172.21.15.97:0/3406272419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981a21d0 con 0x7f85981056b0 2024-03-26T18:56:28.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.057+0000 7f859f400640 1 -- 172.21.15.97:0/3406272419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859810adc0 con 0x7f85981056b0 2024-03-26T18:56:28.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.059+0000 7f8595600640 1 -- 172.21.15.97:0/3406272419 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f85900054d0 con 0x7f85981056b0 2024-03-26T18:56:28.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.060+0000 7f8595600640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85800a2f20 0x7f85800a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:28.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.060+0000 7f8595600640 1 -- 172.21.15.97:0/3406272419 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f85900cf690 con 0x7f85981056b0 2024-03-26T18:56:28.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.060+0000 7f859ca00640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85800a2f20 0x7f85800a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:28.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.061+0000 7f859ca00640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85800a2f20 0x7f85800a53e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8598104b50 tx=0x7f858c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:28.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.068+0000 7f8595600640 1 -- 172.21.15.97:0/3406272419 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8590016030 con 0x7f85981056b0 2024-03-26T18:56:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:27 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/509276433' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T18:56:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:27 smithi097 ceph-mon[31955]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:56:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:27 smithi097 ceph-mon[31955]: pgmap v314: 97 pgs: 40 active+clean, 36 creating+peering, 21 unknown; 578 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 1 op/s 2024-03-26T18:56:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:27 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3182495500' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T18:56:28.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:27 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T18:56:28.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:27 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/509276433' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T18:56:28.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:27 smithi144 ceph-mon[36053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:56:28.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:27 smithi144 ceph-mon[36053]: pgmap v314: 97 pgs: 40 active+clean, 36 creating+peering, 21 unknown; 578 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 1 op/s 2024-03-26T18:56:28.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:27 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3182495500' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T18:56:28.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:27 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T18:56:28.210 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:27 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b[66299]: 2024-03-26T18:56:27.973+0000 7fa0bf265700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-26T18:56:28.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:27 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/509276433' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T18:56:28.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:27 smithi105 ceph-mon[36270]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:56:28.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:27 smithi105 ceph-mon[36270]: pgmap v314: 97 pgs: 40 active+clean, 36 creating+peering, 21 unknown; 578 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 1 op/s 2024-03-26T18:56:28.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:27 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3182495500' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T18:56:28.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:27 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T18:56:28.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.309+0000 7f859f400640 1 -- 172.21.15.97:0/3406272419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f859802d240 con 0x7f85981056b0 2024-03-26T18:56:28.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.840+0000 7f8595600640 1 -- 172.21.15.97:0/3406272419 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f859009bf60 con 0x7f85981056b0 2024-03-26T18:56:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.844+0000 7f859f400640 1 -- 172.21.15.97:0/3406272419 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85800a2f20 msgr2=0x7f85800a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.844+0000 7f859f400640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85800a2f20 0x7f85800a53e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8598104b50 tx=0x7f858c005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.844+0000 7f859f400640 1 -- 172.21.15.97:0/3406272419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85981056b0 msgr2=0x7f85981a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.844+0000 7f859f400640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85981056b0 0x7f85981a13a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f859000e940 tx=0x7f859000ee00 comp rx=0 tx=0).stop 2024-03-26T18:56:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.845+0000 7f859f400640 1 -- 172.21.15.97:0/3406272419 shutdown_connections 2024-03-26T18:56:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.845+0000 7f859f400640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85800a2f20 0x7f85800a53e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.845+0000 7f859f400640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85981056b0 0x7f85981a13a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.845+0000 7f859f400640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598104cf0 0x7f859819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.845+0000 7f859f400640 1 --2- 172.21.15.97:0/3406272419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598103b00 0x7f8598199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.845+0000 7f859f400640 1 -- 172.21.15.97:0/3406272419 >> 172.21.15.97:0/3406272419 conn(0x7f85980ff070 msgr2=0x7f8598107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.845+0000 7f859f400640 1 -- 172.21.15.97:0/3406272419 shutdown_connections 2024-03-26T18:56:28.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:28.845+0000 7f859f400640 1 -- 172.21.15.97:0/3406272419 wait complete. 2024-03-26T18:56:28.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-26T18:56:29.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.063+0000 7fa412a00640 1 -- 172.21.15.97:0/1356086893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c103b00 msgr2=0x7fa40c103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:29.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.063+0000 7fa412a00640 1 --2- 172.21.15.97:0/1356086893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c103b00 0x7fa40c103f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009900 tx=0x7fa3fc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:29.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.063+0000 7fa412a00640 1 -- 172.21.15.97:0/1356086893 shutdown_connections 2024-03-26T18:56:29.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.063+0000 7fa412a00640 1 --2- 172.21.15.97:0/1356086893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa40c1056b0 0x7fa40c10bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:29.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.063+0000 7fa412a00640 1 --2- 172.21.15.97:0/1356086893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa40c104cf0 0x7fa40c105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:29.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.063+0000 7fa412a00640 1 --2- 172.21.15.97:0/1356086893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c103b00 0x7fa40c103f20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:29.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.063+0000 7fa412a00640 1 -- 172.21.15.97:0/1356086893 >> 172.21.15.97:0/1356086893 conn(0x7fa40c0ff070 msgr2=0x7fa40c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:29.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.063+0000 7fa412a00640 1 -- 172.21.15.97:0/1356086893 shutdown_connections 2024-03-26T18:56:29.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.063+0000 7fa412a00640 1 -- 172.21.15.97:0/1356086893 wait complete. 2024-03-26T18:56:29.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.064+0000 7fa412a00640 1 Processor -- start 2024-03-26T18:56:29.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.064+0000 7fa412a00640 1 -- start start 2024-03-26T18:56:29.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.064+0000 7fa412a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa40c103b00 0x7fa40c199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:29.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.064+0000 7fa412a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa40c104cf0 0x7fa40c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:29.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.064+0000 7fa412a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c1056b0 0x7fa40c1a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:29.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.064+0000 7fa412a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c10f7a0 con 0x7fa40c103b00 2024-03-26T18:56:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.064+0000 7fa412a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c10f620 con 0x7fa40c1056b0 2024-03-26T18:56:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.064+0000 7fa412a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c10f920 con 0x7fa40c104cf0 2024-03-26T18:56:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa40be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa40c103b00 0x7fa40c199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa410a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c1056b0 0x7fa40c1a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa40be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa40c103b00 0x7fa40c199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54946/0 (socket says 172.21.15.97:54946) 2024-03-26T18:56:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa40be00640 1 -- 172.21.15.97:0/1952661037 learned_addr learned my addr 172.21.15.97:0/1952661037 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa40b400640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa40c104cf0 0x7fa40c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa410a00640 1 -- 172.21.15.97:0/1952661037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa40c104cf0 msgr2=0x7fa40c19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa410a00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa40c104cf0 0x7fa40c19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa410a00640 1 -- 172.21.15.97:0/1952661037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa40c103b00 msgr2=0x7fa40c199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa410a00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa40c103b00 0x7fa40c199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa410a00640 1 -- 172.21.15.97:0/1952661037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c1a1a70 con 0x7fa40c1056b0 2024-03-26T18:56:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa410a00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c1056b0 0x7fa40c1a1400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa404009fd0 tx=0x7fa40400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.065+0000 7fa408c00640 1 -- 172.21.15.97:0/1952661037 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40400ce20 con 0x7fa40c1056b0 2024-03-26T18:56:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.066+0000 7fa408c00640 1 -- 172.21.15.97:0/1952661037 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa404005020 con 0x7fa40c1056b0 2024-03-26T18:56:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.066+0000 7fa408c00640 1 -- 172.21.15.97:0/1952661037 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404010720 con 0x7fa40c1056b0 2024-03-26T18:56:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.066+0000 7fa412a00640 1 -- 172.21.15.97:0/1952661037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c1a1d60 con 0x7fa40c1056b0 2024-03-26T18:56:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.066+0000 7fa412a00640 1 -- 172.21.15.97:0/1952661037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1a2200 con 0x7fa40c1056b0 2024-03-26T18:56:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.067+0000 7fa412a00640 1 -- 172.21.15.97:0/1952661037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c10ae20 con 0x7fa40c1056b0 2024-03-26T18:56:29.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.069+0000 7fa408c00640 1 -- 172.21.15.97:0/1952661037 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fa404005540 con 0x7fa40c1056b0 2024-03-26T18:56:29.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.070+0000 7fa408c00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3f00a2fa0 0x7fa3f00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:29.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.070+0000 7fa408c00640 1 -- 172.21.15.97:0/1952661037 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa4040cfb00 con 0x7fa40c1056b0 2024-03-26T18:56:29.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.070+0000 7fa40be00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3f00a2fa0 0x7fa3f00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:29.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.071+0000 7fa40be00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3f00a2fa0 0x7fa3f00a5460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc006010 tx=0x7fa3fc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:29.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.072+0000 7fa408c00640 1 -- 172.21.15.97:0/1952661037 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa40409c9e0 con 0x7fa40c1056b0 2024-03-26T18:56:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:28 smithi097 ceph-mon[31955]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T18:56:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:28 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T18:56:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:28 smithi097 ceph-mon[31955]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T18:56:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:28 smithi097 ceph-mon[31955]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:56:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:28 smithi097 ceph-mon[31955]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-26T18:56:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:28 smithi097 ceph-mon[31955]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-26T18:56:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:28 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3406272419' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T18:56:29.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:28 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T18:56:29.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:28 smithi144 ceph-mon[36053]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T18:56:29.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:28 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T18:56:29.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:28 smithi144 ceph-mon[36053]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T18:56:29.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:28 smithi144 ceph-mon[36053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:56:29.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:28 smithi144 ceph-mon[36053]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-26T18:56:29.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:28 smithi144 ceph-mon[36053]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-26T18:56:29.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:28 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3406272419' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T18:56:29.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:28 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T18:56:29.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:28 smithi105 ceph-mon[36270]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T18:56:29.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:28 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T18:56:29.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:28 smithi105 ceph-mon[36270]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T18:56:29.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:28 smithi105 ceph-mon[36270]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:56:29.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:28 smithi105 ceph-mon[36270]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-26T18:56:29.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:28 smithi105 ceph-mon[36270]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-26T18:56:29.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:28 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3406272419' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T18:56:29.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:28 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T18:56:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.340+0000 7fa412a00640 1 -- 172.21.15.97:0/1952661037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa40c02d240 con 0x7fa40c1056b0 2024-03-26T18:56:29.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.808+0000 7fa408c00640 1 -- 172.21.15.97:0/1952661037 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7fa40409c2e0 con 0x7fa40c1056b0 2024-03-26T18:56:29.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.812+0000 7fa412a00640 1 -- 172.21.15.97:0/1952661037 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3f00a2fa0 msgr2=0x7fa3f00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:29.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.812+0000 7fa412a00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3f00a2fa0 0x7fa3f00a5460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc006010 tx=0x7fa3fc02df90 comp rx=0 tx=0).stop 2024-03-26T18:56:29.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.812+0000 7fa412a00640 1 -- 172.21.15.97:0/1952661037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c1056b0 msgr2=0x7fa40c1a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:29.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.812+0000 7fa412a00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c1056b0 0x7fa40c1a1400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa404009fd0 tx=0x7fa40400ef10 comp rx=0 tx=0).stop 2024-03-26T18:56:29.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.813+0000 7fa412a00640 1 -- 172.21.15.97:0/1952661037 shutdown_connections 2024-03-26T18:56:29.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.813+0000 7fa412a00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3f00a2fa0 0x7fa3f00a5460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:29.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.813+0000 7fa412a00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40c1056b0 0x7fa40c1a1400 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:29.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.813+0000 7fa412a00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa40c104cf0 0x7fa40c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:29.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.813+0000 7fa412a00640 1 --2- 172.21.15.97:0/1952661037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa40c103b00 0x7fa40c199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:29.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.813+0000 7fa412a00640 1 -- 172.21.15.97:0/1952661037 >> 172.21.15.97:0/1952661037 conn(0x7fa40c0ff070 msgr2=0x7fa40c107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:29.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.813+0000 7fa412a00640 1 -- 172.21.15.97:0/1952661037 shutdown_connections 2024-03-26T18:56:29.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:29.813+0000 7fa412a00640 1 -- 172.21.15.97:0/1952661037 wait complete. 2024-03-26T18:56:29.833 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-26T18:56:29.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-26T18:56:30.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.023+0000 7f5fb2a00640 1 -- 172.21.15.97:0/270824745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fac1001b0 msgr2=0x7f5fac1005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:30.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.023+0000 7f5fb2a00640 1 --2- 172.21.15.97:0/270824745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fac1001b0 0x7f5fac1005d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009900 tx=0x7f5f9c02bcc0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.023+0000 7f5fb2a00640 1 -- 172.21.15.97:0/270824745 shutdown_connections 2024-03-26T18:56:30.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.023+0000 7f5fb2a00640 1 --2- 172.21.15.97:0/270824745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1014b0 0x7f5fac101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.023+0000 7f5fb2a00640 1 --2- 172.21.15.97:0/270824745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fac100b10 0x7f5fac100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.023+0000 7f5fb2a00640 1 --2- 172.21.15.97:0/270824745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fac1001b0 0x7f5fac1005d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.023+0000 7f5fb2a00640 1 -- 172.21.15.97:0/270824745 >> 172.21.15.97:0/270824745 conn(0x7f5fac0fbd50 msgr2=0x7f5fac0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:30.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.023+0000 7f5fb2a00640 1 -- 172.21.15.97:0/270824745 shutdown_connections 2024-03-26T18:56:30.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.023+0000 7f5fb2a00640 1 -- 172.21.15.97:0/270824745 wait complete. 2024-03-26T18:56:30.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.024+0000 7f5fb2a00640 1 Processor -- start 2024-03-26T18:56:30.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.024+0000 7f5fb2a00640 1 -- start start 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.024+0000 7f5fb2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1001b0 0x7f5fac199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.024+0000 7f5fb2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fac100b10 0x7f5fac19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.024+0000 7f5fb2a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fac1014b0 0x7f5fac1a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.024+0000 7f5fb2a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac10f780 con 0x7f5fac100b10 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.024+0000 7f5fb2a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac10f600 con 0x7f5fac1014b0 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.024+0000 7f5fb2a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac10f900 con 0x7f5fac1001b0 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fac100b10 0x7f5fac19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1001b0 0x7f5fac199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fac100b10 0x7f5fac19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54964/0 (socket says 172.21.15.97:54964) 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1001b0 0x7f5fac199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34794/0 (socket says 172.21.15.97:34794) 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fb0a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fac1014b0 0x7f5fac1a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fab400640 1 -- 172.21.15.97:0/2036904119 learned_addr learned my addr 172.21.15.97:0/2036904119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:30.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fb0a00640 1 -- 172.21.15.97:0/2036904119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1001b0 msgr2=0x7f5fac199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fb0a00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1001b0 0x7f5fac199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fb0a00640 1 -- 172.21.15.97:0/2036904119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fac100b10 msgr2=0x7f5fac19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fb0a00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fac100b10 0x7f5fac19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fb0a00640 1 -- 172.21.15.97:0/2036904119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac1a1a40 con 0x7f5fac1014b0 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fabe00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1001b0 0x7f5fac199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fab400640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fac100b10 0x7f5fac19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.025+0000 7f5fb0a00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fac1014b0 0x7f5fac1a13d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009fd0 tx=0x7f5fa400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.026+0000 7f5fa8c00640 1 -- 172.21.15.97:0/2036904119 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa400ce20 con 0x7f5fac1014b0 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.026+0000 7f5fa8c00640 1 -- 172.21.15.97:0/2036904119 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa4005020 con 0x7f5fac1014b0 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.026+0000 7f5fb2a00640 1 -- 172.21.15.97:0/2036904119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac1a1d30 con 0x7f5fac1014b0 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.026+0000 7f5fa8c00640 1 -- 172.21.15.97:0/2036904119 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4010720 con 0x7f5fac1014b0 2024-03-26T18:56:30.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.026+0000 7f5fb2a00640 1 -- 172.21.15.97:0/2036904119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1a2220 con 0x7f5fac1014b0 2024-03-26T18:56:30.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.027+0000 7f5fb2a00640 1 -- 172.21.15.97:0/2036904119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac10b0e0 con 0x7f5fac1014b0 2024-03-26T18:56:30.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.028+0000 7f5fa8c00640 1 -- 172.21.15.97:0/2036904119 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f5fa4005540 con 0x7f5fac1014b0 2024-03-26T18:56:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.030+0000 7f5fa8c00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f900a2f20 0x7f5f900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.030+0000 7f5fa8c00640 1 -- 172.21.15.97:0/2036904119 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f5fa40cf970 con 0x7f5fac1014b0 2024-03-26T18:56:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.030+0000 7f5fabe00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f900a2f20 0x7f5f900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.030+0000 7f5fabe00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f900a2f20 0x7f5f900a53e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c02c2d0 tx=0x7f5f9c02c910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.033+0000 7f5fa8c00640 1 -- 172.21.15.97:0/2036904119 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5fa409c850 con 0x7f5fac1014b0 2024-03-26T18:56:30.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T18:56:30.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] up:active 2024-03-26T18:56:30.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T18:56:30.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-26T18:56:30.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-26T18:56:30.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-26T18:56:30.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: pgmap v315: 97 pgs: 83 active+clean, 14 creating+peering; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 5 op/s 2024-03-26T18:56:30.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1952661037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T18:56:30.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T18:56:30.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T18:56:30.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] up:active 2024-03-26T18:56:30.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T18:56:30.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T18:56:30.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-03-26T18:56:30.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:29 smithi097 ceph-mon[31955]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-26T18:56:30.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T18:56:30.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] up:active 2024-03-26T18:56:30.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T18:56:30.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-26T18:56:30.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-26T18:56:30.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-26T18:56:30.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: pgmap v315: 97 pgs: 83 active+clean, 14 creating+peering; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 5 op/s 2024-03-26T18:56:30.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1952661037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T18:56:30.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T18:56:30.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T18:56:30.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] up:active 2024-03-26T18:56:30.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T18:56:30.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T18:56:30.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-03-26T18:56:30.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:29 smithi144 ceph-mon[36053]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-26T18:56:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T18:56:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] up:active 2024-03-26T18:56:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T18:56:30.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: pgmap v315: 97 pgs: 83 active+clean, 14 creating+peering; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 5 op/s 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1952661037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] up:active 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-03-26T18:56:30.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:29 smithi105 ceph-mon[36270]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-26T18:56:30.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.261+0000 7f5fb2a00640 1 -- 172.21.15.97:0/2036904119 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5fac061d00 con 0x7f5f900a2f20 2024-03-26T18:56:30.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.289+0000 7f5fa8c00640 1 -- 172.21.15.97:0/2036904119 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5fac061d00 con 0x7f5f900a2f20 2024-03-26T18:56:30.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 -- 172.21.15.97:0/2036904119 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f900a2f20 msgr2=0x7f5f900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f900a2f20 0x7f5f900a53e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c02c2d0 tx=0x7f5f9c02c910 comp rx=0 tx=0).stop 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 -- 172.21.15.97:0/2036904119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fac1014b0 msgr2=0x7f5fac1a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fac1014b0 0x7f5fac1a13d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009fd0 tx=0x7f5fa400ef10 comp rx=0 tx=0).stop 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 -- 172.21.15.97:0/2036904119 shutdown_connections 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f900a2f20 0x7f5f900a53e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fac1014b0 0x7f5fac1a13d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fac100b10 0x7f5fac19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 --2- 172.21.15.97:0/2036904119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1001b0 0x7f5fac199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 -- 172.21.15.97:0/2036904119 >> 172.21.15.97:0/2036904119 conn(0x7f5fac0fbd50 msgr2=0x7f5fac0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 -- 172.21.15.97:0/2036904119 shutdown_connections 2024-03-26T18:56:30.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.292+0000 7f5fb2a00640 1 -- 172.21.15.97:0/2036904119 wait complete. 2024-03-26T18:56:30.306 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-26T18:56:30.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.553+0000 7faa93200640 1 -- 172.21.15.97:0/2457138866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa941017c0 msgr2=0x7faa94101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:30.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.553+0000 7faa93200640 1 --2- 172.21.15.97:0/2457138866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa941017c0 0x7faa94101be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faa84009900 tx=0x7faa8402bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:30.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.553+0000 7faa93200640 1 -- 172.21.15.97:0/2457138866 shutdown_connections 2024-03-26T18:56:30.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.553+0000 7faa93200640 1 --2- 172.21.15.97:0/2457138866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa94103370 0x7faa94109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.553+0000 7faa93200640 1 --2- 172.21.15.97:0/2457138866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa941029b0 0x7faa94102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.553+0000 7faa93200640 1 --2- 172.21.15.97:0/2457138866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa941017c0 0x7faa94101be0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.554+0000 7faa93200640 1 -- 172.21.15.97:0/2457138866 >> 172.21.15.97:0/2457138866 conn(0x7faa940fcd30 msgr2=0x7faa940ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:30.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.554+0000 7faa93200640 1 -- 172.21.15.97:0/2457138866 shutdown_connections 2024-03-26T18:56:30.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.554+0000 7faa93200640 1 -- 172.21.15.97:0/2457138866 wait complete. 2024-03-26T18:56:30.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa93200640 1 Processor -- start 2024-03-26T18:56:30.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa93200640 1 -- start start 2024-03-26T18:56:30.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa93200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa941017c0 0x7faa94199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa93200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa941029b0 0x7faa9419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa93200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94103370 0x7faa941a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa93200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa9410d460 con 0x7faa94103370 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa93200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faa9410d2e0 con 0x7faa941017c0 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa93200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa9410d5e0 con 0x7faa941029b0 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa91e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa941017c0 0x7faa94199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa91e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa941017c0 0x7faa94199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60362/0 (socket says 172.21.15.97:60362) 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa91e00640 1 -- 172.21.15.97:0/253722899 learned_addr learned my addr 172.21.15.97:0/253722899 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.555+0000 7faa92800640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94103370 0x7faa941a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa91400640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa941029b0 0x7faa9419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa91e00640 1 -- 172.21.15.97:0/253722899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa941029b0 msgr2=0x7faa9419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa91e00640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa941029b0 0x7faa9419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa91e00640 1 -- 172.21.15.97:0/253722899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94103370 msgr2=0x7faa941a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa91e00640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94103370 0x7faa941a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa91e00640 1 -- 172.21.15.97:0/253722899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa941a1960 con 0x7faa941017c0 2024-03-26T18:56:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa91400640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa941029b0 0x7faa9419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:56:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa91e00640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa941017c0 0x7faa94199d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faa84000c00 tx=0x7faa8402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa8aa00640 1 -- 172.21.15.97:0/253722899 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84043020 con 0x7faa941017c0 2024-03-26T18:56:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa8aa00640 1 -- 172.21.15.97:0/253722899 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa840054e0 con 0x7faa941017c0 2024-03-26T18:56:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa93200640 1 -- 172.21.15.97:0/253722899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa941a1bf0 con 0x7faa941017c0 2024-03-26T18:56:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa8aa00640 1 -- 172.21.15.97:0/253722899 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84003c40 con 0x7faa941017c0 2024-03-26T18:56:30.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.556+0000 7faa93200640 1 -- 172.21.15.97:0/253722899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa941a2140 con 0x7faa941017c0 2024-03-26T18:56:30.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.558+0000 7faa93200640 1 -- 172.21.15.97:0/253722899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa941a2310 con 0x7faa941017c0 2024-03-26T18:56:30.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.559+0000 7faa8aa00640 1 -- 172.21.15.97:0/253722899 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7faa84046020 con 0x7faa941017c0 2024-03-26T18:56:30.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.561+0000 7faa8aa00640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7faa780a2f20 0x7faa780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:30.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.561+0000 7faa8aa00640 1 -- 172.21.15.97:0/253722899 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7faa840f1470 con 0x7faa941017c0 2024-03-26T18:56:30.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.562+0000 7faa91400640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7faa780a2f20 0x7faa780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:30.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.562+0000 7faa91400640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7faa780a2f20 0x7faa780a53e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa9419b250 tx=0x7faa80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:30.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.564+0000 7faa8aa00640 1 -- 172.21.15.97:0/253722899 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faa8404d020 con 0x7faa941017c0 2024-03-26T18:56:30.603 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:56:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:56:30.271+0000 7f46e3800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:56:30.603 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:56:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:56:30.271+0000 7f46e3800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:56:30.603 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:56:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:56:30.271+0000 7f46e3800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:56:30.604 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:56:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:56:30.271+0000 7f46e3800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:56:30.604 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:56:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:56:30.271+0000 7f46e3800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.804+0000 7faa93200640 1 -- 172.21.15.97:0/253722899 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faa941098f0 con 0x7faa780a2f20 2024-03-26T18:56:30.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.844+0000 7faa8aa00640 1 -- 172.21.15.97:0/253722899 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa941098f0 con 0x7faa780a2f20 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 -- 172.21.15.97:0/253722899 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7faa780a2f20 msgr2=0x7faa780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7faa780a2f20 0x7faa780a53e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa9419b250 tx=0x7faa80005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 -- 172.21.15.97:0/253722899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa941017c0 msgr2=0x7faa94199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa941017c0 0x7faa94199d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faa84000c00 tx=0x7faa8402c650 comp rx=0 tx=0).stop 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 -- 172.21.15.97:0/253722899 shutdown_connections 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7faa780a2f20 0x7faa780a53e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94103370 0x7faa941a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa941029b0 0x7faa9419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 --2- 172.21.15.97:0/253722899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa941017c0 0x7faa94199d40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 -- 172.21.15.97:0/253722899 >> 172.21.15.97:0/253722899 conn(0x7faa940fcd30 msgr2=0x7faa941081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 -- 172.21.15.97:0/253722899 shutdown_connections 2024-03-26T18:56:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:30.846+0000 7faa93200640 1 -- 172.21.15.97:0/253722899 wait complete. 2024-03-26T18:56:30.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-26T18:56:31.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.046+0000 7f5045000640 1 -- 172.21.15.97:0/319035493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5040102ac0 msgr2=0x7f5040102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:31.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.046+0000 7f5045000640 1 --2- 172.21.15.97:0/319035493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5040102ac0 0x7f5040102f40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5028009900 tx=0x7f502802bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:31.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.046+0000 7f5045000640 1 -- 172.21.15.97:0/319035493 shutdown_connections 2024-03-26T18:56:31.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.046+0000 7f5045000640 1 --2- 172.21.15.97:0/319035493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5040103480 0x7f5040109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.046+0000 7f5045000640 1 --2- 172.21.15.97:0/319035493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5040102ac0 0x7f5040102f40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.046+0000 7f5045000640 1 --2- 172.21.15.97:0/319035493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50401018d0 0x7f5040101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.046+0000 7f5045000640 1 -- 172.21.15.97:0/319035493 >> 172.21.15.97:0/319035493 conn(0x7f50400fce60 msgr2=0x7f50400ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:31.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.046+0000 7f5045000640 1 -- 172.21.15.97:0/319035493 shutdown_connections 2024-03-26T18:56:31.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.046+0000 7f5045000640 1 -- 172.21.15.97:0/319035493 wait complete. 2024-03-26T18:56:31.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.047+0000 7f5045000640 1 Processor -- start 2024-03-26T18:56:31.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.047+0000 7f5045000640 1 -- start start 2024-03-26T18:56:31.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5045000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50401018d0 0x7f5040199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5045000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5040102ac0 0x7f504019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5045000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5040103480 0x7f50401a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5045000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f504010d5a0 con 0x7f5040103480 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5045000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f504010d420 con 0x7f50401018d0 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5045000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f504010d720 con 0x7f5040102ac0 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f503de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50401018d0 0x7f5040199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5037e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5040102ac0 0x7f504019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f503de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50401018d0 0x7f5040199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46478/0 (socket says 172.21.15.97:46478) 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5037e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5040102ac0 0x7f504019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:59324/0 (socket says 172.21.15.97:59324) 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f503e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5040103480 0x7f50401a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f503de00640 1 -- 172.21.15.97:0/1507296575 learned_addr learned my addr 172.21.15.97:0/1507296575 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5037e00640 1 -- 172.21.15.97:0/1507296575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50401018d0 msgr2=0x7f5040199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5037e00640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50401018d0 0x7f5040199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5037e00640 1 -- 172.21.15.97:0/1507296575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5040103480 msgr2=0x7f50401a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:31.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5037e00640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5040103480 0x7f50401a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f5037e00640 1 -- 172.21.15.97:0/1507296575 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50401a1ad0 con 0x7f5040102ac0 2024-03-26T18:56:31.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.048+0000 7f503de00640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50401018d0 0x7f5040199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:31.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.049+0000 7f5037e00640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5040102ac0 0x7f504019a3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f502802c170 tx=0x7f502802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:31.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.049+0000 7f5036a00640 1 -- 172.21.15.97:0/1507296575 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028043020 con 0x7f5040102ac0 2024-03-26T18:56:31.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.049+0000 7f5045000640 1 -- 172.21.15.97:0/1507296575 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50401a1d60 con 0x7f5040102ac0 2024-03-26T18:56:31.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.049+0000 7f5036a00640 1 -- 172.21.15.97:0/1507296575 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50280054e0 con 0x7f5040102ac0 2024-03-26T18:56:31.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.049+0000 7f5036a00640 1 -- 172.21.15.97:0/1507296575 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028003c10 con 0x7f5040102ac0 2024-03-26T18:56:31.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.049+0000 7f5045000640 1 -- 172.21.15.97:0/1507296575 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50401a2220 con 0x7f5040102ac0 2024-03-26T18:56:31.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.050+0000 7f5045000640 1 -- 172.21.15.97:0/1507296575 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50401a23f0 con 0x7f5040102ac0 2024-03-26T18:56:31.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.053+0000 7f5036a00640 1 -- 172.21.15.97:0/1507296575 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f5028046020 con 0x7f5040102ac0 2024-03-26T18:56:31.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.054+0000 7f5036a00640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f50240a2f20 0x7f50240a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:31.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.054+0000 7f5036a00640 1 -- 172.21.15.97:0/1507296575 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f50280f1360 con 0x7f5040102ac0 2024-03-26T18:56:31.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.054+0000 7f503de00640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f50240a2f20 0x7f50240a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:31.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.055+0000 7f503de00640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f50240a2f20 0x7f50240a53e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f50300095f0 tx=0x7f5030005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:31.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.058+0000 7f5036a00640 1 -- 172.21.15.97:0/1507296575 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f502804d020 con 0x7f5040102ac0 2024-03-26T18:56:31.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.293+0000 7f5045000640 1 -- 172.21.15.97:0/1507296575 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f5040075210 con 0x7f50240a2f20 2024-03-26T18:56:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.301+0000 7f5036a00640 1 -- 172.21.15.97:0/1507296575 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5040075210 con 0x7f50240a2f20 2024-03-26T18:56:31.294 INFO:teuthology.orchestra.run.smithi097.stdout:/volumes/qa/sv_0/281b1ddd-1e5b-4212-a4ad-49368a4e4876 2024-03-26T18:56:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 -- 172.21.15.97:0/1507296575 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f50240a2f20 msgr2=0x7f50240a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f50240a2f20 0x7f50240a53e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f50300095f0 tx=0x7f5030005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 -- 172.21.15.97:0/1507296575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5040102ac0 msgr2=0x7f504019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5040102ac0 0x7f504019a3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f502802c170 tx=0x7f502802c650 comp rx=0 tx=0).stop 2024-03-26T18:56:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 -- 172.21.15.97:0/1507296575 shutdown_connections 2024-03-26T18:56:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f50240a2f20 0x7f50240a53e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5040103480 0x7f50401a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5040102ac0 0x7f504019a3f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 --2- 172.21.15.97:0/1507296575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50401018d0 0x7f5040199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 -- 172.21.15.97:0/1507296575 >> 172.21.15.97:0/1507296575 conn(0x7f50400fce60 msgr2=0x7f5040073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 -- 172.21.15.97:0/1507296575 shutdown_connections 2024-03-26T18:56:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.304+0000 7f5045000640 1 -- 172.21.15.97:0/1507296575 wait complete. 2024-03-26T18:56:31.317 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-26T18:56:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.516+0000 7f2f6e400640 1 -- 172.21.15.97:0/195045783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f68104cd0 msgr2=0x7f2f68105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.516+0000 7f2f6e400640 1 --2- 172.21.15.97:0/195045783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f68104cd0 0x7f2f68105150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009900 tx=0x7f2f5402bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.516+0000 7f2f6e400640 1 -- 172.21.15.97:0/195045783 shutdown_connections 2024-03-26T18:56:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.516+0000 7f2f6e400640 1 --2- 172.21.15.97:0/195045783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f68105690 0x7f2f6810bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.516+0000 7f2f6e400640 1 --2- 172.21.15.97:0/195045783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f68104cd0 0x7f2f68105150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.516+0000 7f2f6e400640 1 --2- 172.21.15.97:0/195045783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f68103ae0 0x7f2f68103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.516+0000 7f2f6e400640 1 -- 172.21.15.97:0/195045783 >> 172.21.15.97:0/195045783 conn(0x7f2f680ff070 msgr2=0x7f2f68101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.516+0000 7f2f6e400640 1 -- 172.21.15.97:0/195045783 shutdown_connections 2024-03-26T18:56:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.516+0000 7f2f6e400640 1 -- 172.21.15.97:0/195045783 wait complete. 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.517+0000 7f2f6e400640 1 Processor -- start 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.517+0000 7f2f6e400640 1 -- start start 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.517+0000 7f2f6e400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f68103ae0 0x7f2f68199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.517+0000 7f2f6e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f68104cd0 0x7f2f6819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.517+0000 7f2f6e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f68105690 0x7f2f681a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.517+0000 7f2f6e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6810f790 con 0x7f2f68104cd0 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.517+0000 7f2f6e400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6810f610 con 0x7f2f68103ae0 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.517+0000 7f2f6e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6810f910 con 0x7f2f68105690 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f68103ae0 0x7f2f68199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f68103ae0 0x7f2f68199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46504/0 (socket says 172.21.15.97:46504) 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f66a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f68104cd0 0x7f2f6819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67400640 1 -- 172.21.15.97:0/4045061073 learned_addr learned my addr 172.21.15.97:0/4045061073 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:31.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67e00640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f68105690 0x7f2f681a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67400640 1 -- 172.21.15.97:0/4045061073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f68105690 msgr2=0x7f2f681a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67400640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f68105690 0x7f2f681a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67400640 1 -- 172.21.15.97:0/4045061073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f68104cd0 msgr2=0x7f2f6819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67400640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f68104cd0 0x7f2f6819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67400640 1 -- 172.21.15.97:0/4045061073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f681a1a50 con 0x7f2f68103ae0 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67e00640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f68105690 0x7f2f681a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.518+0000 7f2f67400640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f68103ae0 0x7f2f68199e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800e9c0 tx=0x7f2f5800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.519+0000 7f2f5fe00640 1 -- 172.21.15.97:0/4045061073 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800ccc0 con 0x7f2f68103ae0 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.519+0000 7f2f5fe00640 1 -- 172.21.15.97:0/4045061073 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f580035f0 con 0x7f2f68103ae0 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.519+0000 7f2f6e400640 1 -- 172.21.15.97:0/4045061073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f681a1d40 con 0x7f2f68103ae0 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.519+0000 7f2f6e400640 1 -- 172.21.15.97:0/4045061073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f681a21b0 con 0x7f2f68103ae0 2024-03-26T18:56:31.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.520+0000 7f2f5fe00640 1 -- 172.21.15.97:0/4045061073 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800ce60 con 0x7f2f68103ae0 2024-03-26T18:56:31.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.520+0000 7f2f6e400640 1 -- 172.21.15.97:0/4045061073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6810ae50 con 0x7f2f68103ae0 2024-03-26T18:56:31.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.522+0000 7f2f5fe00640 1 -- 172.21.15.97:0/4045061073 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f2f580037a0 con 0x7f2f68103ae0 2024-03-26T18:56:31.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.524+0000 7f2f5fe00640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2f4c0a2f20 0x7f2f4c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.525+0000 7f2f5fe00640 1 -- 172.21.15.97:0/4045061073 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f2f580cf670 con 0x7f2f68103ae0 2024-03-26T18:56:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.525+0000 7f2f66a00640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2f4c0a2f20 0x7f2f4c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.526+0000 7f2f66a00640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2f4c0a2f20 0x7f2f4c0a53e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2f5402c240 tx=0x7f2f5402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:31.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.532+0000 7f2f5fe00640 1 -- 172.21.15.97:0/4045061073 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2f5809c550 con 0x7f2f68103ae0 2024-03-26T18:56:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.844+0000 7f2f5fe00640 1 -- 172.21.15.97:0/4045061073 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f2f580984c0 con 0x7f2f68103ae0 2024-03-26T18:56:31.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.877+0000 7f2f6e400640 1 -- 172.21.15.97:0/4045061073 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2f68061d00 con 0x7f2f4c0a2f20 2024-03-26T18:56:31.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.906+0000 7f2f5fe00640 1 -- 172.21.15.97:0/4045061073 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2f68061d00 con 0x7f2f4c0a2f20 2024-03-26T18:56:31.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 -- 172.21.15.97:0/4045061073 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2f4c0a2f20 msgr2=0x7f2f4c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:31.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2f4c0a2f20 0x7f2f4c0a53e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2f5402c240 tx=0x7f2f5402df90 comp rx=0 tx=0).stop 2024-03-26T18:56:31.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 -- 172.21.15.97:0/4045061073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f68103ae0 msgr2=0x7f2f68199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:31.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f68103ae0 0x7f2f68199e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800e9c0 tx=0x7f2f5800edc0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 -- 172.21.15.97:0/4045061073 shutdown_connections 2024-03-26T18:56:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2f4c0a2f20 0x7f2f4c0a53e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f68105690 0x7f2f681a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f68104cd0 0x7f2f6819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 --2- 172.21.15.97:0/4045061073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f68103ae0 0x7f2f68199e30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 -- 172.21.15.97:0/4045061073 >> 172.21.15.97:0/4045061073 conn(0x7f2f680ff070 msgr2=0x7f2f68108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 -- 172.21.15.97:0/4045061073 shutdown_connections 2024-03-26T18:56:31.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:31.909+0000 7f2f6e400640 1 -- 172.21.15.97:0/4045061073 wait complete. 2024-03-26T18:56:31.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-26T18:56:32.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:31 smithi097 ceph-mon[31955]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:32.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:31 smithi097 ceph-mon[31955]: from='client.24677 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:32.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:31 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] up:active 2024-03-26T18:56:32.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:31 smithi097 ceph-mon[31955]: daemon mds.d assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T18:56:32.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:31 smithi097 ceph-mon[31955]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T18:56:32.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:31 smithi097 ceph-mon[31955]: Cluster is now healthy 2024-03-26T18:56:32.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:31 smithi097 ceph-mon[31955]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-03-26T18:56:32.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:31 smithi097 ceph-mon[31955]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-26T18:56:32.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:31 smithi097 ceph-mon[31955]: daemon mds.d is now active in filesystem cephfs as rank 4 2024-03-26T18:56:32.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:31 smithi097 ceph-mon[31955]: pgmap v316: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 790 B/s rd, 4.9 KiB/s wr, 16 op/s 2024-03-26T18:56:32.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.127+0000 7fce8f200640 1 -- 172.21.15.97:0/3658412040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90101890 msgr2=0x7fce90101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:32.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.127+0000 7fce8f200640 1 --2- 172.21.15.97:0/3658412040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90101890 0x7fce90101d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce78009900 tx=0x7fce7802bc40 comp rx=0 tx=0).stop 2024-03-26T18:56:32.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.127+0000 7fce8f200640 1 -- 172.21.15.97:0/3658412040 shutdown_connections 2024-03-26T18:56:32.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.127+0000 7fce8f200640 1 --2- 172.21.15.97:0/3658412040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce90102250 0x7fce90108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.127+0000 7fce8f200640 1 --2- 172.21.15.97:0/3658412040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90101890 0x7fce90101d10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.127+0000 7fce8f200640 1 --2- 172.21.15.97:0/3658412040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce901006a0 0x7fce90100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.127+0000 7fce8f200640 1 -- 172.21.15.97:0/3658412040 >> 172.21.15.97:0/3658412040 conn(0x7fce900fbc30 msgr2=0x7fce900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:32.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.127+0000 7fce8f200640 1 -- 172.21.15.97:0/3658412040 shutdown_connections 2024-03-26T18:56:32.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.127+0000 7fce8f200640 1 -- 172.21.15.97:0/3658412040 wait complete. 2024-03-26T18:56:32.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.128+0000 7fce8f200640 1 Processor -- start 2024-03-26T18:56:32.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.128+0000 7fce8f200640 1 -- start start 2024-03-26T18:56:32.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce901006a0 0x7fce90069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:32.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90101890 0x7fce9019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:32.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce90102250 0x7fce9019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:32.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010c320 con 0x7fce90102250 2024-03-26T18:56:32.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8f200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010c1a0 con 0x7fce90101890 2024-03-26T18:56:32.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010c4a0 con 0x7fce901006a0 2024-03-26T18:56:32.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90101890 0x7fce9019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:32.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90101890 0x7fce9019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46528/0 (socket says 172.21.15.97:46528) 2024-03-26T18:56:32.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce901006a0 0x7fce90069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8d400640 1 -- 172.21.15.97:0/2494401500 learned_addr learned my addr 172.21.15.97:0/2494401500 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8d400640 1 -- 172.21.15.97:0/2494401500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce901006a0 msgr2=0x7fce90069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8d400640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce901006a0 0x7fce90069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8d400640 1 -- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce90102250 msgr2=0x7fce9019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8e800640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce90102250 0x7fce9019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8d400640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce90102250 0x7fce9019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8d400640 1 -- 172.21.15.97:0/2494401500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9019f730 con 0x7fce90101890 2024-03-26T18:56:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.129+0000 7fce8e800640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce90102250 0x7fce9019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:56:32.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.130+0000 7fce8d400640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90101890 0x7fce9019c800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fce78009230 tx=0x7fce78005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:32.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.130+0000 7fce86a00640 1 -- 172.21.15.97:0/2494401500 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78041070 con 0x7fce90101890 2024-03-26T18:56:32.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.130+0000 7fce86a00640 1 -- 172.21.15.97:0/2494401500 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce780044d0 con 0x7fce90101890 2024-03-26T18:56:32.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.130+0000 7fce8f200640 1 -- 172.21.15.97:0/2494401500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9019f960 con 0x7fce90101890 2024-03-26T18:56:32.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.130+0000 7fce86a00640 1 -- 172.21.15.97:0/2494401500 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78034440 con 0x7fce90101890 2024-03-26T18:56:32.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.130+0000 7fce8f200640 1 -- 172.21.15.97:0/2494401500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9019fe50 con 0x7fce90101890 2024-03-26T18:56:32.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.131+0000 7fce8f200640 1 -- 172.21.15.97:0/2494401500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce90107a90 con 0x7fce90101890 2024-03-26T18:56:32.130 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:31 smithi144 ceph-mon[36053]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:32.130 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:31 smithi144 ceph-mon[36053]: from='client.24677 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:32.130 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:31 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] up:active 2024-03-26T18:56:32.130 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:31 smithi144 ceph-mon[36053]: daemon mds.d assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T18:56:32.130 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:31 smithi144 ceph-mon[36053]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T18:56:32.130 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:31 smithi144 ceph-mon[36053]: Cluster is now healthy 2024-03-26T18:56:32.130 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:31 smithi144 ceph-mon[36053]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-03-26T18:56:32.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:31 smithi144 ceph-mon[36053]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-26T18:56:32.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:31 smithi144 ceph-mon[36053]: daemon mds.d is now active in filesystem cephfs as rank 4 2024-03-26T18:56:32.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:31 smithi144 ceph-mon[36053]: pgmap v316: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 790 B/s rd, 4.9 KiB/s wr, 16 op/s 2024-03-26T18:56:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.133+0000 7fce86a00640 1 -- 172.21.15.97:0/2494401500 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fce78044070 con 0x7fce90101890 2024-03-26T18:56:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.135+0000 7fce86a00640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fce740a2fb0 0x7fce740a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.135+0000 7fce86a00640 1 -- 172.21.15.97:0/2494401500 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fce780f0ff0 con 0x7fce90101890 2024-03-26T18:56:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.135+0000 7fce8de00640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fce740a2fb0 0x7fce740a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.136+0000 7fce8de00640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fce740a2fb0 0x7fce740a5470 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fce901016f0 tx=0x7fce80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:32.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.142+0000 7fce86a00640 1 -- 172.21.15.97:0/2494401500 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fce780bdee0 con 0x7fce90101890 2024-03-26T18:56:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:31 smithi105 ceph-mon[36270]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:31 smithi105 ceph-mon[36270]: from='client.24677 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:31 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] up:active 2024-03-26T18:56:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:31 smithi105 ceph-mon[36270]: daemon mds.d assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T18:56:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:31 smithi105 ceph-mon[36270]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T18:56:32.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:31 smithi105 ceph-mon[36270]: Cluster is now healthy 2024-03-26T18:56:32.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:31 smithi105 ceph-mon[36270]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-03-26T18:56:32.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:31 smithi105 ceph-mon[36270]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-26T18:56:32.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:31 smithi105 ceph-mon[36270]: daemon mds.d is now active in filesystem cephfs as rank 4 2024-03-26T18:56:32.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:31 smithi105 ceph-mon[36270]: pgmap v316: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 790 B/s rd, 4.9 KiB/s wr, 16 op/s 2024-03-26T18:56:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.381+0000 7fce8f200640 1 -- 172.21.15.97:0/2494401500 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fce90061d90 con 0x7fce740a2fb0 2024-03-26T18:56:32.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.388+0000 7fce86a00640 1 -- 172.21.15.97:0/2494401500 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fce90061d90 con 0x7fce740a2fb0 2024-03-26T18:56:32.382 INFO:teuthology.orchestra.run.smithi097.stdout:/volumes/qa/sv_1/7cc1b6af-7058-492a-908c-827e2e8de910 2024-03-26T18:56:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 -- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fce740a2fb0 msgr2=0x7fce740a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fce740a2fb0 0x7fce740a5470 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fce901016f0 tx=0x7fce80005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 -- 172.21.15.97:0/2494401500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90101890 msgr2=0x7fce9019c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90101890 0x7fce9019c800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fce78009230 tx=0x7fce78005d50 comp rx=0 tx=0).stop 2024-03-26T18:56:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 -- 172.21.15.97:0/2494401500 shutdown_connections 2024-03-26T18:56:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fce740a2fb0 0x7fce740a5470 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce90102250 0x7fce9019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90101890 0x7fce9019c800 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 --2- 172.21.15.97:0/2494401500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce901006a0 0x7fce90069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 -- 172.21.15.97:0/2494401500 >> 172.21.15.97:0/2494401500 conn(0x7fce900fbc30 msgr2=0x7fce90104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 -- 172.21.15.97:0/2494401500 shutdown_connections 2024-03-26T18:56:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.391+0000 7fce8f200640 1 -- 172.21.15.97:0/2494401500 wait complete. 2024-03-26T18:56:32.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:56:32.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.617+0000 7ff650a00640 1 -- 172.21.15.97:0/2053026241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff64c101720 msgr2=0x7ff64c101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.617+0000 7ff650a00640 1 --2- 172.21.15.97:0/2053026241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff64c101720 0x7ff64c101b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009900 tx=0x7ff63c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.617+0000 7ff650a00640 1 -- 172.21.15.97:0/2053026241 shutdown_connections 2024-03-26T18:56:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.617+0000 7ff650a00640 1 --2- 172.21.15.97:0/2053026241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff64c1032d0 0x7ff64c109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.617+0000 7ff650a00640 1 --2- 172.21.15.97:0/2053026241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff64c102910 0x7ff64c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.617+0000 7ff650a00640 1 --2- 172.21.15.97:0/2053026241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff64c101720 0x7ff64c101b40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.618+0000 7ff650a00640 1 -- 172.21.15.97:0/2053026241 >> 172.21.15.97:0/2053026241 conn(0x7ff64c0fcc90 msgr2=0x7ff64c0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.618+0000 7ff650a00640 1 -- 172.21.15.97:0/2053026241 shutdown_connections 2024-03-26T18:56:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.618+0000 7ff650a00640 1 -- 172.21.15.97:0/2053026241 wait complete. 2024-03-26T18:56:32.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.619+0000 7ff650a00640 1 Processor -- start 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.619+0000 7ff650a00640 1 -- start start 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.619+0000 7ff650a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff64c101720 0x7ff64c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.619+0000 7ff650a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff64c102910 0x7ff64c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff650a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff64c1032d0 0x7ff64c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff650a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10d370 con 0x7ff64c102910 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff650a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10d1f0 con 0x7ff64c101720 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff650a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10d4f0 con 0x7ff64c1032d0 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff64c101720 0x7ff64c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff64c101720 0x7ff64c197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46554/0 (socket says 172.21.15.97:46554) 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64b400640 1 -- 172.21.15.97:0/2134608182 learned_addr learned my addr 172.21.15.97:0/2134608182 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64aa00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff64c102910 0x7ff64c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64be00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff64c1032d0 0x7ff64c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64b400640 1 -- 172.21.15.97:0/2134608182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff64c1032d0 msgr2=0x7ff64c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64b400640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff64c1032d0 0x7ff64c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64b400640 1 -- 172.21.15.97:0/2134608182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff64c102910 msgr2=0x7ff64c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:32.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64b400640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff64c102910 0x7ff64c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64b400640 1 -- 172.21.15.97:0/2134608182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c19f660 con 0x7ff64c101720 2024-03-26T18:56:32.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.620+0000 7ff64b400640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff64c101720 0x7ff64c197a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff63c000c00 tx=0x7ff63c02c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:32.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.621+0000 7ff643e00640 1 -- 172.21.15.97:0/2134608182 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c041070 con 0x7ff64c101720 2024-03-26T18:56:32.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.621+0000 7ff650a00640 1 -- 172.21.15.97:0/2134608182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c19f8f0 con 0x7ff64c101720 2024-03-26T18:56:32.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.621+0000 7ff643e00640 1 -- 172.21.15.97:0/2134608182 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff63c004450 con 0x7ff64c101720 2024-03-26T18:56:32.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.621+0000 7ff643e00640 1 -- 172.21.15.97:0/2134608182 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c034440 con 0x7ff64c101720 2024-03-26T18:56:32.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.621+0000 7ff650a00640 1 -- 172.21.15.97:0/2134608182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c19fde0 con 0x7ff64c101720 2024-03-26T18:56:32.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.622+0000 7ff650a00640 1 -- 172.21.15.97:0/2134608182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64c108a60 con 0x7ff64c101720 2024-03-26T18:56:32.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.624+0000 7ff643e00640 1 -- 172.21.15.97:0/2134608182 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7ff63c044070 con 0x7ff64c101720 2024-03-26T18:56:32.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.625+0000 7ff643e00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff6300a2fb0 0x7ff6300a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.625+0000 7ff643e00640 1 -- 172.21.15.97:0/2134608182 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff63c0f0e90 con 0x7ff64c101720 2024-03-26T18:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.625+0000 7ff64aa00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff6300a2fb0 0x7ff6300a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.626+0000 7ff64aa00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff6300a2fb0 0x7ff6300a5470 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff64c198f50 tx=0x7ff638005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.629+0000 7ff643e00640 1 -- 172.21.15.97:0/2134608182 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff63c0bd760 con 0x7ff64c101720 2024-03-26T18:56:32.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.871+0000 7ff650a00640 1 -- 172.21.15.97:0/2134608182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff64c0624c0 con 0x7ff64c101720 2024-03-26T18:56:32.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.872+0000 7ff643e00640 1 -- 172.21.15.97:0/2134608182 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11935 (secure 0 0 0) 0x7ff63c0c1800 con 0x7ff64c101720 2024-03-26T18:56:32.866 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:32.866 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T18:56:23.642405+0000","modified":"2024-03-26T18:56:31.851339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24535,"mds_3":24494,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2391073232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2391073232},{"type":"v1","addr":"172.21.15.105:6839","nonce":2391073232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24535,24494,24481]},"id":1}]} 2024-03-26T18:56:32.866 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-03-26T18:56:32.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 -- 172.21.15.97:0/2134608182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff6300a2fb0 msgr2=0x7ff6300a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:32.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff6300a2fb0 0x7ff6300a5470 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff64c198f50 tx=0x7ff638005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:32.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 -- 172.21.15.97:0/2134608182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff64c101720 msgr2=0x7ff64c197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:32.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff64c101720 0x7ff64c197a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff63c000c00 tx=0x7ff63c02c640 comp rx=0 tx=0).stop 2024-03-26T18:56:32.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 -- 172.21.15.97:0/2134608182 shutdown_connections 2024-03-26T18:56:32.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff6300a2fb0 0x7ff6300a5470 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff64c1032d0 0x7ff64c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff64c102910 0x7ff64c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 --2- 172.21.15.97:0/2134608182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff64c101720 0x7ff64c197a40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:32.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 -- 172.21.15.97:0/2134608182 >> 172.21.15.97:0/2134608182 conn(0x7ff64c0fcc90 msgr2=0x7ff64c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:32.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 -- 172.21.15.97:0/2134608182 shutdown_connections 2024-03-26T18:56:32.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:32.875+0000 7ff650a00640 1 -- 172.21.15.97:0/2134608182 wait complete. 2024-03-26T18:56:32.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:56:33.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.092+0000 7f9c8aa00640 1 -- 172.21.15.97:0/2048818502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c840ffed0 msgr2=0x7f9c841070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:33.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.092+0000 7f9c8aa00640 1 --2- 172.21.15.97:0/2048818502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c840ffed0 0x7f9c841070b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009900 tx=0x7f9c7002bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:33.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.092+0000 7f9c8aa00640 1 -- 172.21.15.97:0/2048818502 shutdown_connections 2024-03-26T18:56:33.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.092+0000 7f9c8aa00640 1 --2- 172.21.15.97:0/2048818502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c841075f0 0x7f9c841099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.092+0000 7f9c8aa00640 1 --2- 172.21.15.97:0/2048818502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c840ffed0 0x7f9c841070b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.092+0000 7f9c8aa00640 1 --2- 172.21.15.97:0/2048818502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c840ff4e0 0x7f9c840ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.092+0000 7f9c8aa00640 1 -- 172.21.15.97:0/2048818502 >> 172.21.15.97:0/2048818502 conn(0x7f9c840fb0c0 msgr2=0x7f9c840fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:33.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.092+0000 7f9c8aa00640 1 -- 172.21.15.97:0/2048818502 shutdown_connections 2024-03-26T18:56:33.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.092+0000 7f9c8aa00640 1 -- 172.21.15.97:0/2048818502 wait complete. 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.093+0000 7f9c8aa00640 1 Processor -- start 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.093+0000 7f9c8aa00640 1 -- start start 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c8aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c840ff4e0 0x7f9c841029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c8aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c840ffed0 0x7f9c84101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c8aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c841075f0 0x7f9c84101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c8aa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410d590 con 0x7f9c840ff4e0 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c8aa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410d410 con 0x7f9c840ffed0 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c8aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410d710 con 0x7f9c841075f0 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c840ff4e0 0x7f9c841029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c83400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c840ffed0 0x7f9c84101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:33.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c83400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c840ffed0 0x7f9c84101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46568/0 (socket says 172.21.15.97:46568) 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c83400640 1 -- 172.21.15.97:0/3440779044 learned_addr learned my addr 172.21.15.97:0/3440779044 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.094+0000 7f9c88a00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c841075f0 0x7f9c84101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c83400640 1 -- 172.21.15.97:0/3440779044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c841075f0 msgr2=0x7f9c84101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c83400640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c841075f0 0x7f9c84101580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c83400640 1 -- 172.21.15.97:0/3440779044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c840ff4e0 msgr2=0x7f9c841029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c83400640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c840ff4e0 0x7f9c841029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c83400640 1 -- 172.21.15.97:0/3440779044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c84101e40 con 0x7f9c840ffed0 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c88a00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c841075f0 0x7f9c84101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c83400640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c840ffed0 0x7f9c84101040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9c70000c00 tx=0x7f9c7002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c80c00640 1 -- 172.21.15.97:0/3440779044 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70043020 con 0x7f9c840ffed0 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c80c00640 1 -- 172.21.15.97:0/3440779044 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c700054e0 con 0x7f9c840ffed0 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c80c00640 1 -- 172.21.15.97:0/3440779044 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70003cd0 con 0x7f9c840ffed0 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c8aa00640 1 -- 172.21.15.97:0/3440779044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c841a1ca0 con 0x7f9c840ffed0 2024-03-26T18:56:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.095+0000 7f9c8aa00640 1 -- 172.21.15.97:0/3440779044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c841a2170 con 0x7f9c840ffed0 2024-03-26T18:56:33.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.096+0000 7f9c8aa00640 1 -- 172.21.15.97:0/3440779044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c84105ec0 con 0x7f9c840ffed0 2024-03-26T18:56:33.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.098+0000 7f9c80c00640 1 -- 172.21.15.97:0/3440779044 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f9c70046020 con 0x7f9c840ffed0 2024-03-26T18:56:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.099+0000 7f9c80c00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9c680a3030 0x7f9c680a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.099+0000 7f9c80c00640 1 -- 172.21.15.97:0/3440779044 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9c700f1390 con 0x7f9c840ffed0 2024-03-26T18:56:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.100+0000 7f9c83e00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9c680a3030 0x7f9c680a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.100+0000 7f9c83e00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9c680a3030 0x7f9c680a54f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c74005fd0 tx=0x7f9c74009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.103+0000 7f9c80c00640 1 -- 172.21.15.97:0/3440779044 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9c7004d020 con 0x7f9c840ffed0 2024-03-26T18:56:33.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:32 smithi097 ceph-mon[31955]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:33.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:32 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] up:active 2024-03-26T18:56:33.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:32 smithi097 ceph-mon[31955]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-26T18:56:33.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:32 smithi097 ceph-mon[31955]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-26T18:56:33.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:32 smithi097 ceph-mon[31955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:56:33.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:32 smithi097 ceph-mon[31955]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:33.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:32 smithi144 ceph-mon[36053]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:33.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:32 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] up:active 2024-03-26T18:56:33.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:32 smithi144 ceph-mon[36053]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-26T18:56:33.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:32 smithi144 ceph-mon[36053]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-26T18:56:33.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:32 smithi144 ceph-mon[36053]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:56:33.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:32 smithi144 ceph-mon[36053]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:33.210 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:32 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b[66299]: 2024-03-26T18:56:32.973+0000 7fa0bf265700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-26T18:56:33.210 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:32 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b[66299]: 2024-03-26T18:56:32.973+0000 7fa0bf265700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-26T18:56:33.210 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 18:56:32 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b[66299]: 2024-03-26T18:56:32.973+0000 7fa0bf265700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-26T18:56:33.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:32 smithi105 ceph-mon[36270]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:33.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:32 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] up:active 2024-03-26T18:56:33.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:32 smithi105 ceph-mon[36270]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-26T18:56:33.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:32 smithi105 ceph-mon[36270]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-26T18:56:33.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:32 smithi105 ceph-mon[36270]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:56:33.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:32 smithi105 ceph-mon[36270]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:33.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.315+0000 7f9c8aa00640 1 -- 172.21.15.97:0/3440779044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c840624c0 con 0x7f9c840ffed0 2024-03-26T18:56:33.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.316+0000 7f9c80c00640 1 -- 172.21.15.97:0/3440779044 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22194 (secure 0 0 0) 0x7f9c700be220 con 0x7f9c840ffed0 2024-03-26T18:56:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:33.310 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:56:25.123057+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 -- 172.21.15.97:0/3440779044 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9c680a3030 msgr2=0x7f9c680a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9c680a3030 0x7f9c680a54f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c74005fd0 tx=0x7f9c74009500 comp rx=0 tx=0).stop 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 -- 172.21.15.97:0/3440779044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c840ffed0 msgr2=0x7f9c84101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c840ffed0 0x7f9c84101040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9c70000c00 tx=0x7f9c7002c650 comp rx=0 tx=0).stop 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 -- 172.21.15.97:0/3440779044 shutdown_connections 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9c680a3030 0x7f9c680a54f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c841075f0 0x7f9c84101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c840ffed0 0x7f9c84101040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 --2- 172.21.15.97:0/3440779044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c840ff4e0 0x7f9c841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 -- 172.21.15.97:0/3440779044 >> 172.21.15.97:0/3440779044 conn(0x7f9c840fb0c0 msgr2=0x7f9c84103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 -- 172.21.15.97:0/3440779044 shutdown_connections 2024-03-26T18:56:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.319+0000 7f9c8aa00640 1 -- 172.21.15.97:0/3440779044 wait complete. 2024-03-26T18:56:33.331 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-26T18:56:33.332 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T18:56:33.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.574+0000 7ff533200640 1 -- 172.21.15.97:0/1737285195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5341095c0 msgr2=0x7ff53410ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:33.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.574+0000 7ff533200640 1 --2- 172.21.15.97:0/1737285195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5341095c0 0x7ff53410ba50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009ca0 tx=0x7ff52c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.574+0000 7ff533200640 1 -- 172.21.15.97:0/1737285195 shutdown_connections 2024-03-26T18:56:33.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.574+0000 7ff533200640 1 --2- 172.21.15.97:0/1737285195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5341095c0 0x7ff53410ba50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.574+0000 7ff533200640 1 --2- 172.21.15.97:0/1737285195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff534074320 0x7ff5340747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.574+0000 7ff533200640 1 --2- 172.21.15.97:0/1737285195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff534075900 0x7ff534073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.574+0000 7ff533200640 1 -- 172.21.15.97:0/1737285195 >> 172.21.15.97:0/1737285195 conn(0x7ff5340fcc50 msgr2=0x7ff5340ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.574+0000 7ff533200640 1 -- 172.21.15.97:0/1737285195 shutdown_connections 2024-03-26T18:56:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.575+0000 7ff533200640 1 -- 172.21.15.97:0/1737285195 wait complete. 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.575+0000 7ff533200640 1 Processor -- start 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff533200640 1 -- start start 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff533200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff534074320 0x7ff534199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff533200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff534075900 0x7ff53419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff533200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341095c0 0x7ff5341a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff533200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff53410f540 con 0x7ff5341095c0 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff533200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff53410f3c0 con 0x7ff534075900 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff533200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff53410f6c0 con 0x7ff534074320 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff531e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff534074320 0x7ff534199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff531e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff534074320 0x7ff534199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:59414/0 (socket says 172.21.15.97:59414) 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff531e00640 1 -- 172.21.15.97:0/3676896228 learned_addr learned my addr 172.21.15.97:0/3676896228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff52be00640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff534075900 0x7ff53419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.576+0000 7ff532800640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341095c0 0x7ff5341a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff531e00640 1 -- 172.21.15.97:0/3676896228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff534075900 msgr2=0x7ff53419a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff531e00640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff534075900 0x7ff53419a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff531e00640 1 -- 172.21.15.97:0/3676896228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341095c0 msgr2=0x7ff5341a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff531e00640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341095c0 0x7ff5341a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff531e00640 1 -- 172.21.15.97:0/3676896228 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5341a17d0 con 0x7ff534074320 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff52be00640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff534075900 0x7ff53419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff531e00640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff534074320 0x7ff534199bb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff52400e9f0 tx=0x7ff52400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff52aa00640 1 -- 172.21.15.97:0/3676896228 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52400cd80 con 0x7ff534074320 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff52aa00640 1 -- 172.21.15.97:0/3676896228 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff524010910 con 0x7ff534074320 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff533200640 1 -- 172.21.15.97:0/3676896228 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5341a1ac0 con 0x7ff534074320 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.577+0000 7ff52aa00640 1 -- 172.21.15.97:0/3676896228 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524018990 con 0x7ff534074320 2024-03-26T18:56:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.578+0000 7ff533200640 1 -- 172.21.15.97:0/3676896228 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5341a2060 con 0x7ff534074320 2024-03-26T18:56:33.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.579+0000 7ff533200640 1 -- 172.21.15.97:0/3676896228 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5341082f0 con 0x7ff534074320 2024-03-26T18:56:33.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.581+0000 7ff52aa00640 1 -- 172.21.15.97:0/3676896228 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7ff524010ac0 con 0x7ff534074320 2024-03-26T18:56:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.583+0000 7ff52aa00640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5180a3030 0x7ff5180a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.583+0000 7ff52aa00640 1 -- 172.21.15.97:0/3676896228 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff5240cf640 con 0x7ff534074320 2024-03-26T18:56:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.583+0000 7ff52be00640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5180a3030 0x7ff5180a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:33.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.583+0000 7ff52be00640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5180a3030 0x7ff5180a54f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff53419b0c0 tx=0x7ff51c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:33.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.587+0000 7ff52aa00640 1 -- 172.21.15.97:0/3676896228 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff52409be70 con 0x7ff534074320 2024-03-26T18:56:33.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.782+0000 7ff533200640 1 -- 172.21.15.97:0/3676896228 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff534061d00 con 0x7ff5180a3030 2024-03-26T18:56:33.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.792+0000 7ff52aa00640 1 -- 172.21.15.97:0/3676896228 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258909 (secure 0 0 0) 0x7ff534061d00 con 0x7ff5180a3030 2024-03-26T18:56:33.787 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:33.791 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-26T18:56:33.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 -- 172.21.15.97:0/3676896228 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5180a3030 msgr2=0x7ff5180a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:33.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5180a3030 0x7ff5180a54f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff53419b0c0 tx=0x7ff51c005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:33.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 -- 172.21.15.97:0/3676896228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff534074320 msgr2=0x7ff534199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:33.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff534074320 0x7ff534199bb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff52400e9f0 tx=0x7ff52400eeb0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 -- 172.21.15.97:0/3676896228 shutdown_connections 2024-03-26T18:56:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5180a3030 0x7ff5180a54f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341095c0 0x7ff5341a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff534075900 0x7ff53419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 --2- 172.21.15.97:0/3676896228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff534074320 0x7ff534199bb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 -- 172.21.15.97:0/3676896228 >> 172.21.15.97:0/3676896228 conn(0x7ff5340fcc50 msgr2=0x7ff5341054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 -- 172.21.15.97:0/3676896228 shutdown_connections 2024-03-26T18:56:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:33.797+0000 7ff533200640 1 -- 172.21.15.97:0/3676896228 wait complete. 2024-03-26T18:56:33.814 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-03-26T18:56:32.908002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595590,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":133,"num_read_kb":115,"num_write":304,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360292,"kb_used_data":11128,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581532,"statfs":{"total":1151940427776,"available":1151571488768,"internally_reserved":0,"allocated":11395072,"data_stored":7095696,"data_compressed":108958,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5222,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":123,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.781961"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884766+0000","last_change":"2024-03-26T18:56:24.177714+0000","last_active":"2024-03-26T18:56:25.884766+0000","last_peered":"2024-03-26T18:56:25.884766+0000","last_clean":"2024-03-26T18:56:25.884766+0000","last_became_active":"2024-03-26T18:56:24.177420+0000","last_became_peered":"2024-03-26T18:56:24.177420+0000","last_unstale":"2024-03-26T18:56:25.884766+0000","last_undegraded":"2024-03-26T18:56:25.884766+0000","last_fullsized":"2024-03-26T18:56:25.884766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:21:23.738513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216557+0000","last_change":"2024-03-26T18:56:24.160191+0000","last_active":"2024-03-26T18:56:25.216557+0000","last_peered":"2024-03-26T18:56:25.216557+0000","last_clean":"2024-03-26T18:56:25.216557+0000","last_became_active":"2024-03-26T18:56:24.159718+0000","last_became_peered":"2024-03-26T18:56:24.159718+0000","last_unstale":"2024-03-26T18:56:25.216557+0000","last_undegraded":"2024-03-26T18:56:25.216557+0000","last_fullsized":"2024-03-26T18:56:25.216557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:12:04.287352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216230+0000","last_change":"2024-03-26T18:56:24.169795+0000","last_active":"2024-03-26T18:56:25.216230+0000","last_peered":"2024-03-26T18:56:25.216230+0000","last_clean":"2024-03-26T18:56:25.216230+0000","last_became_active":"2024-03-26T18:56:24.166044+0000","last_became_peered":"2024-03-26T18:56:24.166044+0000","last_unstale":"2024-03-26T18:56:25.216230+0000","last_undegraded":"2024-03-26T18:56:25.216230+0000","last_fullsized":"2024-03-26T18:56:25.216230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:21:58.951019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.288457+0000","last_change":"2024-03-26T18:56:24.152606+0000","last_active":"2024-03-26T18:56:25.288457+0000","last_peered":"2024-03-26T18:56:25.288457+0000","last_clean":"2024-03-26T18:56:25.288457+0000","last_became_active":"2024-03-26T18:56:24.152172+0000","last_became_peered":"2024-03-26T18:56:24.152172+0000","last_unstale":"2024-03-26T18:56:25.288457+0000","last_undegraded":"2024-03-26T18:56:25.288457+0000","last_fullsized":"2024-03-26T18:56:25.288457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:21:25.035154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549169+0000","last_change":"2024-03-26T18:56:24.175902+0000","last_active":"2024-03-26T18:56:25.549169+0000","last_peered":"2024-03-26T18:56:25.549169+0000","last_clean":"2024-03-26T18:56:25.549169+0000","last_became_active":"2024-03-26T18:56:24.175568+0000","last_became_peered":"2024-03-26T18:56:24.175568+0000","last_unstale":"2024-03-26T18:56:25.549169+0000","last_undegraded":"2024-03-26T18:56:25.549169+0000","last_fullsized":"2024-03-26T18:56:25.549169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:33:14.054102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732694+0000","last_change":"2024-03-26T18:56:24.145629+0000","last_active":"2024-03-26T18:56:25.732694+0000","last_peered":"2024-03-26T18:56:25.732694+0000","last_clean":"2024-03-26T18:56:25.732694+0000","last_became_active":"2024-03-26T18:56:24.145286+0000","last_became_peered":"2024-03-26T18:56:24.145286+0000","last_unstale":"2024-03-26T18:56:25.732694+0000","last_undegraded":"2024-03-26T18:56:25.732694+0000","last_fullsized":"2024-03-26T18:56:25.732694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:37:24.989479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732081+0000","last_change":"2024-03-26T18:56:24.150869+0000","last_active":"2024-03-26T18:56:25.732081+0000","last_peered":"2024-03-26T18:56:25.732081+0000","last_clean":"2024-03-26T18:56:25.732081+0000","last_became_active":"2024-03-26T18:56:24.150474+0000","last_became_peered":"2024-03-26T18:56:24.150474+0000","last_unstale":"2024-03-26T18:56:25.732081+0000","last_undegraded":"2024-03-26T18:56:25.732081+0000","last_fullsized":"2024-03-26T18:56:25.732081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:55.113389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.550559+0000","last_change":"2024-03-26T18:56:24.170218+0000","last_active":"2024-03-26T18:56:25.550559+0000","last_peered":"2024-03-26T18:56:25.550559+0000","last_clean":"2024-03-26T18:56:25.550559+0000","last_became_active":"2024-03-26T18:56:24.169343+0000","last_became_peered":"2024-03-26T18:56:24.169343+0000","last_unstale":"2024-03-26T18:56:25.550559+0000","last_undegraded":"2024-03-26T18:56:25.550559+0000","last_fullsized":"2024-03-26T18:56:25.550559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:28:41.609616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127828+0000","last_change":"2024-03-26T18:56:24.146892+0000","last_active":"2024-03-26T18:56:25.127828+0000","last_peered":"2024-03-26T18:56:25.127828+0000","last_clean":"2024-03-26T18:56:25.127828+0000","last_became_active":"2024-03-26T18:56:24.146564+0000","last_became_peered":"2024-03-26T18:56:24.146564+0000","last_unstale":"2024-03-26T18:56:25.127828+0000","last_undegraded":"2024-03-26T18:56:25.127828+0000","last_fullsized":"2024-03-26T18:56:25.127828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:12:42.104605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732962+0000","last_change":"2024-03-26T18:56:24.148110+0000","last_active":"2024-03-26T18:56:25.732962+0000","last_peered":"2024-03-26T18:56:25.732962+0000","last_clean":"2024-03-26T18:56:25.732962+0000","last_became_active":"2024-03-26T18:56:24.147763+0000","last_became_peered":"2024-03-26T18:56:24.147763+0000","last_unstale":"2024-03-26T18:56:25.732962+0000","last_undegraded":"2024-03-26T18:56:25.732962+0000","last_fullsized":"2024-03-26T18:56:25.732962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:23.199428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732357+0000","last_change":"2024-03-26T18:56:24.149064+0000","last_active":"2024-03-26T18:56:25.732357+0000","last_peered":"2024-03-26T18:56:25.732357+0000","last_clean":"2024-03-26T18:56:25.732357+0000","last_became_active":"2024-03-26T18:56:24.148400+0000","last_became_peered":"2024-03-26T18:56:24.148400+0000","last_unstale":"2024-03-26T18:56:25.732357+0000","last_undegraded":"2024-03-26T18:56:25.732357+0000","last_fullsized":"2024-03-26T18:56:25.732357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:08:14.141931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.288434+0000","last_change":"2024-03-26T18:56:24.154166+0000","last_active":"2024-03-26T18:56:25.288434+0000","last_peered":"2024-03-26T18:56:25.288434+0000","last_clean":"2024-03-26T18:56:25.288434+0000","last_became_active":"2024-03-26T18:56:24.153630+0000","last_became_peered":"2024-03-26T18:56:24.153630+0000","last_unstale":"2024-03-26T18:56:25.288434+0000","last_undegraded":"2024-03-26T18:56:25.288434+0000","last_fullsized":"2024-03-26T18:56:25.288434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:01:10.067127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.734099+0000","last_change":"2024-03-26T18:56:24.165819+0000","last_active":"2024-03-26T18:56:25.734099+0000","last_peered":"2024-03-26T18:56:25.734099+0000","last_clean":"2024-03-26T18:56:25.734099+0000","last_became_active":"2024-03-26T18:56:24.164401+0000","last_became_peered":"2024-03-26T18:56:24.164401+0000","last_unstale":"2024-03-26T18:56:25.734099+0000","last_undegraded":"2024-03-26T18:56:25.734099+0000","last_fullsized":"2024-03-26T18:56:25.734099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:08:32.301805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216413+0000","last_change":"2024-03-26T18:56:24.174229+0000","last_active":"2024-03-26T18:56:25.216413+0000","last_peered":"2024-03-26T18:56:25.216413+0000","last_clean":"2024-03-26T18:56:25.216413+0000","last_became_active":"2024-03-26T18:56:24.172924+0000","last_became_peered":"2024-03-26T18:56:24.172924+0000","last_unstale":"2024-03-26T18:56:25.216413+0000","last_undegraded":"2024-03-26T18:56:25.216413+0000","last_fullsized":"2024-03-26T18:56:25.216413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:26:00.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.215795+0000","last_change":"2024-03-26T18:56:24.165029+0000","last_active":"2024-03-26T18:56:25.215795+0000","last_peered":"2024-03-26T18:56:25.215795+0000","last_clean":"2024-03-26T18:56:25.215795+0000","last_became_active":"2024-03-26T18:56:24.160238+0000","last_became_peered":"2024-03-26T18:56:24.160238+0000","last_unstale":"2024-03-26T18:56:25.215795+0000","last_undegraded":"2024-03-26T18:56:25.215795+0000","last_fullsized":"2024-03-26T18:56:25.215795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:56:14.509564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127429+0000","last_change":"2024-03-26T18:56:24.167668+0000","last_active":"2024-03-26T18:56:25.127429+0000","last_peered":"2024-03-26T18:56:25.127429+0000","last_clean":"2024-03-26T18:56:25.127429+0000","last_became_active":"2024-03-26T18:56:24.167267+0000","last_became_peered":"2024-03-26T18:56:24.167267+0000","last_unstale":"2024-03-26T18:56:25.127429+0000","last_undegraded":"2024-03-26T18:56:25.127429+0000","last_fullsized":"2024-03-26T18:56:25.127429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:13:00.503977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884777+0000","last_change":"2024-03-26T18:56:24.177623+0000","last_active":"2024-03-26T18:56:25.884777+0000","last_peered":"2024-03-26T18:56:25.884777+0000","last_clean":"2024-03-26T18:56:25.884777+0000","last_became_active":"2024-03-26T18:56:24.177372+0000","last_became_peered":"2024-03-26T18:56:24.177372+0000","last_unstale":"2024-03-26T18:56:25.884777+0000","last_undegraded":"2024-03-26T18:56:25.884777+0000","last_fullsized":"2024-03-26T18:56:25.884777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:02:20.650520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.131162+0000","last_change":"2024-03-26T18:56:24.152407+0000","last_active":"2024-03-26T18:56:25.131162+0000","last_peered":"2024-03-26T18:56:25.131162+0000","last_clean":"2024-03-26T18:56:25.131162+0000","last_became_active":"2024-03-26T18:56:24.151935+0000","last_became_peered":"2024-03-26T18:56:24.151935+0000","last_unstale":"2024-03-26T18:56:25.131162+0000","last_undegraded":"2024-03-26T18:56:25.131162+0000","last_fullsized":"2024-03-26T18:56:25.131162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:52:13.211124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.130169+0000","last_change":"2024-03-26T18:56:24.162582+0000","last_active":"2024-03-26T18:56:25.130169+0000","last_peered":"2024-03-26T18:56:25.130169+0000","last_clean":"2024-03-26T18:56:25.130169+0000","last_became_active":"2024-03-26T18:56:24.162228+0000","last_became_peered":"2024-03-26T18:56:24.162228+0000","last_unstale":"2024-03-26T18:56:25.130169+0000","last_undegraded":"2024-03-26T18:56:25.130169+0000","last_fullsized":"2024-03-26T18:56:25.130169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:16.841466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.864317+0000","last_change":"2024-03-26T18:56:23.149475+0000","last_active":"2024-03-26T18:56:28.864317+0000","last_peered":"2024-03-26T18:56:28.864317+0000","last_clean":"2024-03-26T18:56:28.864317+0000","last_became_active":"2024-03-26T18:56:23.148813+0000","last_became_peered":"2024-03-26T18:56:23.148813+0000","last_unstale":"2024-03-26T18:56:28.864317+0000","last_undegraded":"2024-03-26T18:56:28.864317+0000","last_fullsized":"2024-03-26T18:56:28.864317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:38.315414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551660+0000","last_change":"2024-03-26T18:56:24.173904+0000","last_active":"2024-03-26T18:56:25.551660+0000","last_peered":"2024-03-26T18:56:25.551660+0000","last_clean":"2024-03-26T18:56:25.551660+0000","last_became_active":"2024-03-26T18:56:24.173523+0000","last_became_peered":"2024-03-26T18:56:24.173523+0000","last_unstale":"2024-03-26T18:56:25.551660+0000","last_undegraded":"2024-03-26T18:56:25.551660+0000","last_fullsized":"2024-03-26T18:56:25.551660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:40:35.651821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551461+0000","last_change":"2024-03-26T18:56:23.173583+0000","last_active":"2024-03-26T18:56:25.551461+0000","last_peered":"2024-03-26T18:56:25.551461+0000","last_clean":"2024-03-26T18:56:25.551461+0000","last_became_active":"2024-03-26T18:56:23.172071+0000","last_became_peered":"2024-03-26T18:56:23.172071+0000","last_unstale":"2024-03-26T18:56:25.551461+0000","last_undegraded":"2024-03-26T18:56:25.551461+0000","last_fullsized":"2024-03-26T18:56:25.551461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:29:39.194411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216502+0000","last_change":"2024-03-26T18:56:24.175535+0000","last_active":"2024-03-26T18:56:25.216502+0000","last_peered":"2024-03-26T18:56:25.216502+0000","last_clean":"2024-03-26T18:56:25.216502+0000","last_became_active":"2024-03-26T18:56:24.173748+0000","last_became_peered":"2024-03-26T18:56:24.173748+0000","last_unstale":"2024-03-26T18:56:25.216502+0000","last_undegraded":"2024-03-26T18:56:25.216502+0000","last_fullsized":"2024-03-26T18:56:25.216502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:38.960606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.857139+0000","last_change":"2024-03-26T18:56:23.150303+0000","last_active":"2024-03-26T18:56:28.857139+0000","last_peered":"2024-03-26T18:56:28.857139+0000","last_clean":"2024-03-26T18:56:28.857139+0000","last_became_active":"2024-03-26T18:56:23.149840+0000","last_became_peered":"2024-03-26T18:56:23.149840+0000","last_unstale":"2024-03-26T18:56:28.857139+0000","last_undegraded":"2024-03-26T18:56:28.857139+0000","last_fullsized":"2024-03-26T18:56:28.857139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:01:09.356776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127553+0000","last_change":"2024-03-26T18:56:24.161351+0000","last_active":"2024-03-26T18:56:25.127553+0000","last_peered":"2024-03-26T18:56:25.127553+0000","last_clean":"2024-03-26T18:56:25.127553+0000","last_became_active":"2024-03-26T18:56:24.161013+0000","last_became_peered":"2024-03-26T18:56:24.161013+0000","last_unstale":"2024-03-26T18:56:25.127553+0000","last_undegraded":"2024-03-26T18:56:25.127553+0000","last_fullsized":"2024-03-26T18:56:25.127553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:24:25.620772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549381+0000","last_change":"2024-03-26T18:56:23.148866+0000","last_active":"2024-03-26T18:56:25.549381+0000","last_peered":"2024-03-26T18:56:25.549381+0000","last_clean":"2024-03-26T18:56:25.549381+0000","last_became_active":"2024-03-26T18:56:23.148434+0000","last_became_peered":"2024-03-26T18:56:23.148434+0000","last_unstale":"2024-03-26T18:56:25.549381+0000","last_undegraded":"2024-03-26T18:56:25.549381+0000","last_fullsized":"2024-03-26T18:56:25.549381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:35:35.180038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733128+0000","last_change":"2024-03-26T18:56:24.147429+0000","last_active":"2024-03-26T18:56:25.733128+0000","last_peered":"2024-03-26T18:56:25.733128+0000","last_clean":"2024-03-26T18:56:25.733128+0000","last_became_active":"2024-03-26T18:56:24.147100+0000","last_became_peered":"2024-03-26T18:56:24.147100+0000","last_unstale":"2024-03-26T18:56:25.733128+0000","last_undegraded":"2024-03-26T18:56:25.733128+0000","last_fullsized":"2024-03-26T18:56:25.733128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:08.884223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.849738+0000","last_change":"2024-03-26T18:56:23.149301+0000","last_active":"2024-03-26T18:56:27.849738+0000","last_peered":"2024-03-26T18:56:27.849738+0000","last_clean":"2024-03-26T18:56:27.849738+0000","last_became_active":"2024-03-26T18:56:23.148939+0000","last_became_peered":"2024-03-26T18:56:23.148939+0000","last_unstale":"2024-03-26T18:56:27.849738+0000","last_undegraded":"2024-03-26T18:56:27.849738+0000","last_fullsized":"2024-03-26T18:56:27.849738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:50:06.649474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549172+0000","last_change":"2024-03-26T18:56:24.157317+0000","last_active":"2024-03-26T18:56:25.549172+0000","last_peered":"2024-03-26T18:56:25.549172+0000","last_clean":"2024-03-26T18:56:25.549172+0000","last_became_active":"2024-03-26T18:56:24.156971+0000","last_became_peered":"2024-03-26T18:56:24.156971+0000","last_unstale":"2024-03-26T18:56:25.549172+0000","last_undegraded":"2024-03-26T18:56:25.549172+0000","last_fullsized":"2024-03-26T18:56:25.549172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:22:39.360404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:29.828242+0000","last_change":"2024-03-26T18:56:23.162848+0000","last_active":"2024-03-26T18:56:29.828242+0000","last_peered":"2024-03-26T18:56:29.828242+0000","last_clean":"2024-03-26T18:56:29.828242+0000","last_became_active":"2024-03-26T18:56:23.162478+0000","last_became_peered":"2024-03-26T18:56:23.162478+0000","last_unstale":"2024-03-26T18:56:29.828242+0000","last_undegraded":"2024-03-26T18:56:29.828242+0000","last_fullsized":"2024-03-26T18:56:29.828242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:07:51.139538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732908+0000","last_change":"2024-03-26T18:56:24.156451+0000","last_active":"2024-03-26T18:56:25.732908+0000","last_peered":"2024-03-26T18:56:25.732908+0000","last_clean":"2024-03-26T18:56:25.732908+0000","last_became_active":"2024-03-26T18:56:24.156237+0000","last_became_peered":"2024-03-26T18:56:24.156237+0000","last_unstale":"2024-03-26T18:56:25.732908+0000","last_undegraded":"2024-03-26T18:56:25.732908+0000","last_fullsized":"2024-03-26T18:56:25.732908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:52:54.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733315+0000","last_change":"2024-03-26T18:56:23.172926+0000","last_active":"2024-03-26T18:56:25.733315+0000","last_peered":"2024-03-26T18:56:25.733315+0000","last_clean":"2024-03-26T18:56:25.733315+0000","last_became_active":"2024-03-26T18:56:23.170512+0000","last_became_peered":"2024-03-26T18:56:23.170512+0000","last_unstale":"2024-03-26T18:56:25.733315+0000","last_undegraded":"2024-03-26T18:56:25.733315+0000","last_fullsized":"2024-03-26T18:56:25.733315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:16:13.747853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551275+0000","last_change":"2024-03-26T18:56:24.173733+0000","last_active":"2024-03-26T18:56:25.551275+0000","last_peered":"2024-03-26T18:56:25.551275+0000","last_clean":"2024-03-26T18:56:25.551275+0000","last_became_active":"2024-03-26T18:56:24.171895+0000","last_became_peered":"2024-03-26T18:56:24.171895+0000","last_unstale":"2024-03-26T18:56:25.551275+0000","last_undegraded":"2024-03-26T18:56:25.551275+0000","last_fullsized":"2024-03-26T18:56:25.551275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:07:59.273853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732445+0000","last_change":"2024-03-26T18:56:23.150140+0000","last_active":"2024-03-26T18:56:25.732445+0000","last_peered":"2024-03-26T18:56:25.732445+0000","last_clean":"2024-03-26T18:56:25.732445+0000","last_became_active":"2024-03-26T18:56:23.149634+0000","last_became_peered":"2024-03-26T18:56:23.149634+0000","last_unstale":"2024-03-26T18:56:25.732445+0000","last_undegraded":"2024-03-26T18:56:25.732445+0000","last_fullsized":"2024-03-26T18:56:25.732445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:52:48.025868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.130666+0000","last_change":"2024-03-26T18:56:24.157855+0000","last_active":"2024-03-26T18:56:25.130666+0000","last_peered":"2024-03-26T18:56:25.130666+0000","last_clean":"2024-03-26T18:56:25.130666+0000","last_became_active":"2024-03-26T18:56:24.157220+0000","last_became_peered":"2024-03-26T18:56:24.157220+0000","last_unstale":"2024-03-26T18:56:25.130666+0000","last_undegraded":"2024-03-26T18:56:25.130666+0000","last_fullsized":"2024-03-26T18:56:25.130666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:40:00.815178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:29.815058+0000","last_change":"2024-03-26T18:56:23.148341+0000","last_active":"2024-03-26T18:56:29.815058+0000","last_peered":"2024-03-26T18:56:29.815058+0000","last_clean":"2024-03-26T18:56:29.815058+0000","last_became_active":"2024-03-26T18:56:23.148021+0000","last_became_peered":"2024-03-26T18:56:23.148021+0000","last_unstale":"2024-03-26T18:56:29.815058+0000","last_undegraded":"2024-03-26T18:56:29.815058+0000","last_fullsized":"2024-03-26T18:56:29.815058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:42.577047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127877+0000","last_change":"2024-03-26T18:56:24.160017+0000","last_active":"2024-03-26T18:56:25.127877+0000","last_peered":"2024-03-26T18:56:25.127877+0000","last_clean":"2024-03-26T18:56:25.127877+0000","last_became_active":"2024-03-26T18:56:24.159686+0000","last_became_peered":"2024-03-26T18:56:24.159686+0000","last_unstale":"2024-03-26T18:56:25.127877+0000","last_undegraded":"2024-03-26T18:56:25.127877+0000","last_fullsized":"2024-03-26T18:56:25.127877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:33:32.693850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.848126+0000","last_change":"2024-03-26T18:56:23.146487+0000","last_active":"2024-03-26T18:56:27.848126+0000","last_peered":"2024-03-26T18:56:27.848126+0000","last_clean":"2024-03-26T18:56:27.848126+0000","last_became_active":"2024-03-26T18:56:23.146155+0000","last_became_peered":"2024-03-26T18:56:23.146155+0000","last_unstale":"2024-03-26T18:56:27.848126+0000","last_undegraded":"2024-03-26T18:56:27.848126+0000","last_fullsized":"2024-03-26T18:56:27.848126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:09:51.836211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216332+0000","last_change":"2024-03-26T18:56:24.172652+0000","last_active":"2024-03-26T18:56:25.216332+0000","last_peered":"2024-03-26T18:56:25.216332+0000","last_clean":"2024-03-26T18:56:25.216332+0000","last_became_active":"2024-03-26T18:56:24.170778+0000","last_became_peered":"2024-03-26T18:56:24.170778+0000","last_unstale":"2024-03-26T18:56:25.216332+0000","last_undegraded":"2024-03-26T18:56:25.216332+0000","last_fullsized":"2024-03-26T18:56:25.216332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:47:16.946692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733112+0000","last_change":"2024-03-26T18:56:23.151669+0000","last_active":"2024-03-26T18:56:25.733112+0000","last_peered":"2024-03-26T18:56:25.733112+0000","last_clean":"2024-03-26T18:56:25.733112+0000","last_became_active":"2024-03-26T18:56:23.151276+0000","last_became_peered":"2024-03-26T18:56:23.151276+0000","last_unstale":"2024-03-26T18:56:25.733112+0000","last_undegraded":"2024-03-26T18:56:25.733112+0000","last_fullsized":"2024-03-26T18:56:25.733112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:28:37.827930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127702+0000","last_change":"2024-03-26T18:56:24.151007+0000","last_active":"2024-03-26T18:56:25.127702+0000","last_peered":"2024-03-26T18:56:25.127702+0000","last_clean":"2024-03-26T18:56:25.127702+0000","last_became_active":"2024-03-26T18:56:24.150668+0000","last_became_peered":"2024-03-26T18:56:24.150668+0000","last_unstale":"2024-03-26T18:56:25.127702+0000","last_undegraded":"2024-03-26T18:56:25.127702+0000","last_fullsized":"2024-03-26T18:56:25.127702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:07:04.778039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:29.813444+0000","last_change":"2024-03-26T18:56:23.144522+0000","last_active":"2024-03-26T18:56:29.813444+0000","last_peered":"2024-03-26T18:56:29.813444+0000","last_clean":"2024-03-26T18:56:29.813444+0000","last_became_active":"2024-03-26T18:56:23.144176+0000","last_became_peered":"2024-03-26T18:56:23.144176+0000","last_unstale":"2024-03-26T18:56:29.813444+0000","last_undegraded":"2024-03-26T18:56:29.813444+0000","last_fullsized":"2024-03-26T18:56:29.813444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:42:20.319478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549452+0000","last_change":"2024-03-26T18:56:24.148307+0000","last_active":"2024-03-26T18:56:25.549452+0000","last_peered":"2024-03-26T18:56:25.549452+0000","last_clean":"2024-03-26T18:56:25.549452+0000","last_became_active":"2024-03-26T18:56:24.147973+0000","last_became_peered":"2024-03-26T18:56:24.147973+0000","last_unstale":"2024-03-26T18:56:25.549452+0000","last_undegraded":"2024-03-26T18:56:25.549452+0000","last_fullsized":"2024-03-26T18:56:25.549452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:08:45.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:29.811380+0000","last_change":"2024-03-26T18:56:23.163757+0000","last_active":"2024-03-26T18:56:29.811380+0000","last_peered":"2024-03-26T18:56:29.811380+0000","last_clean":"2024-03-26T18:56:29.811380+0000","last_became_active":"2024-03-26T18:56:23.163418+0000","last_became_peered":"2024-03-26T18:56:23.163418+0000","last_unstale":"2024-03-26T18:56:29.811380+0000","last_undegraded":"2024-03-26T18:56:29.811380+0000","last_fullsized":"2024-03-26T18:56:29.811380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:04:09.178853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733422+0000","last_change":"2024-03-26T18:56:24.167274+0000","last_active":"2024-03-26T18:56:25.733422+0000","last_peered":"2024-03-26T18:56:25.733422+0000","last_clean":"2024-03-26T18:56:25.733422+0000","last_became_active":"2024-03-26T18:56:24.166353+0000","last_became_peered":"2024-03-26T18:56:24.166353+0000","last_unstale":"2024-03-26T18:56:25.733422+0000","last_undegraded":"2024-03-26T18:56:25.733422+0000","last_fullsized":"2024-03-26T18:56:25.733422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:14:44.794112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.850379+0000","last_change":"2024-03-26T18:56:23.162115+0000","last_active":"2024-03-26T18:56:27.850379+0000","last_peered":"2024-03-26T18:56:27.850379+0000","last_clean":"2024-03-26T18:56:27.850379+0000","last_became_active":"2024-03-26T18:56:23.161746+0000","last_became_peered":"2024-03-26T18:56:23.161746+0000","last_unstale":"2024-03-26T18:56:27.850379+0000","last_undegraded":"2024-03-26T18:56:27.850379+0000","last_fullsized":"2024-03-26T18:56:27.850379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:08:26.583281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732367+0000","last_change":"2024-03-26T18:56:24.166407+0000","last_active":"2024-03-26T18:56:25.732367+0000","last_peered":"2024-03-26T18:56:25.732367+0000","last_clean":"2024-03-26T18:56:25.732367+0000","last_became_active":"2024-03-26T18:56:24.166077+0000","last_became_peered":"2024-03-26T18:56:24.166077+0000","last_unstale":"2024-03-26T18:56:25.732367+0000","last_undegraded":"2024-03-26T18:56:25.732367+0000","last_fullsized":"2024-03-26T18:56:25.732367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:36:35.160247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.288189+0000","last_change":"2024-03-26T18:56:23.146908+0000","last_active":"2024-03-26T18:56:25.288189+0000","last_peered":"2024-03-26T18:56:25.288189+0000","last_clean":"2024-03-26T18:56:25.288189+0000","last_became_active":"2024-03-26T18:56:23.146576+0000","last_became_peered":"2024-03-26T18:56:23.146576+0000","last_unstale":"2024-03-26T18:56:25.288189+0000","last_undegraded":"2024-03-26T18:56:25.288189+0000","last_fullsized":"2024-03-26T18:56:25.288189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:00:55.708101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732015+0000","last_change":"2024-03-26T18:56:24.147460+0000","last_active":"2024-03-26T18:56:25.732015+0000","last_peered":"2024-03-26T18:56:25.732015+0000","last_clean":"2024-03-26T18:56:25.732015+0000","last_became_active":"2024-03-26T18:56:24.147065+0000","last_became_peered":"2024-03-26T18:56:24.147065+0000","last_unstale":"2024-03-26T18:56:25.732015+0000","last_undegraded":"2024-03-26T18:56:25.732015+0000","last_fullsized":"2024-03-26T18:56:25.732015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:02:14.962735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.865573+0000","last_change":"2024-03-26T18:56:23.152295+0000","last_active":"2024-03-26T18:56:28.865573+0000","last_peered":"2024-03-26T18:56:28.865573+0000","last_clean":"2024-03-26T18:56:28.865573+0000","last_became_active":"2024-03-26T18:56:23.151862+0000","last_became_peered":"2024-03-26T18:56:23.151862+0000","last_unstale":"2024-03-26T18:56:28.865573+0000","last_undegraded":"2024-03-26T18:56:28.865573+0000","last_fullsized":"2024-03-26T18:56:28.865573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:29:41.168993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732810+0000","last_change":"2024-03-26T18:56:24.146993+0000","last_active":"2024-03-26T18:56:25.732810+0000","last_peered":"2024-03-26T18:56:25.732810+0000","last_clean":"2024-03-26T18:56:25.732810+0000","last_became_active":"2024-03-26T18:56:24.146692+0000","last_became_peered":"2024-03-26T18:56:24.146692+0000","last_unstale":"2024-03-26T18:56:25.732810+0000","last_undegraded":"2024-03-26T18:56:25.732810+0000","last_fullsized":"2024-03-26T18:56:25.732810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:08:33.731716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549397+0000","last_change":"2024-03-26T18:56:23.148459+0000","last_active":"2024-03-26T18:56:25.549397+0000","last_peered":"2024-03-26T18:56:25.549397+0000","last_clean":"2024-03-26T18:56:25.549397+0000","last_became_active":"2024-03-26T18:56:23.148097+0000","last_became_peered":"2024-03-26T18:56:23.148097+0000","last_unstale":"2024-03-26T18:56:25.549397+0000","last_undegraded":"2024-03-26T18:56:25.549397+0000","last_fullsized":"2024-03-26T18:56:25.549397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:08:51.990705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884618+0000","last_change":"2024-03-26T18:56:24.146399+0000","last_active":"2024-03-26T18:56:25.884618+0000","last_peered":"2024-03-26T18:56:25.884618+0000","last_clean":"2024-03-26T18:56:25.884618+0000","last_became_active":"2024-03-26T18:56:24.146014+0000","last_became_peered":"2024-03-26T18:56:24.146014+0000","last_unstale":"2024-03-26T18:56:25.884618+0000","last_undegraded":"2024-03-26T18:56:25.884618+0000","last_fullsized":"2024-03-26T18:56:25.884618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:24:26.809602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.130865+0000","last_change":"2024-03-26T18:56:23.148037+0000","last_active":"2024-03-26T18:56:25.130865+0000","last_peered":"2024-03-26T18:56:25.130865+0000","last_clean":"2024-03-26T18:56:25.130865+0000","last_became_active":"2024-03-26T18:56:23.147598+0000","last_became_peered":"2024-03-26T18:56:23.147598+0000","last_unstale":"2024-03-26T18:56:25.130865+0000","last_undegraded":"2024-03-26T18:56:25.130865+0000","last_fullsized":"2024-03-26T18:56:25.130865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:21:55.105299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732745+0000","last_change":"2024-03-26T18:56:24.167488+0000","last_active":"2024-03-26T18:56:25.732745+0000","last_peered":"2024-03-26T18:56:25.732745+0000","last_clean":"2024-03-26T18:56:25.732745+0000","last_became_active":"2024-03-26T18:56:24.167154+0000","last_became_peered":"2024-03-26T18:56:24.167154+0000","last_unstale":"2024-03-26T18:56:25.732745+0000","last_undegraded":"2024-03-26T18:56:25.732745+0000","last_fullsized":"2024-03-26T18:56:25.732745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:00:35.606573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732495+0000","last_change":"2024-03-26T18:56:23.160694+0000","last_active":"2024-03-26T18:56:25.732495+0000","last_peered":"2024-03-26T18:56:25.732495+0000","last_clean":"2024-03-26T18:56:25.732495+0000","last_became_active":"2024-03-26T18:56:23.160416+0000","last_became_peered":"2024-03-26T18:56:23.160416+0000","last_unstale":"2024-03-26T18:56:25.732495+0000","last_undegraded":"2024-03-26T18:56:25.732495+0000","last_fullsized":"2024-03-26T18:56:25.732495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:31:15.616473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732714+0000","last_change":"2024-03-26T18:56:24.170503+0000","last_active":"2024-03-26T18:56:25.732714+0000","last_peered":"2024-03-26T18:56:25.732714+0000","last_clean":"2024-03-26T18:56:25.732714+0000","last_became_active":"2024-03-26T18:56:24.170113+0000","last_became_peered":"2024-03-26T18:56:24.170113+0000","last_unstale":"2024-03-26T18:56:25.732714+0000","last_undegraded":"2024-03-26T18:56:25.732714+0000","last_fullsized":"2024-03-26T18:56:25.732714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:44.076756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732623+0000","last_change":"2024-03-26T18:56:24.146856+0000","last_active":"2024-03-26T18:56:25.732623+0000","last_peered":"2024-03-26T18:56:25.732623+0000","last_clean":"2024-03-26T18:56:25.732623+0000","last_became_active":"2024-03-26T18:56:24.146545+0000","last_became_peered":"2024-03-26T18:56:24.146545+0000","last_unstale":"2024-03-26T18:56:25.732623+0000","last_undegraded":"2024-03-26T18:56:25.732623+0000","last_fullsized":"2024-03-26T18:56:25.732623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:58:26.876614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733292+0000","last_change":"2024-03-26T18:56:23.173650+0000","last_active":"2024-03-26T18:56:25.733292+0000","last_peered":"2024-03-26T18:56:25.733292+0000","last_clean":"2024-03-26T18:56:25.733292+0000","last_became_active":"2024-03-26T18:56:23.173343+0000","last_became_peered":"2024-03-26T18:56:23.173343+0000","last_unstale":"2024-03-26T18:56:25.733292+0000","last_undegraded":"2024-03-26T18:56:25.733292+0000","last_fullsized":"2024-03-26T18:56:25.733292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:42:51.138892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551409+0000","last_change":"2024-03-26T18:56:24.175697+0000","last_active":"2024-03-26T18:56:25.551409+0000","last_peered":"2024-03-26T18:56:25.551409+0000","last_clean":"2024-03-26T18:56:25.551409+0000","last_became_active":"2024-03-26T18:56:24.175159+0000","last_became_peered":"2024-03-26T18:56:24.175159+0000","last_unstale":"2024-03-26T18:56:25.551409+0000","last_undegraded":"2024-03-26T18:56:25.551409+0000","last_fullsized":"2024-03-26T18:56:25.551409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:02:35.581432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.846613+0000","last_change":"2024-03-26T18:56:23.154766+0000","last_active":"2024-03-26T18:56:27.846613+0000","last_peered":"2024-03-26T18:56:27.846613+0000","last_clean":"2024-03-26T18:56:27.846613+0000","last_became_active":"2024-03-26T18:56:23.154475+0000","last_became_peered":"2024-03-26T18:56:23.154475+0000","last_unstale":"2024-03-26T18:56:27.846613+0000","last_undegraded":"2024-03-26T18:56:27.846613+0000","last_fullsized":"2024-03-26T18:56:27.846613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:29:38.484201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732001+0000","last_change":"2024-03-26T18:55:03.797254+0000","last_active":"2024-03-26T18:56:25.732001+0000","last_peered":"2024-03-26T18:56:25.732001+0000","last_clean":"2024-03-26T18:56:25.732001+0000","last_became_active":"2024-03-26T18:55:03.796795+0000","last_became_peered":"2024-03-26T18:55:03.796795+0000","last_unstale":"2024-03-26T18:56:25.732001+0000","last_undegraded":"2024-03-26T18:56:25.732001+0000","last_fullsized":"2024-03-26T18:56:25.732001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:51:11.590736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:51:11.590736+0000","last_clean_scrub_stamp":"2024-03-26T18:51:11.590736+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:48.215695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733151+0000","last_change":"2024-03-26T18:56:24.165552+0000","last_active":"2024-03-26T18:56:25.733151+0000","last_peered":"2024-03-26T18:56:25.733151+0000","last_clean":"2024-03-26T18:56:25.733151+0000","last_became_active":"2024-03-26T18:56:24.164230+0000","last_became_peered":"2024-03-26T18:56:24.164230+0000","last_unstale":"2024-03-26T18:56:25.733151+0000","last_undegraded":"2024-03-26T18:56:25.733151+0000","last_fullsized":"2024-03-26T18:56:25.733151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:36:19.750451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.857740+0000","last_change":"2024-03-26T18:56:23.148791+0000","last_active":"2024-03-26T18:56:28.857740+0000","last_peered":"2024-03-26T18:56:28.857740+0000","last_clean":"2024-03-26T18:56:28.857740+0000","last_became_active":"2024-03-26T18:56:23.148435+0000","last_became_peered":"2024-03-26T18:56:23.148435+0000","last_unstale":"2024-03-26T18:56:28.857740+0000","last_undegraded":"2024-03-26T18:56:28.857740+0000","last_fullsized":"2024-03-26T18:56:28.857740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:13:26.785932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549451+0000","last_change":"2024-03-26T18:56:23.154066+0000","last_active":"2024-03-26T18:56:25.549451+0000","last_peered":"2024-03-26T18:56:25.549451+0000","last_clean":"2024-03-26T18:56:25.549451+0000","last_became_active":"2024-03-26T18:56:23.153813+0000","last_became_peered":"2024-03-26T18:56:23.153813+0000","last_unstale":"2024-03-26T18:56:25.549451+0000","last_undegraded":"2024-03-26T18:56:25.549451+0000","last_fullsized":"2024-03-26T18:56:25.549451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:25:07.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733782+0000","last_change":"2024-03-26T18:56:24.162166+0000","last_active":"2024-03-26T18:56:25.733782+0000","last_peered":"2024-03-26T18:56:25.733782+0000","last_clean":"2024-03-26T18:56:25.733782+0000","last_became_active":"2024-03-26T18:56:24.161623+0000","last_became_peered":"2024-03-26T18:56:24.161623+0000","last_unstale":"2024-03-26T18:56:25.733782+0000","last_undegraded":"2024-03-26T18:56:25.733782+0000","last_fullsized":"2024-03-26T18:56:25.733782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:42.588275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.863926+0000","last_change":"2024-03-26T18:56:23.167462+0000","last_active":"2024-03-26T18:56:28.863926+0000","last_peered":"2024-03-26T18:56:28.863926+0000","last_clean":"2024-03-26T18:56:28.863926+0000","last_became_active":"2024-03-26T18:56:23.167124+0000","last_became_peered":"2024-03-26T18:56:23.167124+0000","last_unstale":"2024-03-26T18:56:28.863926+0000","last_undegraded":"2024-03-26T18:56:28.863926+0000","last_fullsized":"2024-03-26T18:56:28.863926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:38.550421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551224+0000","last_change":"2024-03-26T18:56:24.173164+0000","last_active":"2024-03-26T18:56:25.551224+0000","last_peered":"2024-03-26T18:56:25.551224+0000","last_clean":"2024-03-26T18:56:25.551224+0000","last_became_active":"2024-03-26T18:56:24.170083+0000","last_became_peered":"2024-03-26T18:56:24.170083+0000","last_unstale":"2024-03-26T18:56:25.551224+0000","last_undegraded":"2024-03-26T18:56:25.551224+0000","last_fullsized":"2024-03-26T18:56:25.551224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:26.696625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.846490+0000","last_change":"2024-03-26T18:56:23.162835+0000","last_active":"2024-03-26T18:56:27.846490+0000","last_peered":"2024-03-26T18:56:27.846490+0000","last_clean":"2024-03-26T18:56:27.846490+0000","last_became_active":"2024-03-26T18:56:23.162497+0000","last_became_peered":"2024-03-26T18:56:23.162497+0000","last_unstale":"2024-03-26T18:56:27.846490+0000","last_undegraded":"2024-03-26T18:56:27.846490+0000","last_fullsized":"2024-03-26T18:56:27.846490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:29:47.903712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216354+0000","last_change":"2024-03-26T18:56:24.178140+0000","last_active":"2024-03-26T18:56:25.216354+0000","last_peered":"2024-03-26T18:56:25.216354+0000","last_clean":"2024-03-26T18:56:25.216354+0000","last_became_active":"2024-03-26T18:56:24.177769+0000","last_became_peered":"2024-03-26T18:56:24.177769+0000","last_unstale":"2024-03-26T18:56:25.216354+0000","last_undegraded":"2024-03-26T18:56:25.216354+0000","last_fullsized":"2024-03-26T18:56:25.216354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:39.714335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.849324+0000","last_change":"2024-03-26T18:56:23.143972+0000","last_active":"2024-03-26T18:56:27.849324+0000","last_peered":"2024-03-26T18:56:27.849324+0000","last_clean":"2024-03-26T18:56:27.849324+0000","last_became_active":"2024-03-26T18:56:23.143395+0000","last_became_peered":"2024-03-26T18:56:23.143395+0000","last_unstale":"2024-03-26T18:56:27.849324+0000","last_undegraded":"2024-03-26T18:56:27.849324+0000","last_fullsized":"2024-03-26T18:56:27.849324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:46:32.548546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549003+0000","last_change":"2024-03-26T18:56:24.144839+0000","last_active":"2024-03-26T18:56:25.549003+0000","last_peered":"2024-03-26T18:56:25.549003+0000","last_clean":"2024-03-26T18:56:25.549003+0000","last_became_active":"2024-03-26T18:56:24.144519+0000","last_became_peered":"2024-03-26T18:56:24.144519+0000","last_unstale":"2024-03-26T18:56:25.549003+0000","last_undegraded":"2024-03-26T18:56:25.549003+0000","last_fullsized":"2024-03-26T18:56:25.549003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:54:59.787475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.851891+0000","last_change":"2024-03-26T18:56:23.161380+0000","last_active":"2024-03-26T18:56:27.851891+0000","last_peered":"2024-03-26T18:56:27.851891+0000","last_clean":"2024-03-26T18:56:27.851891+0000","last_became_active":"2024-03-26T18:56:23.160376+0000","last_became_peered":"2024-03-26T18:56:23.160376+0000","last_unstale":"2024-03-26T18:56:27.851891+0000","last_undegraded":"2024-03-26T18:56:27.851891+0000","last_fullsized":"2024-03-26T18:56:27.851891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:49.610002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884589+0000","last_change":"2024-03-26T18:56:24.170284+0000","last_active":"2024-03-26T18:56:25.884589+0000","last_peered":"2024-03-26T18:56:25.884589+0000","last_clean":"2024-03-26T18:56:25.884589+0000","last_became_active":"2024-03-26T18:56:24.169954+0000","last_became_peered":"2024-03-26T18:56:24.169954+0000","last_unstale":"2024-03-26T18:56:25.884589+0000","last_undegraded":"2024-03-26T18:56:25.884589+0000","last_fullsized":"2024-03-26T18:56:25.884589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:05:21.383976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.845229+0000","last_change":"2024-03-26T18:56:23.163386+0000","last_active":"2024-03-26T18:56:28.845229+0000","last_peered":"2024-03-26T18:56:28.845229+0000","last_clean":"2024-03-26T18:56:28.845229+0000","last_became_active":"2024-03-26T18:56:23.163019+0000","last_became_peered":"2024-03-26T18:56:23.163019+0000","last_unstale":"2024-03-26T18:56:28.845229+0000","last_undegraded":"2024-03-26T18:56:28.845229+0000","last_fullsized":"2024-03-26T18:56:28.845229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:03:05.562147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551125+0000","last_change":"2024-03-26T18:56:24.176940+0000","last_active":"2024-03-26T18:56:25.551125+0000","last_peered":"2024-03-26T18:56:25.551125+0000","last_clean":"2024-03-26T18:56:25.551125+0000","last_became_active":"2024-03-26T18:56:24.176396+0000","last_became_peered":"2024-03-26T18:56:24.176396+0000","last_unstale":"2024-03-26T18:56:25.551125+0000","last_undegraded":"2024-03-26T18:56:25.551125+0000","last_fullsized":"2024-03-26T18:56:25.551125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:53:06.994055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.846898+0000","last_change":"2024-03-26T18:56:23.149738+0000","last_active":"2024-03-26T18:56:27.846898+0000","last_peered":"2024-03-26T18:56:27.846898+0000","last_clean":"2024-03-26T18:56:27.846898+0000","last_became_active":"2024-03-26T18:56:23.149399+0000","last_became_peered":"2024-03-26T18:56:23.149399+0000","last_unstale":"2024-03-26T18:56:27.846898+0000","last_undegraded":"2024-03-26T18:56:27.846898+0000","last_fullsized":"2024-03-26T18:56:27.846898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:44:50.666181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732595+0000","last_change":"2024-03-26T18:56:24.161505+0000","last_active":"2024-03-26T18:56:25.732595+0000","last_peered":"2024-03-26T18:56:25.732595+0000","last_clean":"2024-03-26T18:56:25.732595+0000","last_became_active":"2024-03-26T18:56:24.161035+0000","last_became_peered":"2024-03-26T18:56:24.161035+0000","last_unstale":"2024-03-26T18:56:25.732595+0000","last_undegraded":"2024-03-26T18:56:25.732595+0000","last_fullsized":"2024-03-26T18:56:25.732595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:22:30.062398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:29.828826+0000","last_change":"2024-03-26T18:56:23.150900+0000","last_active":"2024-03-26T18:56:29.828826+0000","last_peered":"2024-03-26T18:56:29.828826+0000","last_clean":"2024-03-26T18:56:29.828826+0000","last_became_active":"2024-03-26T18:56:23.150578+0000","last_became_peered":"2024-03-26T18:56:23.150578+0000","last_unstale":"2024-03-26T18:56:29.828826+0000","last_undegraded":"2024-03-26T18:56:29.828826+0000","last_fullsized":"2024-03-26T18:56:29.828826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:18:09.039791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732445+0000","last_change":"2024-03-26T18:56:24.148744+0000","last_active":"2024-03-26T18:56:25.732445+0000","last_peered":"2024-03-26T18:56:25.732445+0000","last_clean":"2024-03-26T18:56:25.732445+0000","last_became_active":"2024-03-26T18:56:24.147857+0000","last_became_peered":"2024-03-26T18:56:24.147857+0000","last_unstale":"2024-03-26T18:56:25.732445+0000","last_undegraded":"2024-03-26T18:56:25.732445+0000","last_fullsized":"2024-03-26T18:56:25.732445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:59:15.890399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732859+0000","last_change":"2024-03-26T18:56:23.165228+0000","last_active":"2024-03-26T18:56:25.732859+0000","last_peered":"2024-03-26T18:56:25.732859+0000","last_clean":"2024-03-26T18:56:25.732859+0000","last_became_active":"2024-03-26T18:56:23.164852+0000","last_became_peered":"2024-03-26T18:56:23.164852+0000","last_unstale":"2024-03-26T18:56:25.732859+0000","last_undegraded":"2024-03-26T18:56:25.732859+0000","last_fullsized":"2024-03-26T18:56:25.732859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:16:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732315+0000","last_change":"2024-03-26T18:56:24.154235+0000","last_active":"2024-03-26T18:56:25.732315+0000","last_peered":"2024-03-26T18:56:25.732315+0000","last_clean":"2024-03-26T18:56:25.732315+0000","last_became_active":"2024-03-26T18:56:24.153864+0000","last_became_peered":"2024-03-26T18:56:24.153864+0000","last_unstale":"2024-03-26T18:56:25.732315+0000","last_undegraded":"2024-03-26T18:56:25.732315+0000","last_fullsized":"2024-03-26T18:56:25.732315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:30:36.577695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.734109+0000","last_change":"2024-03-26T18:56:23.175807+0000","last_active":"2024-03-26T18:56:25.734109+0000","last_peered":"2024-03-26T18:56:25.734109+0000","last_clean":"2024-03-26T18:56:25.734109+0000","last_became_active":"2024-03-26T18:56:23.175477+0000","last_became_peered":"2024-03-26T18:56:23.175477+0000","last_unstale":"2024-03-26T18:56:25.734109+0000","last_undegraded":"2024-03-26T18:56:25.734109+0000","last_fullsized":"2024-03-26T18:56:25.734109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:45:28.494823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732294+0000","last_change":"2024-03-26T18:56:24.145201+0000","last_active":"2024-03-26T18:56:25.732294+0000","last_peered":"2024-03-26T18:56:25.732294+0000","last_clean":"2024-03-26T18:56:25.732294+0000","last_became_active":"2024-03-26T18:56:24.144793+0000","last_became_peered":"2024-03-26T18:56:24.144793+0000","last_unstale":"2024-03-26T18:56:25.732294+0000","last_undegraded":"2024-03-26T18:56:25.732294+0000","last_fullsized":"2024-03-26T18:56:25.732294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:13:55.476075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884555+0000","last_change":"2024-03-26T18:56:24.145445+0000","last_active":"2024-03-26T18:56:25.884555+0000","last_peered":"2024-03-26T18:56:25.884555+0000","last_clean":"2024-03-26T18:56:25.884555+0000","last_became_active":"2024-03-26T18:56:24.145115+0000","last_became_peered":"2024-03-26T18:56:24.145115+0000","last_unstale":"2024-03-26T18:56:25.884555+0000","last_undegraded":"2024-03-26T18:56:25.884555+0000","last_fullsized":"2024-03-26T18:56:25.884555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:30:45.728966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.129873+0000","last_change":"2024-03-26T18:56:24.148307+0000","last_active":"2024-03-26T18:56:25.129873+0000","last_peered":"2024-03-26T18:56:25.129873+0000","last_clean":"2024-03-26T18:56:25.129873+0000","last_became_active":"2024-03-26T18:56:24.147973+0000","last_became_peered":"2024-03-26T18:56:24.147973+0000","last_unstale":"2024-03-26T18:56:25.129873+0000","last_undegraded":"2024-03-26T18:56:25.129873+0000","last_fullsized":"2024-03-26T18:56:25.129873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:36:23.330139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.129890+0000","last_change":"2024-03-26T18:56:24.175594+0000","last_active":"2024-03-26T18:56:25.129890+0000","last_peered":"2024-03-26T18:56:25.129890+0000","last_clean":"2024-03-26T18:56:25.129890+0000","last_became_active":"2024-03-26T18:56:24.175261+0000","last_became_peered":"2024-03-26T18:56:24.175261+0000","last_unstale":"2024-03-26T18:56:25.129890+0000","last_undegraded":"2024-03-26T18:56:25.129890+0000","last_fullsized":"2024-03-26T18:56:25.129890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:12:02.744475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.215936+0000","last_change":"2024-03-26T18:56:24.177648+0000","last_active":"2024-03-26T18:56:25.215936+0000","last_peered":"2024-03-26T18:56:25.215936+0000","last_clean":"2024-03-26T18:56:25.215936+0000","last_became_active":"2024-03-26T18:56:24.176768+0000","last_became_peered":"2024-03-26T18:56:24.176768+0000","last_unstale":"2024-03-26T18:56:25.215936+0000","last_undegraded":"2024-03-26T18:56:25.215936+0000","last_fullsized":"2024-03-26T18:56:25.215936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:24:10.207407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733960+0000","last_change":"2024-03-26T18:56:24.166746+0000","last_active":"2024-03-26T18:56:25.733960+0000","last_peered":"2024-03-26T18:56:25.733960+0000","last_clean":"2024-03-26T18:56:25.733960+0000","last_became_active":"2024-03-26T18:56:24.166220+0000","last_became_peered":"2024-03-26T18:56:24.166220+0000","last_unstale":"2024-03-26T18:56:25.733960+0000","last_undegraded":"2024-03-26T18:56:25.733960+0000","last_fullsized":"2024-03-26T18:56:25.733960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:16:54.411740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884619+0000","last_change":"2024-03-26T18:56:24.145511+0000","last_active":"2024-03-26T18:56:25.884619+0000","last_peered":"2024-03-26T18:56:25.884619+0000","last_clean":"2024-03-26T18:56:25.884619+0000","last_became_active":"2024-03-26T18:56:24.145164+0000","last_became_peered":"2024-03-26T18:56:24.145164+0000","last_unstale":"2024-03-26T18:56:25.884619+0000","last_undegraded":"2024-03-26T18:56:25.884619+0000","last_fullsized":"2024-03-26T18:56:25.884619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:25:05.239952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.130258+0000","last_change":"2024-03-26T18:56:24.145367+0000","last_active":"2024-03-26T18:56:25.130258+0000","last_peered":"2024-03-26T18:56:25.130258+0000","last_clean":"2024-03-26T18:56:25.130258+0000","last_became_active":"2024-03-26T18:56:24.144058+0000","last_became_peered":"2024-03-26T18:56:24.144058+0000","last_unstale":"2024-03-26T18:56:25.130258+0000","last_undegraded":"2024-03-26T18:56:25.130258+0000","last_fullsized":"2024-03-26T18:56:25.130258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:10.440142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551458+0000","last_change":"2024-03-26T18:56:24.174209+0000","last_active":"2024-03-26T18:56:25.551458+0000","last_peered":"2024-03-26T18:56:25.551458+0000","last_clean":"2024-03-26T18:56:25.551458+0000","last_became_active":"2024-03-26T18:56:24.173873+0000","last_became_peered":"2024-03-26T18:56:24.173873+0000","last_unstale":"2024-03-26T18:56:25.551458+0000","last_undegraded":"2024-03-26T18:56:25.551458+0000","last_fullsized":"2024-03-26T18:56:25.551458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:21:34.317699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732643+0000","last_change":"2024-03-26T18:56:24.166633+0000","last_active":"2024-03-26T18:56:25.732643+0000","last_peered":"2024-03-26T18:56:25.732643+0000","last_clean":"2024-03-26T18:56:25.732643+0000","last_became_active":"2024-03-26T18:56:24.166372+0000","last_became_peered":"2024-03-26T18:56:24.166372+0000","last_unstale":"2024-03-26T18:56:25.732643+0000","last_undegraded":"2024-03-26T18:56:25.732643+0000","last_fullsized":"2024-03-26T18:56:25.732643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:38:18.005223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.550635+0000","last_change":"2024-03-26T18:56:24.179979+0000","last_active":"2024-03-26T18:56:25.550635+0000","last_peered":"2024-03-26T18:56:25.550635+0000","last_clean":"2024-03-26T18:56:25.550635+0000","last_became_active":"2024-03-26T18:56:24.179533+0000","last_became_peered":"2024-03-26T18:56:24.179533+0000","last_unstale":"2024-03-26T18:56:25.550635+0000","last_undegraded":"2024-03-26T18:56:25.550635+0000","last_fullsized":"2024-03-26T18:56:25.550635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:19:26.104430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127635+0000","last_change":"2024-03-26T18:56:24.175802+0000","last_active":"2024-03-26T18:56:25.127635+0000","last_peered":"2024-03-26T18:56:25.127635+0000","last_clean":"2024-03-26T18:56:25.127635+0000","last_became_active":"2024-03-26T18:56:24.175451+0000","last_became_peered":"2024-03-26T18:56:24.175451+0000","last_unstale":"2024-03-26T18:56:25.127635+0000","last_undegraded":"2024-03-26T18:56:25.127635+0000","last_fullsized":"2024-03-26T18:56:25.127635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:48:07.444891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.734012+0000","last_change":"2024-03-26T18:56:24.167708+0000","last_active":"2024-03-26T18:56:25.734012+0000","last_peered":"2024-03-26T18:56:25.734012+0000","last_clean":"2024-03-26T18:56:25.734012+0000","last_became_active":"2024-03-26T18:56:24.166985+0000","last_became_peered":"2024-03-26T18:56:24.166985+0000","last_unstale":"2024-03-26T18:56:25.734012+0000","last_undegraded":"2024-03-26T18:56:25.734012+0000","last_fullsized":"2024-03-26T18:56:25.734012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:13:35.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884455+0000","last_change":"2024-03-26T18:56:24.151474+0000","last_active":"2024-03-26T18:56:25.884455+0000","last_peered":"2024-03-26T18:56:25.884455+0000","last_clean":"2024-03-26T18:56:25.884455+0000","last_became_active":"2024-03-26T18:56:24.151151+0000","last_became_peered":"2024-03-26T18:56:24.151151+0000","last_unstale":"2024-03-26T18:56:25.884455+0000","last_undegraded":"2024-03-26T18:56:25.884455+0000","last_fullsized":"2024-03-26T18:56:25.884455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:34.926612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280636,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5222,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":123,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":315392,"data_stored":63288,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":439305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":1030093,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":479901,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":448705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":921600,"data_stored":438625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":41380,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T18:56:33.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T18:56:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.057+0000 7f7da2200640 1 -- 172.21.15.97:0/1257882117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d9c075900 msgr2=0x7f7d9c073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.057+0000 7f7da2200640 1 --2- 172.21.15.97:0/1257882117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d9c075900 0x7f7d9c073d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009900 tx=0x7f7d8c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:56:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.057+0000 7f7da2200640 1 -- 172.21.15.97:0/1257882117 shutdown_connections 2024-03-26T18:56:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.057+0000 7f7da2200640 1 --2- 172.21.15.97:0/1257882117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d9c1097a0 0x7f7d9c10bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.057+0000 7f7da2200640 1 --2- 172.21.15.97:0/1257882117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9c074320 0x7f7d9c0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.057+0000 7f7da2200640 1 --2- 172.21.15.97:0/1257882117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d9c075900 0x7f7d9c073d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.057+0000 7f7da2200640 1 -- 172.21.15.97:0/1257882117 >> 172.21.15.97:0/1257882117 conn(0x7f7d9c0fce70 msgr2=0x7f7d9c0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.057+0000 7f7da2200640 1 -- 172.21.15.97:0/1257882117 shutdown_connections 2024-03-26T18:56:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.057+0000 7f7da2200640 1 -- 172.21.15.97:0/1257882117 wait complete. 2024-03-26T18:56:34.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.059+0000 7f7da2200640 1 Processor -- start 2024-03-26T18:56:34.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.059+0000 7f7da2200640 1 -- start start 2024-03-26T18:56:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7da2200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d9c074320 0x7f7d9c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7da2200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d9c075900 0x7f7d9c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7da2200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9c1097a0 0x7f7d9c1a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7da2200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10f740 con 0x7f7d9c1097a0 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7da2200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10f5c0 con 0x7f7d9c075900 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7da2200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10f8c0 con 0x7f7d9c074320 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d9c074320 0x7f7d9c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d9c075900 0x7f7d9c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d9c074320 0x7f7d9c199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:59426/0 (socket says 172.21.15.97:59426) 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9b400640 1 -- 172.21.15.97:0/4219967839 learned_addr learned my addr 172.21.15.97:0/4219967839 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9be00640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9c1097a0 0x7f7d9c1a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9b400640 1 -- 172.21.15.97:0/4219967839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d9c075900 msgr2=0x7f7d9c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9b400640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d9c075900 0x7f7d9c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9b400640 1 -- 172.21.15.97:0/4219967839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9c1097a0 msgr2=0x7f7d9c1a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9b400640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9c1097a0 0x7f7d9c1a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9b400640 1 -- 172.21.15.97:0/4219967839 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c1a19f0 con 0x7f7d9c074320 2024-03-26T18:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.060+0000 7f7d9aa00640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d9c075900 0x7f7d9c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.061+0000 7f7d9b400640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d9c074320 0x7f7d9c199dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009230 tx=0x7f7d8c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.061+0000 7f7d93e00640 1 -- 172.21.15.97:0/4219967839 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c041070 con 0x7f7d9c074320 2024-03-26T18:56:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.061+0000 7f7d93e00640 1 -- 172.21.15.97:0/4219967839 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d8c004450 con 0x7f7d9c074320 2024-03-26T18:56:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.061+0000 7f7da2200640 1 -- 172.21.15.97:0/4219967839 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c1a1c80 con 0x7f7d9c074320 2024-03-26T18:56:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.061+0000 7f7d93e00640 1 -- 172.21.15.97:0/4219967839 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c003e20 con 0x7f7d9c074320 2024-03-26T18:56:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.061+0000 7f7da2200640 1 -- 172.21.15.97:0/4219967839 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9c1a21c0 con 0x7f7d9c074320 2024-03-26T18:56:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.062+0000 7f7da2200640 1 -- 172.21.15.97:0/4219967839 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9c1084d0 con 0x7f7d9c074320 2024-03-26T18:56:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.065+0000 7f7d93e00640 1 -- 172.21.15.97:0/4219967839 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f7d8c044070 con 0x7f7d9c074320 2024-03-26T18:56:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.068+0000 7f7d93e00640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7d800a3030 0x7f7d800a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.068+0000 7f7d93e00640 1 -- 172.21.15.97:0/4219967839 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7d8c0f0ff0 con 0x7f7d9c074320 2024-03-26T18:56:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.068+0000 7f7d9aa00640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7d800a3030 0x7f7d800a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.068+0000 7f7d93e00640 1 -- 172.21.15.97:0/4219967839 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7d8c0bdec0 con 0x7f7d9c074320 2024-03-26T18:56:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.069+0000 7f7d9aa00640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7d800a3030 0x7f7d800a54f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c19b2e0 tx=0x7f7d88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:34.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:33 smithi097 ceph-mon[31955]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:34.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:33 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2134608182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:34.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:33 smithi097 ceph-mon[31955]: pgmap v317: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 699 B/s rd, 4.3 KiB/s wr, 14 op/s 2024-03-26T18:56:34.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:33 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3440779044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:34.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:33 smithi144 ceph-mon[36053]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:34.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:33 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2134608182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:34.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:33 smithi144 ceph-mon[36053]: pgmap v317: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 699 B/s rd, 4.3 KiB/s wr, 14 op/s 2024-03-26T18:56:34.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:33 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3440779044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:34.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:33 smithi105 ceph-mon[36270]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:56:34.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:33 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2134608182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:56:34.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:33 smithi105 ceph-mon[36270]: pgmap v317: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 699 B/s rd, 4.3 KiB/s wr, 14 op/s 2024-03-26T18:56:34.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:33 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3440779044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:34.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.264+0000 7f7da2200640 1 -- 172.21.15.97:0/4219967839 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d9c061d00 con 0x7f7d800a3030 2024-03-26T18:56:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.273+0000 7f7d93e00640 1 -- 172.21.15.97:0/4219967839 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258909 (secure 0 0 0) 0x7f7d9c061d00 con 0x7f7d800a3030 2024-03-26T18:56:34.269 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:34.270 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-26T18:56:34.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 -- 172.21.15.97:0/4219967839 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7d800a3030 msgr2=0x7f7d800a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:34.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7d800a3030 0x7f7d800a54f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c19b2e0 tx=0x7f7d88005f90 comp rx=0 tx=0).stop 2024-03-26T18:56:34.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 -- 172.21.15.97:0/4219967839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d9c074320 msgr2=0x7f7d9c199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:34.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d9c074320 0x7f7d9c199dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009230 tx=0x7f7d8c02c650 comp rx=0 tx=0).stop 2024-03-26T18:56:34.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 -- 172.21.15.97:0/4219967839 shutdown_connections 2024-03-26T18:56:34.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7d800a3030 0x7f7d800a54f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:34.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9c1097a0 0x7f7d9c1a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:34.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d9c075900 0x7f7d9c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:34.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 --2- 172.21.15.97:0/4219967839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d9c074320 0x7f7d9c199dd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:34.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 -- 172.21.15.97:0/4219967839 >> 172.21.15.97:0/4219967839 conn(0x7f7d9c0fce70 msgr2=0x7f7d9c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:34.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 -- 172.21.15.97:0/4219967839 shutdown_connections 2024-03-26T18:56:34.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:34.278+0000 7f7da2200640 1 -- 172.21.15.97:0/4219967839 wait complete. 2024-03-26T18:56:34.291 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-03-26T18:56:32.908002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595590,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":133,"num_read_kb":115,"num_write":304,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360292,"kb_used_data":11128,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581532,"statfs":{"total":1151940427776,"available":1151571488768,"internally_reserved":0,"allocated":11395072,"data_stored":7095696,"data_compressed":108958,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5222,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":123,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.781961"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884766+0000","last_change":"2024-03-26T18:56:24.177714+0000","last_active":"2024-03-26T18:56:25.884766+0000","last_peered":"2024-03-26T18:56:25.884766+0000","last_clean":"2024-03-26T18:56:25.884766+0000","last_became_active":"2024-03-26T18:56:24.177420+0000","last_became_peered":"2024-03-26T18:56:24.177420+0000","last_unstale":"2024-03-26T18:56:25.884766+0000","last_undegraded":"2024-03-26T18:56:25.884766+0000","last_fullsized":"2024-03-26T18:56:25.884766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:21:23.738513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216557+0000","last_change":"2024-03-26T18:56:24.160191+0000","last_active":"2024-03-26T18:56:25.216557+0000","last_peered":"2024-03-26T18:56:25.216557+0000","last_clean":"2024-03-26T18:56:25.216557+0000","last_became_active":"2024-03-26T18:56:24.159718+0000","last_became_peered":"2024-03-26T18:56:24.159718+0000","last_unstale":"2024-03-26T18:56:25.216557+0000","last_undegraded":"2024-03-26T18:56:25.216557+0000","last_fullsized":"2024-03-26T18:56:25.216557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:12:04.287352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216230+0000","last_change":"2024-03-26T18:56:24.169795+0000","last_active":"2024-03-26T18:56:25.216230+0000","last_peered":"2024-03-26T18:56:25.216230+0000","last_clean":"2024-03-26T18:56:25.216230+0000","last_became_active":"2024-03-26T18:56:24.166044+0000","last_became_peered":"2024-03-26T18:56:24.166044+0000","last_unstale":"2024-03-26T18:56:25.216230+0000","last_undegraded":"2024-03-26T18:56:25.216230+0000","last_fullsized":"2024-03-26T18:56:25.216230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:21:58.951019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.288457+0000","last_change":"2024-03-26T18:56:24.152606+0000","last_active":"2024-03-26T18:56:25.288457+0000","last_peered":"2024-03-26T18:56:25.288457+0000","last_clean":"2024-03-26T18:56:25.288457+0000","last_became_active":"2024-03-26T18:56:24.152172+0000","last_became_peered":"2024-03-26T18:56:24.152172+0000","last_unstale":"2024-03-26T18:56:25.288457+0000","last_undegraded":"2024-03-26T18:56:25.288457+0000","last_fullsized":"2024-03-26T18:56:25.288457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:21:25.035154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549169+0000","last_change":"2024-03-26T18:56:24.175902+0000","last_active":"2024-03-26T18:56:25.549169+0000","last_peered":"2024-03-26T18:56:25.549169+0000","last_clean":"2024-03-26T18:56:25.549169+0000","last_became_active":"2024-03-26T18:56:24.175568+0000","last_became_peered":"2024-03-26T18:56:24.175568+0000","last_unstale":"2024-03-26T18:56:25.549169+0000","last_undegraded":"2024-03-26T18:56:25.549169+0000","last_fullsized":"2024-03-26T18:56:25.549169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:33:14.054102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732694+0000","last_change":"2024-03-26T18:56:24.145629+0000","last_active":"2024-03-26T18:56:25.732694+0000","last_peered":"2024-03-26T18:56:25.732694+0000","last_clean":"2024-03-26T18:56:25.732694+0000","last_became_active":"2024-03-26T18:56:24.145286+0000","last_became_peered":"2024-03-26T18:56:24.145286+0000","last_unstale":"2024-03-26T18:56:25.732694+0000","last_undegraded":"2024-03-26T18:56:25.732694+0000","last_fullsized":"2024-03-26T18:56:25.732694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:37:24.989479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732081+0000","last_change":"2024-03-26T18:56:24.150869+0000","last_active":"2024-03-26T18:56:25.732081+0000","last_peered":"2024-03-26T18:56:25.732081+0000","last_clean":"2024-03-26T18:56:25.732081+0000","last_became_active":"2024-03-26T18:56:24.150474+0000","last_became_peered":"2024-03-26T18:56:24.150474+0000","last_unstale":"2024-03-26T18:56:25.732081+0000","last_undegraded":"2024-03-26T18:56:25.732081+0000","last_fullsized":"2024-03-26T18:56:25.732081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:55.113389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.550559+0000","last_change":"2024-03-26T18:56:24.170218+0000","last_active":"2024-03-26T18:56:25.550559+0000","last_peered":"2024-03-26T18:56:25.550559+0000","last_clean":"2024-03-26T18:56:25.550559+0000","last_became_active":"2024-03-26T18:56:24.169343+0000","last_became_peered":"2024-03-26T18:56:24.169343+0000","last_unstale":"2024-03-26T18:56:25.550559+0000","last_undegraded":"2024-03-26T18:56:25.550559+0000","last_fullsized":"2024-03-26T18:56:25.550559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:28:41.609616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127828+0000","last_change":"2024-03-26T18:56:24.146892+0000","last_active":"2024-03-26T18:56:25.127828+0000","last_peered":"2024-03-26T18:56:25.127828+0000","last_clean":"2024-03-26T18:56:25.127828+0000","last_became_active":"2024-03-26T18:56:24.146564+0000","last_became_peered":"2024-03-26T18:56:24.146564+0000","last_unstale":"2024-03-26T18:56:25.127828+0000","last_undegraded":"2024-03-26T18:56:25.127828+0000","last_fullsized":"2024-03-26T18:56:25.127828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:12:42.104605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732962+0000","last_change":"2024-03-26T18:56:24.148110+0000","last_active":"2024-03-26T18:56:25.732962+0000","last_peered":"2024-03-26T18:56:25.732962+0000","last_clean":"2024-03-26T18:56:25.732962+0000","last_became_active":"2024-03-26T18:56:24.147763+0000","last_became_peered":"2024-03-26T18:56:24.147763+0000","last_unstale":"2024-03-26T18:56:25.732962+0000","last_undegraded":"2024-03-26T18:56:25.732962+0000","last_fullsized":"2024-03-26T18:56:25.732962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:23.199428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732357+0000","last_change":"2024-03-26T18:56:24.149064+0000","last_active":"2024-03-26T18:56:25.732357+0000","last_peered":"2024-03-26T18:56:25.732357+0000","last_clean":"2024-03-26T18:56:25.732357+0000","last_became_active":"2024-03-26T18:56:24.148400+0000","last_became_peered":"2024-03-26T18:56:24.148400+0000","last_unstale":"2024-03-26T18:56:25.732357+0000","last_undegraded":"2024-03-26T18:56:25.732357+0000","last_fullsized":"2024-03-26T18:56:25.732357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:08:14.141931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.288434+0000","last_change":"2024-03-26T18:56:24.154166+0000","last_active":"2024-03-26T18:56:25.288434+0000","last_peered":"2024-03-26T18:56:25.288434+0000","last_clean":"2024-03-26T18:56:25.288434+0000","last_became_active":"2024-03-26T18:56:24.153630+0000","last_became_peered":"2024-03-26T18:56:24.153630+0000","last_unstale":"2024-03-26T18:56:25.288434+0000","last_undegraded":"2024-03-26T18:56:25.288434+0000","last_fullsized":"2024-03-26T18:56:25.288434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:01:10.067127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.734099+0000","last_change":"2024-03-26T18:56:24.165819+0000","last_active":"2024-03-26T18:56:25.734099+0000","last_peered":"2024-03-26T18:56:25.734099+0000","last_clean":"2024-03-26T18:56:25.734099+0000","last_became_active":"2024-03-26T18:56:24.164401+0000","last_became_peered":"2024-03-26T18:56:24.164401+0000","last_unstale":"2024-03-26T18:56:25.734099+0000","last_undegraded":"2024-03-26T18:56:25.734099+0000","last_fullsized":"2024-03-26T18:56:25.734099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:08:32.301805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216413+0000","last_change":"2024-03-26T18:56:24.174229+0000","last_active":"2024-03-26T18:56:25.216413+0000","last_peered":"2024-03-26T18:56:25.216413+0000","last_clean":"2024-03-26T18:56:25.216413+0000","last_became_active":"2024-03-26T18:56:24.172924+0000","last_became_peered":"2024-03-26T18:56:24.172924+0000","last_unstale":"2024-03-26T18:56:25.216413+0000","last_undegraded":"2024-03-26T18:56:25.216413+0000","last_fullsized":"2024-03-26T18:56:25.216413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:26:00.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.215795+0000","last_change":"2024-03-26T18:56:24.165029+0000","last_active":"2024-03-26T18:56:25.215795+0000","last_peered":"2024-03-26T18:56:25.215795+0000","last_clean":"2024-03-26T18:56:25.215795+0000","last_became_active":"2024-03-26T18:56:24.160238+0000","last_became_peered":"2024-03-26T18:56:24.160238+0000","last_unstale":"2024-03-26T18:56:25.215795+0000","last_undegraded":"2024-03-26T18:56:25.215795+0000","last_fullsized":"2024-03-26T18:56:25.215795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:56:14.509564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127429+0000","last_change":"2024-03-26T18:56:24.167668+0000","last_active":"2024-03-26T18:56:25.127429+0000","last_peered":"2024-03-26T18:56:25.127429+0000","last_clean":"2024-03-26T18:56:25.127429+0000","last_became_active":"2024-03-26T18:56:24.167267+0000","last_became_peered":"2024-03-26T18:56:24.167267+0000","last_unstale":"2024-03-26T18:56:25.127429+0000","last_undegraded":"2024-03-26T18:56:25.127429+0000","last_fullsized":"2024-03-26T18:56:25.127429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:13:00.503977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884777+0000","last_change":"2024-03-26T18:56:24.177623+0000","last_active":"2024-03-26T18:56:25.884777+0000","last_peered":"2024-03-26T18:56:25.884777+0000","last_clean":"2024-03-26T18:56:25.884777+0000","last_became_active":"2024-03-26T18:56:24.177372+0000","last_became_peered":"2024-03-26T18:56:24.177372+0000","last_unstale":"2024-03-26T18:56:25.884777+0000","last_undegraded":"2024-03-26T18:56:25.884777+0000","last_fullsized":"2024-03-26T18:56:25.884777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:02:20.650520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.131162+0000","last_change":"2024-03-26T18:56:24.152407+0000","last_active":"2024-03-26T18:56:25.131162+0000","last_peered":"2024-03-26T18:56:25.131162+0000","last_clean":"2024-03-26T18:56:25.131162+0000","last_became_active":"2024-03-26T18:56:24.151935+0000","last_became_peered":"2024-03-26T18:56:24.151935+0000","last_unstale":"2024-03-26T18:56:25.131162+0000","last_undegraded":"2024-03-26T18:56:25.131162+0000","last_fullsized":"2024-03-26T18:56:25.131162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:52:13.211124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.130169+0000","last_change":"2024-03-26T18:56:24.162582+0000","last_active":"2024-03-26T18:56:25.130169+0000","last_peered":"2024-03-26T18:56:25.130169+0000","last_clean":"2024-03-26T18:56:25.130169+0000","last_became_active":"2024-03-26T18:56:24.162228+0000","last_became_peered":"2024-03-26T18:56:24.162228+0000","last_unstale":"2024-03-26T18:56:25.130169+0000","last_undegraded":"2024-03-26T18:56:25.130169+0000","last_fullsized":"2024-03-26T18:56:25.130169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:16.841466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.864317+0000","last_change":"2024-03-26T18:56:23.149475+0000","last_active":"2024-03-26T18:56:28.864317+0000","last_peered":"2024-03-26T18:56:28.864317+0000","last_clean":"2024-03-26T18:56:28.864317+0000","last_became_active":"2024-03-26T18:56:23.148813+0000","last_became_peered":"2024-03-26T18:56:23.148813+0000","last_unstale":"2024-03-26T18:56:28.864317+0000","last_undegraded":"2024-03-26T18:56:28.864317+0000","last_fullsized":"2024-03-26T18:56:28.864317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:38.315414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551660+0000","last_change":"2024-03-26T18:56:24.173904+0000","last_active":"2024-03-26T18:56:25.551660+0000","last_peered":"2024-03-26T18:56:25.551660+0000","last_clean":"2024-03-26T18:56:25.551660+0000","last_became_active":"2024-03-26T18:56:24.173523+0000","last_became_peered":"2024-03-26T18:56:24.173523+0000","last_unstale":"2024-03-26T18:56:25.551660+0000","last_undegraded":"2024-03-26T18:56:25.551660+0000","last_fullsized":"2024-03-26T18:56:25.551660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:40:35.651821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551461+0000","last_change":"2024-03-26T18:56:23.173583+0000","last_active":"2024-03-26T18:56:25.551461+0000","last_peered":"2024-03-26T18:56:25.551461+0000","last_clean":"2024-03-26T18:56:25.551461+0000","last_became_active":"2024-03-26T18:56:23.172071+0000","last_became_peered":"2024-03-26T18:56:23.172071+0000","last_unstale":"2024-03-26T18:56:25.551461+0000","last_undegraded":"2024-03-26T18:56:25.551461+0000","last_fullsized":"2024-03-26T18:56:25.551461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:29:39.194411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216502+0000","last_change":"2024-03-26T18:56:24.175535+0000","last_active":"2024-03-26T18:56:25.216502+0000","last_peered":"2024-03-26T18:56:25.216502+0000","last_clean":"2024-03-26T18:56:25.216502+0000","last_became_active":"2024-03-26T18:56:24.173748+0000","last_became_peered":"2024-03-26T18:56:24.173748+0000","last_unstale":"2024-03-26T18:56:25.216502+0000","last_undegraded":"2024-03-26T18:56:25.216502+0000","last_fullsized":"2024-03-26T18:56:25.216502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:38.960606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.857139+0000","last_change":"2024-03-26T18:56:23.150303+0000","last_active":"2024-03-26T18:56:28.857139+0000","last_peered":"2024-03-26T18:56:28.857139+0000","last_clean":"2024-03-26T18:56:28.857139+0000","last_became_active":"2024-03-26T18:56:23.149840+0000","last_became_peered":"2024-03-26T18:56:23.149840+0000","last_unstale":"2024-03-26T18:56:28.857139+0000","last_undegraded":"2024-03-26T18:56:28.857139+0000","last_fullsized":"2024-03-26T18:56:28.857139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:01:09.356776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127553+0000","last_change":"2024-03-26T18:56:24.161351+0000","last_active":"2024-03-26T18:56:25.127553+0000","last_peered":"2024-03-26T18:56:25.127553+0000","last_clean":"2024-03-26T18:56:25.127553+0000","last_became_active":"2024-03-26T18:56:24.161013+0000","last_became_peered":"2024-03-26T18:56:24.161013+0000","last_unstale":"2024-03-26T18:56:25.127553+0000","last_undegraded":"2024-03-26T18:56:25.127553+0000","last_fullsized":"2024-03-26T18:56:25.127553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:24:25.620772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549381+0000","last_change":"2024-03-26T18:56:23.148866+0000","last_active":"2024-03-26T18:56:25.549381+0000","last_peered":"2024-03-26T18:56:25.549381+0000","last_clean":"2024-03-26T18:56:25.549381+0000","last_became_active":"2024-03-26T18:56:23.148434+0000","last_became_peered":"2024-03-26T18:56:23.148434+0000","last_unstale":"2024-03-26T18:56:25.549381+0000","last_undegraded":"2024-03-26T18:56:25.549381+0000","last_fullsized":"2024-03-26T18:56:25.549381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:35:35.180038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733128+0000","last_change":"2024-03-26T18:56:24.147429+0000","last_active":"2024-03-26T18:56:25.733128+0000","last_peered":"2024-03-26T18:56:25.733128+0000","last_clean":"2024-03-26T18:56:25.733128+0000","last_became_active":"2024-03-26T18:56:24.147100+0000","last_became_peered":"2024-03-26T18:56:24.147100+0000","last_unstale":"2024-03-26T18:56:25.733128+0000","last_undegraded":"2024-03-26T18:56:25.733128+0000","last_fullsized":"2024-03-26T18:56:25.733128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:08.884223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.849738+0000","last_change":"2024-03-26T18:56:23.149301+0000","last_active":"2024-03-26T18:56:27.849738+0000","last_peered":"2024-03-26T18:56:27.849738+0000","last_clean":"2024-03-26T18:56:27.849738+0000","last_became_active":"2024-03-26T18:56:23.148939+0000","last_became_peered":"2024-03-26T18:56:23.148939+0000","last_unstale":"2024-03-26T18:56:27.849738+0000","last_undegraded":"2024-03-26T18:56:27.849738+0000","last_fullsized":"2024-03-26T18:56:27.849738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:50:06.649474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549172+0000","last_change":"2024-03-26T18:56:24.157317+0000","last_active":"2024-03-26T18:56:25.549172+0000","last_peered":"2024-03-26T18:56:25.549172+0000","last_clean":"2024-03-26T18:56:25.549172+0000","last_became_active":"2024-03-26T18:56:24.156971+0000","last_became_peered":"2024-03-26T18:56:24.156971+0000","last_unstale":"2024-03-26T18:56:25.549172+0000","last_undegraded":"2024-03-26T18:56:25.549172+0000","last_fullsized":"2024-03-26T18:56:25.549172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:22:39.360404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:29.828242+0000","last_change":"2024-03-26T18:56:23.162848+0000","last_active":"2024-03-26T18:56:29.828242+0000","last_peered":"2024-03-26T18:56:29.828242+0000","last_clean":"2024-03-26T18:56:29.828242+0000","last_became_active":"2024-03-26T18:56:23.162478+0000","last_became_peered":"2024-03-26T18:56:23.162478+0000","last_unstale":"2024-03-26T18:56:29.828242+0000","last_undegraded":"2024-03-26T18:56:29.828242+0000","last_fullsized":"2024-03-26T18:56:29.828242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:07:51.139538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732908+0000","last_change":"2024-03-26T18:56:24.156451+0000","last_active":"2024-03-26T18:56:25.732908+0000","last_peered":"2024-03-26T18:56:25.732908+0000","last_clean":"2024-03-26T18:56:25.732908+0000","last_became_active":"2024-03-26T18:56:24.156237+0000","last_became_peered":"2024-03-26T18:56:24.156237+0000","last_unstale":"2024-03-26T18:56:25.732908+0000","last_undegraded":"2024-03-26T18:56:25.732908+0000","last_fullsized":"2024-03-26T18:56:25.732908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:52:54.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733315+0000","last_change":"2024-03-26T18:56:23.172926+0000","last_active":"2024-03-26T18:56:25.733315+0000","last_peered":"2024-03-26T18:56:25.733315+0000","last_clean":"2024-03-26T18:56:25.733315+0000","last_became_active":"2024-03-26T18:56:23.170512+0000","last_became_peered":"2024-03-26T18:56:23.170512+0000","last_unstale":"2024-03-26T18:56:25.733315+0000","last_undegraded":"2024-03-26T18:56:25.733315+0000","last_fullsized":"2024-03-26T18:56:25.733315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:16:13.747853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551275+0000","last_change":"2024-03-26T18:56:24.173733+0000","last_active":"2024-03-26T18:56:25.551275+0000","last_peered":"2024-03-26T18:56:25.551275+0000","last_clean":"2024-03-26T18:56:25.551275+0000","last_became_active":"2024-03-26T18:56:24.171895+0000","last_became_peered":"2024-03-26T18:56:24.171895+0000","last_unstale":"2024-03-26T18:56:25.551275+0000","last_undegraded":"2024-03-26T18:56:25.551275+0000","last_fullsized":"2024-03-26T18:56:25.551275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:07:59.273853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732445+0000","last_change":"2024-03-26T18:56:23.150140+0000","last_active":"2024-03-26T18:56:25.732445+0000","last_peered":"2024-03-26T18:56:25.732445+0000","last_clean":"2024-03-26T18:56:25.732445+0000","last_became_active":"2024-03-26T18:56:23.149634+0000","last_became_peered":"2024-03-26T18:56:23.149634+0000","last_unstale":"2024-03-26T18:56:25.732445+0000","last_undegraded":"2024-03-26T18:56:25.732445+0000","last_fullsized":"2024-03-26T18:56:25.732445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:52:48.025868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.130666+0000","last_change":"2024-03-26T18:56:24.157855+0000","last_active":"2024-03-26T18:56:25.130666+0000","last_peered":"2024-03-26T18:56:25.130666+0000","last_clean":"2024-03-26T18:56:25.130666+0000","last_became_active":"2024-03-26T18:56:24.157220+0000","last_became_peered":"2024-03-26T18:56:24.157220+0000","last_unstale":"2024-03-26T18:56:25.130666+0000","last_undegraded":"2024-03-26T18:56:25.130666+0000","last_fullsized":"2024-03-26T18:56:25.130666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:40:00.815178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:29.815058+0000","last_change":"2024-03-26T18:56:23.148341+0000","last_active":"2024-03-26T18:56:29.815058+0000","last_peered":"2024-03-26T18:56:29.815058+0000","last_clean":"2024-03-26T18:56:29.815058+0000","last_became_active":"2024-03-26T18:56:23.148021+0000","last_became_peered":"2024-03-26T18:56:23.148021+0000","last_unstale":"2024-03-26T18:56:29.815058+0000","last_undegraded":"2024-03-26T18:56:29.815058+0000","last_fullsized":"2024-03-26T18:56:29.815058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:42.577047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127877+0000","last_change":"2024-03-26T18:56:24.160017+0000","last_active":"2024-03-26T18:56:25.127877+0000","last_peered":"2024-03-26T18:56:25.127877+0000","last_clean":"2024-03-26T18:56:25.127877+0000","last_became_active":"2024-03-26T18:56:24.159686+0000","last_became_peered":"2024-03-26T18:56:24.159686+0000","last_unstale":"2024-03-26T18:56:25.127877+0000","last_undegraded":"2024-03-26T18:56:25.127877+0000","last_fullsized":"2024-03-26T18:56:25.127877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:33:32.693850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.848126+0000","last_change":"2024-03-26T18:56:23.146487+0000","last_active":"2024-03-26T18:56:27.848126+0000","last_peered":"2024-03-26T18:56:27.848126+0000","last_clean":"2024-03-26T18:56:27.848126+0000","last_became_active":"2024-03-26T18:56:23.146155+0000","last_became_peered":"2024-03-26T18:56:23.146155+0000","last_unstale":"2024-03-26T18:56:27.848126+0000","last_undegraded":"2024-03-26T18:56:27.848126+0000","last_fullsized":"2024-03-26T18:56:27.848126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:09:51.836211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216332+0000","last_change":"2024-03-26T18:56:24.172652+0000","last_active":"2024-03-26T18:56:25.216332+0000","last_peered":"2024-03-26T18:56:25.216332+0000","last_clean":"2024-03-26T18:56:25.216332+0000","last_became_active":"2024-03-26T18:56:24.170778+0000","last_became_peered":"2024-03-26T18:56:24.170778+0000","last_unstale":"2024-03-26T18:56:25.216332+0000","last_undegraded":"2024-03-26T18:56:25.216332+0000","last_fullsized":"2024-03-26T18:56:25.216332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:47:16.946692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733112+0000","last_change":"2024-03-26T18:56:23.151669+0000","last_active":"2024-03-26T18:56:25.733112+0000","last_peered":"2024-03-26T18:56:25.733112+0000","last_clean":"2024-03-26T18:56:25.733112+0000","last_became_active":"2024-03-26T18:56:23.151276+0000","last_became_peered":"2024-03-26T18:56:23.151276+0000","last_unstale":"2024-03-26T18:56:25.733112+0000","last_undegraded":"2024-03-26T18:56:25.733112+0000","last_fullsized":"2024-03-26T18:56:25.733112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:28:37.827930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127702+0000","last_change":"2024-03-26T18:56:24.151007+0000","last_active":"2024-03-26T18:56:25.127702+0000","last_peered":"2024-03-26T18:56:25.127702+0000","last_clean":"2024-03-26T18:56:25.127702+0000","last_became_active":"2024-03-26T18:56:24.150668+0000","last_became_peered":"2024-03-26T18:56:24.150668+0000","last_unstale":"2024-03-26T18:56:25.127702+0000","last_undegraded":"2024-03-26T18:56:25.127702+0000","last_fullsized":"2024-03-26T18:56:25.127702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:07:04.778039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:29.813444+0000","last_change":"2024-03-26T18:56:23.144522+0000","last_active":"2024-03-26T18:56:29.813444+0000","last_peered":"2024-03-26T18:56:29.813444+0000","last_clean":"2024-03-26T18:56:29.813444+0000","last_became_active":"2024-03-26T18:56:23.144176+0000","last_became_peered":"2024-03-26T18:56:23.144176+0000","last_unstale":"2024-03-26T18:56:29.813444+0000","last_undegraded":"2024-03-26T18:56:29.813444+0000","last_fullsized":"2024-03-26T18:56:29.813444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:42:20.319478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549452+0000","last_change":"2024-03-26T18:56:24.148307+0000","last_active":"2024-03-26T18:56:25.549452+0000","last_peered":"2024-03-26T18:56:25.549452+0000","last_clean":"2024-03-26T18:56:25.549452+0000","last_became_active":"2024-03-26T18:56:24.147973+0000","last_became_peered":"2024-03-26T18:56:24.147973+0000","last_unstale":"2024-03-26T18:56:25.549452+0000","last_undegraded":"2024-03-26T18:56:25.549452+0000","last_fullsized":"2024-03-26T18:56:25.549452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:08:45.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:29.811380+0000","last_change":"2024-03-26T18:56:23.163757+0000","last_active":"2024-03-26T18:56:29.811380+0000","last_peered":"2024-03-26T18:56:29.811380+0000","last_clean":"2024-03-26T18:56:29.811380+0000","last_became_active":"2024-03-26T18:56:23.163418+0000","last_became_peered":"2024-03-26T18:56:23.163418+0000","last_unstale":"2024-03-26T18:56:29.811380+0000","last_undegraded":"2024-03-26T18:56:29.811380+0000","last_fullsized":"2024-03-26T18:56:29.811380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:04:09.178853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733422+0000","last_change":"2024-03-26T18:56:24.167274+0000","last_active":"2024-03-26T18:56:25.733422+0000","last_peered":"2024-03-26T18:56:25.733422+0000","last_clean":"2024-03-26T18:56:25.733422+0000","last_became_active":"2024-03-26T18:56:24.166353+0000","last_became_peered":"2024-03-26T18:56:24.166353+0000","last_unstale":"2024-03-26T18:56:25.733422+0000","last_undegraded":"2024-03-26T18:56:25.733422+0000","last_fullsized":"2024-03-26T18:56:25.733422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:14:44.794112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.850379+0000","last_change":"2024-03-26T18:56:23.162115+0000","last_active":"2024-03-26T18:56:27.850379+0000","last_peered":"2024-03-26T18:56:27.850379+0000","last_clean":"2024-03-26T18:56:27.850379+0000","last_became_active":"2024-03-26T18:56:23.161746+0000","last_became_peered":"2024-03-26T18:56:23.161746+0000","last_unstale":"2024-03-26T18:56:27.850379+0000","last_undegraded":"2024-03-26T18:56:27.850379+0000","last_fullsized":"2024-03-26T18:56:27.850379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:08:26.583281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732367+0000","last_change":"2024-03-26T18:56:24.166407+0000","last_active":"2024-03-26T18:56:25.732367+0000","last_peered":"2024-03-26T18:56:25.732367+0000","last_clean":"2024-03-26T18:56:25.732367+0000","last_became_active":"2024-03-26T18:56:24.166077+0000","last_became_peered":"2024-03-26T18:56:24.166077+0000","last_unstale":"2024-03-26T18:56:25.732367+0000","last_undegraded":"2024-03-26T18:56:25.732367+0000","last_fullsized":"2024-03-26T18:56:25.732367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:36:35.160247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.288189+0000","last_change":"2024-03-26T18:56:23.146908+0000","last_active":"2024-03-26T18:56:25.288189+0000","last_peered":"2024-03-26T18:56:25.288189+0000","last_clean":"2024-03-26T18:56:25.288189+0000","last_became_active":"2024-03-26T18:56:23.146576+0000","last_became_peered":"2024-03-26T18:56:23.146576+0000","last_unstale":"2024-03-26T18:56:25.288189+0000","last_undegraded":"2024-03-26T18:56:25.288189+0000","last_fullsized":"2024-03-26T18:56:25.288189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:00:55.708101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732015+0000","last_change":"2024-03-26T18:56:24.147460+0000","last_active":"2024-03-26T18:56:25.732015+0000","last_peered":"2024-03-26T18:56:25.732015+0000","last_clean":"2024-03-26T18:56:25.732015+0000","last_became_active":"2024-03-26T18:56:24.147065+0000","last_became_peered":"2024-03-26T18:56:24.147065+0000","last_unstale":"2024-03-26T18:56:25.732015+0000","last_undegraded":"2024-03-26T18:56:25.732015+0000","last_fullsized":"2024-03-26T18:56:25.732015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:02:14.962735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.865573+0000","last_change":"2024-03-26T18:56:23.152295+0000","last_active":"2024-03-26T18:56:28.865573+0000","last_peered":"2024-03-26T18:56:28.865573+0000","last_clean":"2024-03-26T18:56:28.865573+0000","last_became_active":"2024-03-26T18:56:23.151862+0000","last_became_peered":"2024-03-26T18:56:23.151862+0000","last_unstale":"2024-03-26T18:56:28.865573+0000","last_undegraded":"2024-03-26T18:56:28.865573+0000","last_fullsized":"2024-03-26T18:56:28.865573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:29:41.168993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732810+0000","last_change":"2024-03-26T18:56:24.146993+0000","last_active":"2024-03-26T18:56:25.732810+0000","last_peered":"2024-03-26T18:56:25.732810+0000","last_clean":"2024-03-26T18:56:25.732810+0000","last_became_active":"2024-03-26T18:56:24.146692+0000","last_became_peered":"2024-03-26T18:56:24.146692+0000","last_unstale":"2024-03-26T18:56:25.732810+0000","last_undegraded":"2024-03-26T18:56:25.732810+0000","last_fullsized":"2024-03-26T18:56:25.732810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:08:33.731716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549397+0000","last_change":"2024-03-26T18:56:23.148459+0000","last_active":"2024-03-26T18:56:25.549397+0000","last_peered":"2024-03-26T18:56:25.549397+0000","last_clean":"2024-03-26T18:56:25.549397+0000","last_became_active":"2024-03-26T18:56:23.148097+0000","last_became_peered":"2024-03-26T18:56:23.148097+0000","last_unstale":"2024-03-26T18:56:25.549397+0000","last_undegraded":"2024-03-26T18:56:25.549397+0000","last_fullsized":"2024-03-26T18:56:25.549397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:08:51.990705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884618+0000","last_change":"2024-03-26T18:56:24.146399+0000","last_active":"2024-03-26T18:56:25.884618+0000","last_peered":"2024-03-26T18:56:25.884618+0000","last_clean":"2024-03-26T18:56:25.884618+0000","last_became_active":"2024-03-26T18:56:24.146014+0000","last_became_peered":"2024-03-26T18:56:24.146014+0000","last_unstale":"2024-03-26T18:56:25.884618+0000","last_undegraded":"2024-03-26T18:56:25.884618+0000","last_fullsized":"2024-03-26T18:56:25.884618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:24:26.809602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.130865+0000","last_change":"2024-03-26T18:56:23.148037+0000","last_active":"2024-03-26T18:56:25.130865+0000","last_peered":"2024-03-26T18:56:25.130865+0000","last_clean":"2024-03-26T18:56:25.130865+0000","last_became_active":"2024-03-26T18:56:23.147598+0000","last_became_peered":"2024-03-26T18:56:23.147598+0000","last_unstale":"2024-03-26T18:56:25.130865+0000","last_undegraded":"2024-03-26T18:56:25.130865+0000","last_fullsized":"2024-03-26T18:56:25.130865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:21:55.105299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732745+0000","last_change":"2024-03-26T18:56:24.167488+0000","last_active":"2024-03-26T18:56:25.732745+0000","last_peered":"2024-03-26T18:56:25.732745+0000","last_clean":"2024-03-26T18:56:25.732745+0000","last_became_active":"2024-03-26T18:56:24.167154+0000","last_became_peered":"2024-03-26T18:56:24.167154+0000","last_unstale":"2024-03-26T18:56:25.732745+0000","last_undegraded":"2024-03-26T18:56:25.732745+0000","last_fullsized":"2024-03-26T18:56:25.732745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:00:35.606573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732495+0000","last_change":"2024-03-26T18:56:23.160694+0000","last_active":"2024-03-26T18:56:25.732495+0000","last_peered":"2024-03-26T18:56:25.732495+0000","last_clean":"2024-03-26T18:56:25.732495+0000","last_became_active":"2024-03-26T18:56:23.160416+0000","last_became_peered":"2024-03-26T18:56:23.160416+0000","last_unstale":"2024-03-26T18:56:25.732495+0000","last_undegraded":"2024-03-26T18:56:25.732495+0000","last_fullsized":"2024-03-26T18:56:25.732495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:31:15.616473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732714+0000","last_change":"2024-03-26T18:56:24.170503+0000","last_active":"2024-03-26T18:56:25.732714+0000","last_peered":"2024-03-26T18:56:25.732714+0000","last_clean":"2024-03-26T18:56:25.732714+0000","last_became_active":"2024-03-26T18:56:24.170113+0000","last_became_peered":"2024-03-26T18:56:24.170113+0000","last_unstale":"2024-03-26T18:56:25.732714+0000","last_undegraded":"2024-03-26T18:56:25.732714+0000","last_fullsized":"2024-03-26T18:56:25.732714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:44.076756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732623+0000","last_change":"2024-03-26T18:56:24.146856+0000","last_active":"2024-03-26T18:56:25.732623+0000","last_peered":"2024-03-26T18:56:25.732623+0000","last_clean":"2024-03-26T18:56:25.732623+0000","last_became_active":"2024-03-26T18:56:24.146545+0000","last_became_peered":"2024-03-26T18:56:24.146545+0000","last_unstale":"2024-03-26T18:56:25.732623+0000","last_undegraded":"2024-03-26T18:56:25.732623+0000","last_fullsized":"2024-03-26T18:56:25.732623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:58:26.876614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733292+0000","last_change":"2024-03-26T18:56:23.173650+0000","last_active":"2024-03-26T18:56:25.733292+0000","last_peered":"2024-03-26T18:56:25.733292+0000","last_clean":"2024-03-26T18:56:25.733292+0000","last_became_active":"2024-03-26T18:56:23.173343+0000","last_became_peered":"2024-03-26T18:56:23.173343+0000","last_unstale":"2024-03-26T18:56:25.733292+0000","last_undegraded":"2024-03-26T18:56:25.733292+0000","last_fullsized":"2024-03-26T18:56:25.733292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:42:51.138892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551409+0000","last_change":"2024-03-26T18:56:24.175697+0000","last_active":"2024-03-26T18:56:25.551409+0000","last_peered":"2024-03-26T18:56:25.551409+0000","last_clean":"2024-03-26T18:56:25.551409+0000","last_became_active":"2024-03-26T18:56:24.175159+0000","last_became_peered":"2024-03-26T18:56:24.175159+0000","last_unstale":"2024-03-26T18:56:25.551409+0000","last_undegraded":"2024-03-26T18:56:25.551409+0000","last_fullsized":"2024-03-26T18:56:25.551409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:02:35.581432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.846613+0000","last_change":"2024-03-26T18:56:23.154766+0000","last_active":"2024-03-26T18:56:27.846613+0000","last_peered":"2024-03-26T18:56:27.846613+0000","last_clean":"2024-03-26T18:56:27.846613+0000","last_became_active":"2024-03-26T18:56:23.154475+0000","last_became_peered":"2024-03-26T18:56:23.154475+0000","last_unstale":"2024-03-26T18:56:27.846613+0000","last_undegraded":"2024-03-26T18:56:27.846613+0000","last_fullsized":"2024-03-26T18:56:27.846613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:29:38.484201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732001+0000","last_change":"2024-03-26T18:55:03.797254+0000","last_active":"2024-03-26T18:56:25.732001+0000","last_peered":"2024-03-26T18:56:25.732001+0000","last_clean":"2024-03-26T18:56:25.732001+0000","last_became_active":"2024-03-26T18:55:03.796795+0000","last_became_peered":"2024-03-26T18:55:03.796795+0000","last_unstale":"2024-03-26T18:56:25.732001+0000","last_undegraded":"2024-03-26T18:56:25.732001+0000","last_fullsized":"2024-03-26T18:56:25.732001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:51:11.590736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:51:11.590736+0000","last_clean_scrub_stamp":"2024-03-26T18:51:11.590736+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:48.215695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733151+0000","last_change":"2024-03-26T18:56:24.165552+0000","last_active":"2024-03-26T18:56:25.733151+0000","last_peered":"2024-03-26T18:56:25.733151+0000","last_clean":"2024-03-26T18:56:25.733151+0000","last_became_active":"2024-03-26T18:56:24.164230+0000","last_became_peered":"2024-03-26T18:56:24.164230+0000","last_unstale":"2024-03-26T18:56:25.733151+0000","last_undegraded":"2024-03-26T18:56:25.733151+0000","last_fullsized":"2024-03-26T18:56:25.733151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:36:19.750451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.857740+0000","last_change":"2024-03-26T18:56:23.148791+0000","last_active":"2024-03-26T18:56:28.857740+0000","last_peered":"2024-03-26T18:56:28.857740+0000","last_clean":"2024-03-26T18:56:28.857740+0000","last_became_active":"2024-03-26T18:56:23.148435+0000","last_became_peered":"2024-03-26T18:56:23.148435+0000","last_unstale":"2024-03-26T18:56:28.857740+0000","last_undegraded":"2024-03-26T18:56:28.857740+0000","last_fullsized":"2024-03-26T18:56:28.857740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:13:26.785932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549451+0000","last_change":"2024-03-26T18:56:23.154066+0000","last_active":"2024-03-26T18:56:25.549451+0000","last_peered":"2024-03-26T18:56:25.549451+0000","last_clean":"2024-03-26T18:56:25.549451+0000","last_became_active":"2024-03-26T18:56:23.153813+0000","last_became_peered":"2024-03-26T18:56:23.153813+0000","last_unstale":"2024-03-26T18:56:25.549451+0000","last_undegraded":"2024-03-26T18:56:25.549451+0000","last_fullsized":"2024-03-26T18:56:25.549451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:25:07.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733782+0000","last_change":"2024-03-26T18:56:24.162166+0000","last_active":"2024-03-26T18:56:25.733782+0000","last_peered":"2024-03-26T18:56:25.733782+0000","last_clean":"2024-03-26T18:56:25.733782+0000","last_became_active":"2024-03-26T18:56:24.161623+0000","last_became_peered":"2024-03-26T18:56:24.161623+0000","last_unstale":"2024-03-26T18:56:25.733782+0000","last_undegraded":"2024-03-26T18:56:25.733782+0000","last_fullsized":"2024-03-26T18:56:25.733782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:42.588275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.863926+0000","last_change":"2024-03-26T18:56:23.167462+0000","last_active":"2024-03-26T18:56:28.863926+0000","last_peered":"2024-03-26T18:56:28.863926+0000","last_clean":"2024-03-26T18:56:28.863926+0000","last_became_active":"2024-03-26T18:56:23.167124+0000","last_became_peered":"2024-03-26T18:56:23.167124+0000","last_unstale":"2024-03-26T18:56:28.863926+0000","last_undegraded":"2024-03-26T18:56:28.863926+0000","last_fullsized":"2024-03-26T18:56:28.863926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:38.550421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551224+0000","last_change":"2024-03-26T18:56:24.173164+0000","last_active":"2024-03-26T18:56:25.551224+0000","last_peered":"2024-03-26T18:56:25.551224+0000","last_clean":"2024-03-26T18:56:25.551224+0000","last_became_active":"2024-03-26T18:56:24.170083+0000","last_became_peered":"2024-03-26T18:56:24.170083+0000","last_unstale":"2024-03-26T18:56:25.551224+0000","last_undegraded":"2024-03-26T18:56:25.551224+0000","last_fullsized":"2024-03-26T18:56:25.551224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:26.696625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.846490+0000","last_change":"2024-03-26T18:56:23.162835+0000","last_active":"2024-03-26T18:56:27.846490+0000","last_peered":"2024-03-26T18:56:27.846490+0000","last_clean":"2024-03-26T18:56:27.846490+0000","last_became_active":"2024-03-26T18:56:23.162497+0000","last_became_peered":"2024-03-26T18:56:23.162497+0000","last_unstale":"2024-03-26T18:56:27.846490+0000","last_undegraded":"2024-03-26T18:56:27.846490+0000","last_fullsized":"2024-03-26T18:56:27.846490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:29:47.903712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.216354+0000","last_change":"2024-03-26T18:56:24.178140+0000","last_active":"2024-03-26T18:56:25.216354+0000","last_peered":"2024-03-26T18:56:25.216354+0000","last_clean":"2024-03-26T18:56:25.216354+0000","last_became_active":"2024-03-26T18:56:24.177769+0000","last_became_peered":"2024-03-26T18:56:24.177769+0000","last_unstale":"2024-03-26T18:56:25.216354+0000","last_undegraded":"2024-03-26T18:56:25.216354+0000","last_fullsized":"2024-03-26T18:56:25.216354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:39.714335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.849324+0000","last_change":"2024-03-26T18:56:23.143972+0000","last_active":"2024-03-26T18:56:27.849324+0000","last_peered":"2024-03-26T18:56:27.849324+0000","last_clean":"2024-03-26T18:56:27.849324+0000","last_became_active":"2024-03-26T18:56:23.143395+0000","last_became_peered":"2024-03-26T18:56:23.143395+0000","last_unstale":"2024-03-26T18:56:27.849324+0000","last_undegraded":"2024-03-26T18:56:27.849324+0000","last_fullsized":"2024-03-26T18:56:27.849324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:46:32.548546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.549003+0000","last_change":"2024-03-26T18:56:24.144839+0000","last_active":"2024-03-26T18:56:25.549003+0000","last_peered":"2024-03-26T18:56:25.549003+0000","last_clean":"2024-03-26T18:56:25.549003+0000","last_became_active":"2024-03-26T18:56:24.144519+0000","last_became_peered":"2024-03-26T18:56:24.144519+0000","last_unstale":"2024-03-26T18:56:25.549003+0000","last_undegraded":"2024-03-26T18:56:25.549003+0000","last_fullsized":"2024-03-26T18:56:25.549003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:54:59.787475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.851891+0000","last_change":"2024-03-26T18:56:23.161380+0000","last_active":"2024-03-26T18:56:27.851891+0000","last_peered":"2024-03-26T18:56:27.851891+0000","last_clean":"2024-03-26T18:56:27.851891+0000","last_became_active":"2024-03-26T18:56:23.160376+0000","last_became_peered":"2024-03-26T18:56:23.160376+0000","last_unstale":"2024-03-26T18:56:27.851891+0000","last_undegraded":"2024-03-26T18:56:27.851891+0000","last_fullsized":"2024-03-26T18:56:27.851891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:49.610002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884589+0000","last_change":"2024-03-26T18:56:24.170284+0000","last_active":"2024-03-26T18:56:25.884589+0000","last_peered":"2024-03-26T18:56:25.884589+0000","last_clean":"2024-03-26T18:56:25.884589+0000","last_became_active":"2024-03-26T18:56:24.169954+0000","last_became_peered":"2024-03-26T18:56:24.169954+0000","last_unstale":"2024-03-26T18:56:25.884589+0000","last_undegraded":"2024-03-26T18:56:25.884589+0000","last_fullsized":"2024-03-26T18:56:25.884589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:05:21.383976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:28.845229+0000","last_change":"2024-03-26T18:56:23.163386+0000","last_active":"2024-03-26T18:56:28.845229+0000","last_peered":"2024-03-26T18:56:28.845229+0000","last_clean":"2024-03-26T18:56:28.845229+0000","last_became_active":"2024-03-26T18:56:23.163019+0000","last_became_peered":"2024-03-26T18:56:23.163019+0000","last_unstale":"2024-03-26T18:56:28.845229+0000","last_undegraded":"2024-03-26T18:56:28.845229+0000","last_fullsized":"2024-03-26T18:56:28.845229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:03:05.562147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551125+0000","last_change":"2024-03-26T18:56:24.176940+0000","last_active":"2024-03-26T18:56:25.551125+0000","last_peered":"2024-03-26T18:56:25.551125+0000","last_clean":"2024-03-26T18:56:25.551125+0000","last_became_active":"2024-03-26T18:56:24.176396+0000","last_became_peered":"2024-03-26T18:56:24.176396+0000","last_unstale":"2024-03-26T18:56:25.551125+0000","last_undegraded":"2024-03-26T18:56:25.551125+0000","last_fullsized":"2024-03-26T18:56:25.551125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:53:06.994055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:27.846898+0000","last_change":"2024-03-26T18:56:23.149738+0000","last_active":"2024-03-26T18:56:27.846898+0000","last_peered":"2024-03-26T18:56:27.846898+0000","last_clean":"2024-03-26T18:56:27.846898+0000","last_became_active":"2024-03-26T18:56:23.149399+0000","last_became_peered":"2024-03-26T18:56:23.149399+0000","last_unstale":"2024-03-26T18:56:27.846898+0000","last_undegraded":"2024-03-26T18:56:27.846898+0000","last_fullsized":"2024-03-26T18:56:27.846898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:44:50.666181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732595+0000","last_change":"2024-03-26T18:56:24.161505+0000","last_active":"2024-03-26T18:56:25.732595+0000","last_peered":"2024-03-26T18:56:25.732595+0000","last_clean":"2024-03-26T18:56:25.732595+0000","last_became_active":"2024-03-26T18:56:24.161035+0000","last_became_peered":"2024-03-26T18:56:24.161035+0000","last_unstale":"2024-03-26T18:56:25.732595+0000","last_undegraded":"2024-03-26T18:56:25.732595+0000","last_fullsized":"2024-03-26T18:56:25.732595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:22:30.062398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:29.828826+0000","last_change":"2024-03-26T18:56:23.150900+0000","last_active":"2024-03-26T18:56:29.828826+0000","last_peered":"2024-03-26T18:56:29.828826+0000","last_clean":"2024-03-26T18:56:29.828826+0000","last_became_active":"2024-03-26T18:56:23.150578+0000","last_became_peered":"2024-03-26T18:56:23.150578+0000","last_unstale":"2024-03-26T18:56:29.828826+0000","last_undegraded":"2024-03-26T18:56:29.828826+0000","last_fullsized":"2024-03-26T18:56:29.828826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:18:09.039791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732445+0000","last_change":"2024-03-26T18:56:24.148744+0000","last_active":"2024-03-26T18:56:25.732445+0000","last_peered":"2024-03-26T18:56:25.732445+0000","last_clean":"2024-03-26T18:56:25.732445+0000","last_became_active":"2024-03-26T18:56:24.147857+0000","last_became_peered":"2024-03-26T18:56:24.147857+0000","last_unstale":"2024-03-26T18:56:25.732445+0000","last_undegraded":"2024-03-26T18:56:25.732445+0000","last_fullsized":"2024-03-26T18:56:25.732445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:59:15.890399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732859+0000","last_change":"2024-03-26T18:56:23.165228+0000","last_active":"2024-03-26T18:56:25.732859+0000","last_peered":"2024-03-26T18:56:25.732859+0000","last_clean":"2024-03-26T18:56:25.732859+0000","last_became_active":"2024-03-26T18:56:23.164852+0000","last_became_peered":"2024-03-26T18:56:23.164852+0000","last_unstale":"2024-03-26T18:56:25.732859+0000","last_undegraded":"2024-03-26T18:56:25.732859+0000","last_fullsized":"2024-03-26T18:56:25.732859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:16:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732315+0000","last_change":"2024-03-26T18:56:24.154235+0000","last_active":"2024-03-26T18:56:25.732315+0000","last_peered":"2024-03-26T18:56:25.732315+0000","last_clean":"2024-03-26T18:56:25.732315+0000","last_became_active":"2024-03-26T18:56:24.153864+0000","last_became_peered":"2024-03-26T18:56:24.153864+0000","last_unstale":"2024-03-26T18:56:25.732315+0000","last_undegraded":"2024-03-26T18:56:25.732315+0000","last_fullsized":"2024-03-26T18:56:25.732315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:30:36.577695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.734109+0000","last_change":"2024-03-26T18:56:23.175807+0000","last_active":"2024-03-26T18:56:25.734109+0000","last_peered":"2024-03-26T18:56:25.734109+0000","last_clean":"2024-03-26T18:56:25.734109+0000","last_became_active":"2024-03-26T18:56:23.175477+0000","last_became_peered":"2024-03-26T18:56:23.175477+0000","last_unstale":"2024-03-26T18:56:25.734109+0000","last_undegraded":"2024-03-26T18:56:25.734109+0000","last_fullsized":"2024-03-26T18:56:25.734109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:45:28.494823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732294+0000","last_change":"2024-03-26T18:56:24.145201+0000","last_active":"2024-03-26T18:56:25.732294+0000","last_peered":"2024-03-26T18:56:25.732294+0000","last_clean":"2024-03-26T18:56:25.732294+0000","last_became_active":"2024-03-26T18:56:24.144793+0000","last_became_peered":"2024-03-26T18:56:24.144793+0000","last_unstale":"2024-03-26T18:56:25.732294+0000","last_undegraded":"2024-03-26T18:56:25.732294+0000","last_fullsized":"2024-03-26T18:56:25.732294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:13:55.476075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884555+0000","last_change":"2024-03-26T18:56:24.145445+0000","last_active":"2024-03-26T18:56:25.884555+0000","last_peered":"2024-03-26T18:56:25.884555+0000","last_clean":"2024-03-26T18:56:25.884555+0000","last_became_active":"2024-03-26T18:56:24.145115+0000","last_became_peered":"2024-03-26T18:56:24.145115+0000","last_unstale":"2024-03-26T18:56:25.884555+0000","last_undegraded":"2024-03-26T18:56:25.884555+0000","last_fullsized":"2024-03-26T18:56:25.884555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:30:45.728966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.129873+0000","last_change":"2024-03-26T18:56:24.148307+0000","last_active":"2024-03-26T18:56:25.129873+0000","last_peered":"2024-03-26T18:56:25.129873+0000","last_clean":"2024-03-26T18:56:25.129873+0000","last_became_active":"2024-03-26T18:56:24.147973+0000","last_became_peered":"2024-03-26T18:56:24.147973+0000","last_unstale":"2024-03-26T18:56:25.129873+0000","last_undegraded":"2024-03-26T18:56:25.129873+0000","last_fullsized":"2024-03-26T18:56:25.129873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:36:23.330139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.129890+0000","last_change":"2024-03-26T18:56:24.175594+0000","last_active":"2024-03-26T18:56:25.129890+0000","last_peered":"2024-03-26T18:56:25.129890+0000","last_clean":"2024-03-26T18:56:25.129890+0000","last_became_active":"2024-03-26T18:56:24.175261+0000","last_became_peered":"2024-03-26T18:56:24.175261+0000","last_unstale":"2024-03-26T18:56:25.129890+0000","last_undegraded":"2024-03-26T18:56:25.129890+0000","last_fullsized":"2024-03-26T18:56:25.129890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:12:02.744475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.215936+0000","last_change":"2024-03-26T18:56:24.177648+0000","last_active":"2024-03-26T18:56:25.215936+0000","last_peered":"2024-03-26T18:56:25.215936+0000","last_clean":"2024-03-26T18:56:25.215936+0000","last_became_active":"2024-03-26T18:56:24.176768+0000","last_became_peered":"2024-03-26T18:56:24.176768+0000","last_unstale":"2024-03-26T18:56:25.215936+0000","last_undegraded":"2024-03-26T18:56:25.215936+0000","last_fullsized":"2024-03-26T18:56:25.215936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:24:10.207407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.733960+0000","last_change":"2024-03-26T18:56:24.166746+0000","last_active":"2024-03-26T18:56:25.733960+0000","last_peered":"2024-03-26T18:56:25.733960+0000","last_clean":"2024-03-26T18:56:25.733960+0000","last_became_active":"2024-03-26T18:56:24.166220+0000","last_became_peered":"2024-03-26T18:56:24.166220+0000","last_unstale":"2024-03-26T18:56:25.733960+0000","last_undegraded":"2024-03-26T18:56:25.733960+0000","last_fullsized":"2024-03-26T18:56:25.733960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:16:54.411740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884619+0000","last_change":"2024-03-26T18:56:24.145511+0000","last_active":"2024-03-26T18:56:25.884619+0000","last_peered":"2024-03-26T18:56:25.884619+0000","last_clean":"2024-03-26T18:56:25.884619+0000","last_became_active":"2024-03-26T18:56:24.145164+0000","last_became_peered":"2024-03-26T18:56:24.145164+0000","last_unstale":"2024-03-26T18:56:25.884619+0000","last_undegraded":"2024-03-26T18:56:25.884619+0000","last_fullsized":"2024-03-26T18:56:25.884619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:25:05.239952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.130258+0000","last_change":"2024-03-26T18:56:24.145367+0000","last_active":"2024-03-26T18:56:25.130258+0000","last_peered":"2024-03-26T18:56:25.130258+0000","last_clean":"2024-03-26T18:56:25.130258+0000","last_became_active":"2024-03-26T18:56:24.144058+0000","last_became_peered":"2024-03-26T18:56:24.144058+0000","last_unstale":"2024-03-26T18:56:25.130258+0000","last_undegraded":"2024-03-26T18:56:25.130258+0000","last_fullsized":"2024-03-26T18:56:25.130258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:10.440142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.551458+0000","last_change":"2024-03-26T18:56:24.174209+0000","last_active":"2024-03-26T18:56:25.551458+0000","last_peered":"2024-03-26T18:56:25.551458+0000","last_clean":"2024-03-26T18:56:25.551458+0000","last_became_active":"2024-03-26T18:56:24.173873+0000","last_became_peered":"2024-03-26T18:56:24.173873+0000","last_unstale":"2024-03-26T18:56:25.551458+0000","last_undegraded":"2024-03-26T18:56:25.551458+0000","last_fullsized":"2024-03-26T18:56:25.551458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:21:34.317699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.732643+0000","last_change":"2024-03-26T18:56:24.166633+0000","last_active":"2024-03-26T18:56:25.732643+0000","last_peered":"2024-03-26T18:56:25.732643+0000","last_clean":"2024-03-26T18:56:25.732643+0000","last_became_active":"2024-03-26T18:56:24.166372+0000","last_became_peered":"2024-03-26T18:56:24.166372+0000","last_unstale":"2024-03-26T18:56:25.732643+0000","last_undegraded":"2024-03-26T18:56:25.732643+0000","last_fullsized":"2024-03-26T18:56:25.732643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:38:18.005223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.550635+0000","last_change":"2024-03-26T18:56:24.179979+0000","last_active":"2024-03-26T18:56:25.550635+0000","last_peered":"2024-03-26T18:56:25.550635+0000","last_clean":"2024-03-26T18:56:25.550635+0000","last_became_active":"2024-03-26T18:56:24.179533+0000","last_became_peered":"2024-03-26T18:56:24.179533+0000","last_unstale":"2024-03-26T18:56:25.550635+0000","last_undegraded":"2024-03-26T18:56:25.550635+0000","last_fullsized":"2024-03-26T18:56:25.550635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:19:26.104430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.127635+0000","last_change":"2024-03-26T18:56:24.175802+0000","last_active":"2024-03-26T18:56:25.127635+0000","last_peered":"2024-03-26T18:56:25.127635+0000","last_clean":"2024-03-26T18:56:25.127635+0000","last_became_active":"2024-03-26T18:56:24.175451+0000","last_became_peered":"2024-03-26T18:56:24.175451+0000","last_unstale":"2024-03-26T18:56:25.127635+0000","last_undegraded":"2024-03-26T18:56:25.127635+0000","last_fullsized":"2024-03-26T18:56:25.127635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:48:07.444891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.734012+0000","last_change":"2024-03-26T18:56:24.167708+0000","last_active":"2024-03-26T18:56:25.734012+0000","last_peered":"2024-03-26T18:56:25.734012+0000","last_clean":"2024-03-26T18:56:25.734012+0000","last_became_active":"2024-03-26T18:56:24.166985+0000","last_became_peered":"2024-03-26T18:56:24.166985+0000","last_unstale":"2024-03-26T18:56:25.734012+0000","last_undegraded":"2024-03-26T18:56:25.734012+0000","last_fullsized":"2024-03-26T18:56:25.734012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:13:35.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T18:56:25.884455+0000","last_change":"2024-03-26T18:56:24.151474+0000","last_active":"2024-03-26T18:56:25.884455+0000","last_peered":"2024-03-26T18:56:25.884455+0000","last_clean":"2024-03-26T18:56:25.884455+0000","last_became_active":"2024-03-26T18:56:24.151151+0000","last_became_peered":"2024-03-26T18:56:24.151151+0000","last_unstale":"2024-03-26T18:56:25.884455+0000","last_undegraded":"2024-03-26T18:56:25.884455+0000","last_fullsized":"2024-03-26T18:56:25.884455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:34.926612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280636,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5222,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":123,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":315392,"data_stored":63288,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":439305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":1030093,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":479901,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":448705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":921600,"data_stored":438625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":41380,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T18:56:34.295 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-26T18:56:34.296 INFO:tasks.cephadm:Setting up client nodes... 2024-03-26T18:56:34.296 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-26T18:56:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:35.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:34 smithi097 ceph-mon[31955]: from='client.24706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:56:35.125 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:34 smithi144 ceph-mon[36053]: from='client.24706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:56:35.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:34 smithi105 ceph-mon[36270]: from='client.24706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:56:36.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:35 smithi097 ceph-mon[31955]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:56:36.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:35 smithi097 ceph-mon[31955]: pgmap v318: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6.6 KiB/s wr, 25 op/s 2024-03-26T18:56:36.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:35 smithi144 ceph-mon[36053]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:56:36.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:35 smithi144 ceph-mon[36053]: pgmap v318: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6.6 KiB/s wr, 25 op/s 2024-03-26T18:56:36.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:35 smithi105 ceph-mon[36270]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:56:36.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:35 smithi105 ceph-mon[36270]: pgmap v318: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6.6 KiB/s wr, 25 op/s 2024-03-26T18:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.186+0000 7f6443000700 1 -- 172.21.15.97:0/2661149698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c103fb0 msgr2=0x7f643c10a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.186+0000 7f6443000700 1 --2- 172.21.15.97:0/2661149698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c103fb0 0x7f643c10a860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f643400b0a0 tx=0x7f643400b3b0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.186+0000 7f6443000700 1 -- 172.21.15.97:0/2661149698 shutdown_connections 2024-03-26T18:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.186+0000 7f6443000700 1 --2- 172.21.15.97:0/2661149698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c103fb0 0x7f643c10a860 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.186+0000 7f6443000700 1 --2- 172.21.15.97:0/2661149698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f643c102410 0x7f643c102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.186+0000 7f6443000700 1 --2- 172.21.15.97:0/2661149698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f643c103610 0x7f643c103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.186+0000 7f6443000700 1 -- 172.21.15.97:0/2661149698 >> 172.21.15.97:0/2661149698 conn(0x7f643c0fd9b0 msgr2=0x7f643c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.187+0000 7f6443000700 1 -- 172.21.15.97:0/2661149698 shutdown_connections 2024-03-26T18:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.187+0000 7f6443000700 1 -- 172.21.15.97:0/2661149698 wait complete. 2024-03-26T18:56:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.188+0000 7f6443000700 1 Processor -- start 2024-03-26T18:56:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.188+0000 7f6443000700 1 -- start start 2024-03-26T18:56:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6443000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f643c102410 0x7f643c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6443000700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c103610 0x7f643c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6443000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f643c103fb0 0x7f643c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6443000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10e050 con 0x7f643c102410 2024-03-26T18:56:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6443000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10ded0 con 0x7f643c103fb0 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6443000700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10e1d0 con 0x7f643c103610 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6441000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f643c103fb0 0x7f643c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6441000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f643c103fb0 0x7f643c1a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46628/0 (socket says 172.21.15.97:46628) 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6441000700 1 -- 172.21.15.97:0/1358123500 learned_addr learned my addr 172.21.15.97:0/1358123500 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f643b400700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c103610 0x7f643c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f643be00700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f643c102410 0x7f643c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6441000700 1 -- 172.21.15.97:0/1358123500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c103610 msgr2=0x7f643c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.189+0000 7f6441000700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c103610 0x7f643c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.190+0000 7f6441000700 1 -- 172.21.15.97:0/1358123500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f643c102410 msgr2=0x7f643c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.190+0000 7f6441000700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f643c102410 0x7f643c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.190+0000 7f6441000700 1 -- 172.21.15.97:0/1358123500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c1a2360 con 0x7f643c103fb0 2024-03-26T18:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.190+0000 7f643b400700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c103610 0x7f643c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.190+0000 7f6441000700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f643c103fb0 0x7f643c1a1cc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6434006070 tx=0x7f64340094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.190+0000 7f6438c00700 1 -- 172.21.15.97:0/1358123500 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434021070 con 0x7f643c103fb0 2024-03-26T18:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.190+0000 7f6438c00700 1 -- 172.21.15.97:0/1358123500 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6434007c00 con 0x7f643c103fb0 2024-03-26T18:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.190+0000 7f6438c00700 1 -- 172.21.15.97:0/1358123500 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643401d450 con 0x7f643c103fb0 2024-03-26T18:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.190+0000 7f6443000700 1 -- 172.21.15.97:0/1358123500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c1a25f0 con 0x7f643c103fb0 2024-03-26T18:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.191+0000 7f6443000700 1 -- 172.21.15.97:0/1358123500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c1a2a50 con 0x7f643c103fb0 2024-03-26T18:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.192+0000 7f6443000700 1 -- 172.21.15.97:0/1358123500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c0508c0 con 0x7f643c103fb0 2024-03-26T18:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.194+0000 7f6438c00700 1 -- 172.21.15.97:0/1358123500 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f64340040b0 con 0x7f643c103fb0 2024-03-26T18:56:37.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.196+0000 7f6438c00700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64200a3170 0x7f64200a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:37.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.196+0000 7f6438c00700 1 -- 172.21.15.97:0/1358123500 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f64340d0610 con 0x7f643c103fb0 2024-03-26T18:56:37.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.197+0000 7f643be00700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64200a3170 0x7f64200a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.197+0000 7f643be00700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64200a3170 0x7f64200a5630 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f643c103470 tx=0x7f642c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.199+0000 7f6438c00700 1 -- 172.21.15.97:0/1358123500 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f643409d420 con 0x7f643c103fb0 2024-03-26T18:56:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.557+0000 7f6443000700 1 -- 172.21.15.97:0/1358123500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f643c1a2e10 con 0x7f643c103fb0 2024-03-26T18:56:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.563+0000 7f6438c00700 1 -- 172.21.15.97:0/1358123500 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f643409cd20 con 0x7f643c103fb0 2024-03-26T18:56:37.557 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-03-26T18:56:37.557 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBlGgNm+hFjIRAA6LC/bZLHVS2wPwiq10pvIw== 2024-03-26T18:56:37.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.566+0000 7f6443000700 1 -- 172.21.15.97:0/1358123500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64200a3170 msgr2=0x7f64200a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:37.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.566+0000 7f6443000700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64200a3170 0x7f64200a5630 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f643c103470 tx=0x7f642c009260 comp rx=0 tx=0).stop 2024-03-26T18:56:37.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.566+0000 7f6443000700 1 -- 172.21.15.97:0/1358123500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f643c103fb0 msgr2=0x7f643c1a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:37.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.566+0000 7f6443000700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f643c103fb0 0x7f643c1a1cc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6434006070 tx=0x7f64340094f0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.567+0000 7f6443000700 1 -- 172.21.15.97:0/1358123500 shutdown_connections 2024-03-26T18:56:37.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.567+0000 7f6443000700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f643c103fb0 0x7f643c1a1cc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.567+0000 7f6443000700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f643c102410 0x7f643c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.567+0000 7f6443000700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f64200a3170 0x7f64200a5630 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.567+0000 7f6443000700 1 --2- 172.21.15.97:0/1358123500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c103610 0x7f643c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:37.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.567+0000 7f6443000700 1 -- 172.21.15.97:0/1358123500 >> 172.21.15.97:0/1358123500 conn(0x7f643c0fd9b0 msgr2=0x7f643c0745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:37.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.567+0000 7f6443000700 1 -- 172.21.15.97:0/1358123500 shutdown_connections 2024-03-26T18:56:37.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:37.567+0000 7f6443000700 1 -- 172.21.15.97:0/1358123500 wait complete. 2024-03-26T18:56:38.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:37 smithi105 ceph-mon[36270]: pgmap v319: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-03-26T18:56:38.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:37 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1358123500' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:56:38.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:37 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:56:38.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:37 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T18:56:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:37 smithi097 ceph-mon[31955]: pgmap v319: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-03-26T18:56:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:37 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1358123500' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:56:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:37 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:56:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:37 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T18:56:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:37 smithi144 ceph-mon[36053]: pgmap v319: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-03-26T18:56:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:37 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1358123500' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:56:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:37 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:56:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:37 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T18:56:38.529 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:56:38.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-26T18:56:38.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-26T18:56:38.664 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-26T18:56:38.664 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-26T18:56:38.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-26T18:56:39.190 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:40.241 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:39 smithi097 ceph-mon[31955]: pgmap v320: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.5 KiB/s wr, 25 op/s 2024-03-26T18:56:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:39 smithi144 ceph-mon[36053]: pgmap v320: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.5 KiB/s wr, 25 op/s 2024-03-26T18:56:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:39 smithi105 ceph-mon[36270]: pgmap v320: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.5 KiB/s wr, 25 op/s 2024-03-26T18:56:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.611+0000 7ff334e00700 1 -- 172.21.15.97:0/3894444746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3300745e0 msgr2=0x7ff3300749c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.611+0000 7ff334e00700 1 --2- 172.21.15.97:0/3894444746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3300745e0 0x7ff3300749c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff320009990 tx=0x7ff320009ca0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.612+0000 7ff334e00700 1 -- 172.21.15.97:0/3894444746 shutdown_connections 2024-03-26T18:56:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.612+0000 7ff334e00700 1 --2- 172.21.15.97:0/3894444746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff33010ede0 0x7ff3301111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.612+0000 7ff334e00700 1 --2- 172.21.15.97:0/3894444746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3300745e0 0x7ff3300749c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.612+0000 7ff334e00700 1 --2- 172.21.15.97:0/3894444746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff330074f00 0x7ff33010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.612+0000 7ff334e00700 1 -- 172.21.15.97:0/3894444746 >> 172.21.15.97:0/3894444746 conn(0x7ff330079470 msgr2=0x7ff330079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.612+0000 7ff334e00700 1 -- 172.21.15.97:0/3894444746 shutdown_connections 2024-03-26T18:56:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.612+0000 7ff334e00700 1 -- 172.21.15.97:0/3894444746 wait complete. 2024-03-26T18:56:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.613+0000 7ff334e00700 1 Processor -- start 2024-03-26T18:56:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.613+0000 7ff334e00700 1 -- start start 2024-03-26T18:56:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.613+0000 7ff334e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3300745e0 0x7ff33019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff334e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff330074f00 0x7ff33019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff334e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33010ede0 0x7ff33019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff334e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff3301148f0 con 0x7ff33010ede0 2024-03-26T18:56:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff334e00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff330114770 con 0x7ff330074f00 2024-03-26T18:56:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff334e00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff330114a70 con 0x7ff3300745e0 2024-03-26T18:56:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff32de00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3300745e0 0x7ff33019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff32de00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3300745e0 0x7ff33019ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52094/0 (socket says 172.21.15.97:52094) 2024-03-26T18:56:41.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff32de00700 1 -- 172.21.15.97:0/3207502683 learned_addr learned my addr 172.21.15.97:0/3207502683 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:41.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff32d400700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff330074f00 0x7ff33019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:41.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff32e800700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33010ede0 0x7ff33019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:41.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff32de00700 1 -- 172.21.15.97:0/3207502683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff330074f00 msgr2=0x7ff33019b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:41.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff32de00700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff330074f00 0x7ff33019b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff32de00700 1 -- 172.21.15.97:0/3207502683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33010ede0 msgr2=0x7ff33019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:41.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff32de00700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33010ede0 0x7ff33019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.614+0000 7ff32de00700 1 -- 172.21.15.97:0/3207502683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33019fd50 con 0x7ff3300745e0 2024-03-26T18:56:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.615+0000 7ff32de00700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3300745e0 0x7ff33019ad40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff320000c00 tx=0x7ff32000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.615+0000 7ff326a00700 1 -- 172.21.15.97:0/3207502683 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320027070 con 0x7ff3300745e0 2024-03-26T18:56:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.615+0000 7ff334e00700 1 -- 172.21.15.97:0/3207502683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33019ffe0 con 0x7ff3300745e0 2024-03-26T18:56:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.615+0000 7ff326a00700 1 -- 172.21.15.97:0/3207502683 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff320003d70 con 0x7ff3300745e0 2024-03-26T18:56:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.615+0000 7ff326a00700 1 -- 172.21.15.97:0/3207502683 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32000f460 con 0x7ff3300745e0 2024-03-26T18:56:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.615+0000 7ff334e00700 1 -- 172.21.15.97:0/3207502683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3301a78c0 con 0x7ff3300745e0 2024-03-26T18:56:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.616+0000 7ff334e00700 1 -- 172.21.15.97:0/3207502683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff330069070 con 0x7ff3300745e0 2024-03-26T18:56:41.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.620+0000 7ff326a00700 1 -- 172.21.15.97:0/3207502683 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7ff32002a070 con 0x7ff3300745e0 2024-03-26T18:56:41.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.622+0000 7ff326a00700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff3140a3040 0x7ff3140a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:41.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.622+0000 7ff326a00700 1 -- 172.21.15.97:0/3207502683 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff3200d0210 con 0x7ff3300745e0 2024-03-26T18:56:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.622+0000 7ff32d400700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff3140a3040 0x7ff3140a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.623+0000 7ff32d400700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff3140a3040 0x7ff3140a5500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff330110a70 tx=0x7ff31c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:41.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.625+0000 7ff326a00700 1 -- 172.21.15.97:0/3207502683 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff320013070 con 0x7ff3300745e0 2024-03-26T18:56:41.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.916+0000 7ff334e00700 1 -- 172.21.15.97:0/3207502683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff33019c0a0 con 0x7ff3300745e0 2024-03-26T18:56:41.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.921+0000 7ff326a00700 1 -- 172.21.15.97:0/3207502683 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263551 (secure 0 0 0) 0x7ff32009d020 con 0x7ff3300745e0 2024-03-26T18:56:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:41.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 -- 172.21.15.97:0/3207502683 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff3140a3040 msgr2=0x7ff3140a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:41.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff3140a3040 0x7ff3140a5500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff330110a70 tx=0x7ff31c009260 comp rx=0 tx=0).stop 2024-03-26T18:56:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 -- 172.21.15.97:0/3207502683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3300745e0 msgr2=0x7ff33019ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3300745e0 0x7ff33019ad40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff320000c00 tx=0x7ff32000bab0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 -- 172.21.15.97:0/3207502683 shutdown_connections 2024-03-26T18:56:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff33010ede0 0x7ff33019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff3140a3040 0x7ff3140a5500 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3300745e0 0x7ff33019ad40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 --2- 172.21.15.97:0/3207502683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff330074f00 0x7ff33019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:41.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 -- 172.21.15.97:0/3207502683 >> 172.21.15.97:0/3207502683 conn(0x7ff330079470 msgr2=0x7ff3300ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:41.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 -- 172.21.15.97:0/3207502683 shutdown_connections 2024-03-26T18:56:41.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:41.927+0000 7ff334e00700 1 -- 172.21.15.97:0/3207502683 wait complete. 2024-03-26T18:56:42.333 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:41 smithi097 ceph-mon[31955]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 27 op/s 2024-03-26T18:56:42.333 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:41 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3207502683' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T18:56:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:41 smithi144 ceph-mon[36053]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 27 op/s 2024-03-26T18:56:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:41 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3207502683' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T18:56:42.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:41 smithi105 ceph-mon[36270]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 27 op/s 2024-03-26T18:56:42.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:41 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3207502683' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T18:56:42.883 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3680188982},{"type":"v1","addr":"172.21.15.97:6801","nonce":3680188982}]},"active_addr":"172.21.15.97:6801/3680188982","active_change":"2024-03-26T18:48:28.768673+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.97:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2642516573}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":302710042}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1788391090}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2675612236}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1935163212}]}]} 2024-03-26T18:56:42.890 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-26T18:56:42.890 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-26T18:56:42.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T18:56:43.395 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:44.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:43 smithi097 ceph-mon[31955]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 19 op/s 2024-03-26T18:56:44.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:43 smithi144 ceph-mon[36053]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 19 op/s 2024-03-26T18:56:44.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:43 smithi105 ceph-mon[36270]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 19 op/s 2024-03-26T18:56:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.647+0000 7fe9dd200700 1 -- 172.21.15.97:0/3030747615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d8102dc0 msgr2=0x7fe9d810f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.647+0000 7fe9dd200700 1 --2- 172.21.15.97:0/3030747615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d8102dc0 0x7fe9d810f2c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000b0a0 tx=0x7fe9d000b3b0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.647+0000 7fe9dd200700 1 -- 172.21.15.97:0/3030747615 shutdown_connections 2024-03-26T18:56:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.647+0000 7fe9dd200700 1 --2- 172.21.15.97:0/3030747615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d8102dc0 0x7fe9d810f2c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.647+0000 7fe9dd200700 1 --2- 172.21.15.97:0/3030747615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d8108480 0x7fe9d8108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.647+0000 7fe9dd200700 1 --2- 172.21.15.97:0/3030747615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d8102400 0x7fe9d8102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.647+0000 7fe9dd200700 1 -- 172.21.15.97:0/3030747615 >> 172.21.15.97:0/3030747615 conn(0x7fe9d8079510 msgr2=0x7fe9d8079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.647+0000 7fe9dd200700 1 -- 172.21.15.97:0/3030747615 shutdown_connections 2024-03-26T18:56:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.647+0000 7fe9dd200700 1 -- 172.21.15.97:0/3030747615 wait complete. 2024-03-26T18:56:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.649+0000 7fe9dd200700 1 Processor -- start 2024-03-26T18:56:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.649+0000 7fe9dd200700 1 -- start start 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9dd200700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d8102400 0x7fe9d819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9dd200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d8102dc0 0x7fe9d819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9dd200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d8108480 0x7fe9d81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9dd200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d8112740 con 0x7fe9d8108480 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9dd200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81125c0 con 0x7fe9d8102dc0 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9dd200700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81128c0 con 0x7fe9d8102400 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9d5e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d8102400 0x7fe9d819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9d5400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d8102dc0 0x7fe9d819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9d5e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d8102400 0x7fe9d819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52110/0 (socket says 172.21.15.97:52110) 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9d5e00700 1 -- 172.21.15.97:0/594060619 learned_addr learned my addr 172.21.15.97:0/594060619 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9d6800700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d8108480 0x7fe9d81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:45.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9d5e00700 1 -- 172.21.15.97:0/594060619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d8102dc0 msgr2=0x7fe9d819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:45.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9d5e00700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d8102dc0 0x7fe9d819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9d5e00700 1 -- 172.21.15.97:0/594060619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d8108480 msgr2=0x7fe9d81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:45.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9d5e00700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d8108480 0x7fe9d81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.650+0000 7fe9d5e00700 1 -- 172.21.15.97:0/594060619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d8199120 con 0x7fe9d8102400 2024-03-26T18:56:45.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.651+0000 7fe9d5e00700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d8102400 0x7fe9d819e800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800e9a0 tx=0x7fe9c800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:45.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.651+0000 7fe9cea00700 1 -- 172.21.15.97:0/594060619 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800cbd0 con 0x7fe9d8102400 2024-03-26T18:56:45.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.651+0000 7fe9cea00700 1 -- 172.21.15.97:0/594060619 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c800cd70 con 0x7fe9d8102400 2024-03-26T18:56:45.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.651+0000 7fe9cea00700 1 -- 172.21.15.97:0/594060619 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8018750 con 0x7fe9d8102400 2024-03-26T18:56:45.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.651+0000 7fe9dd200700 1 -- 172.21.15.97:0/594060619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d8199410 con 0x7fe9d8102400 2024-03-26T18:56:45.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.651+0000 7fe9dd200700 1 -- 172.21.15.97:0/594060619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d80745e0 con 0x7fe9d8102400 2024-03-26T18:56:45.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.652+0000 7fe9dd200700 1 -- 172.21.15.97:0/594060619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d8069070 con 0x7fe9d8102400 2024-03-26T18:56:45.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.654+0000 7fe9cea00700 1 -- 172.21.15.97:0/594060619 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fe9c8010430 con 0x7fe9d8102400 2024-03-26T18:56:45.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.655+0000 7fe9cea00700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe9bc0a30c0 0x7fe9bc0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:45.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.655+0000 7fe9cea00700 1 -- 172.21.15.97:0/594060619 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe9c80cf6b0 con 0x7fe9d8102400 2024-03-26T18:56:45.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.656+0000 7fe9d5400700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe9bc0a30c0 0x7fe9bc0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:45.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.656+0000 7fe9d5400700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe9bc0a30c0 0x7fe9bc0a5580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8079aa0 tx=0x7fe9c4012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:45.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.659+0000 7fe9cea00700 1 -- 172.21.15.97:0/594060619 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe9c809c430 con 0x7fe9d8102400 2024-03-26T18:56:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.904+0000 7fe9dd200700 1 -- 172.21.15.97:0/594060619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9d802dc00 con 0x7fe9d8102400 2024-03-26T18:56:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.906+0000 7fe9cea00700 1 -- 172.21.15.97:0/594060619 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22194 (secure 0 0 0) 0x7fe9c809bd30 con 0x7fe9d8102400 2024-03-26T18:56:45.900 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:45.900 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:56:25.123057+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:56:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 -- 172.21.15.97:0/594060619 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe9bc0a30c0 msgr2=0x7fe9bc0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe9bc0a30c0 0x7fe9bc0a5580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8079aa0 tx=0x7fe9c4012040 comp rx=0 tx=0).stop 2024-03-26T18:56:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 -- 172.21.15.97:0/594060619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d8102400 msgr2=0x7fe9d819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d8102400 0x7fe9d819e800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800e9a0 tx=0x7fe9c800ecb0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 -- 172.21.15.97:0/594060619 shutdown_connections 2024-03-26T18:56:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d8108480 0x7fe9d81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe9bc0a30c0 0x7fe9bc0a5580 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d8102400 0x7fe9d819e800 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 --2- 172.21.15.97:0/594060619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9d8102dc0 0x7fe9d819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 -- 172.21.15.97:0/594060619 >> 172.21.15.97:0/594060619 conn(0x7fe9d8079510 msgr2=0x7fe9d8100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 -- 172.21.15.97:0/594060619 shutdown_connections 2024-03-26T18:56:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:45.909+0000 7fe9dd200700 1 -- 172.21.15.97:0/594060619 wait complete. 2024-03-26T18:56:46.264 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:45 smithi097 ceph-mon[31955]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-03-26T18:56:46.264 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:45 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/594060619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:46.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:45 smithi144 ceph-mon[36053]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-03-26T18:56:46.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:45 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/594060619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:45 smithi105 ceph-mon[36270]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-03-26T18:56:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:45 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/594060619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:46.832 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-26T18:56:46.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T18:56:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:48.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:47 smithi097 ceph-mon[31955]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-03-26T18:56:48.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:47 smithi144 ceph-mon[36053]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-03-26T18:56:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:47 smithi105 ceph-mon[36270]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-03-26T18:56:49.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.599+0000 7f94e9e00700 1 -- 172.21.15.97:0/53338001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4102fa0 msgr2=0x7f94e4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.599+0000 7f94e9e00700 1 --2- 172.21.15.97:0/53338001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4102fa0 0x7f94e4106ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009990 tx=0x7f94d0009ca0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.599+0000 7f94e9e00700 1 -- 172.21.15.97:0/53338001 shutdown_connections 2024-03-26T18:56:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.599+0000 7f94e9e00700 1 --2- 172.21.15.97:0/53338001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94e41076d0 0x7f94e410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.599+0000 7f94e9e00700 1 --2- 172.21.15.97:0/53338001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e41025f0 0x7f94e41029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.599+0000 7f94e9e00700 1 --2- 172.21.15.97:0/53338001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4102fa0 0x7f94e4106ff0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.599+0000 7f94e9e00700 1 -- 172.21.15.97:0/53338001 >> 172.21.15.97:0/53338001 conn(0x7f94e40fde60 msgr2=0x7f94e4100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.599+0000 7f94e9e00700 1 -- 172.21.15.97:0/53338001 shutdown_connections 2024-03-26T18:56:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.600+0000 7f94e9e00700 1 -- 172.21.15.97:0/53338001 wait complete. 2024-03-26T18:56:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.601+0000 7f94e9e00700 1 Processor -- start 2024-03-26T18:56:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.601+0000 7f94e9e00700 1 -- start start 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.602+0000 7f94e9e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94e41025f0 0x7f94e419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.602+0000 7f94e9e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4102fa0 0x7f94e419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.602+0000 7f94e9e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e41076d0 0x7f94e419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.602+0000 7f94e9e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41126e0 con 0x7f94e41025f0 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.602+0000 7f94e9e00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4112560 con 0x7f94e4102fa0 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.602+0000 7f94e9e00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4112860 con 0x7f94e41076d0 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.602+0000 7f94e2000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4102fa0 0x7f94e419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.602+0000 7f94e2000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4102fa0 0x7f94e419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:37432/0 (socket says 172.21.15.97:37432) 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.602+0000 7f94e2000700 1 -- 172.21.15.97:0/4026082554 learned_addr learned my addr 172.21.15.97:0/4026082554 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.602+0000 7f94e2a00700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94e41025f0 0x7f94e419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.603+0000 7f94e2000700 1 -- 172.21.15.97:0/4026082554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e41076d0 msgr2=0x7f94e419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.603+0000 7f94e3400700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e41076d0 0x7f94e419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.603+0000 7f94e2000700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e41076d0 0x7f94e419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.603+0000 7f94e2000700 1 -- 172.21.15.97:0/4026082554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94e41025f0 msgr2=0x7f94e419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.603+0000 7f94e2000700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94e41025f0 0x7f94e419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.603+0000 7f94e2000700 1 -- 172.21.15.97:0/4026082554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e419fe40 con 0x7f94e4102fa0 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.603+0000 7f94e3400700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e41076d0 0x7f94e419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.603+0000 7f94e2000700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4102fa0 0x7f94e419b370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94d0000c00 tx=0x7f94d00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.604+0000 7f94e8a00700 1 -- 172.21.15.97:0/4026082554 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0027070 con 0x7f94e4102fa0 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.604+0000 7f94e8a00700 1 -- 172.21.15.97:0/4026082554 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d0022470 con 0x7f94e4102fa0 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.604+0000 7f94e8a00700 1 -- 172.21.15.97:0/4026082554 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0017650 con 0x7f94e4102fa0 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.604+0000 7f94e9e00700 1 -- 172.21.15.97:0/4026082554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e41a00d0 con 0x7f94e4102fa0 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.604+0000 7f94e9e00700 1 -- 172.21.15.97:0/4026082554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e41a79b0 con 0x7f94e4102fa0 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.606+0000 7f94d9600700 1 -- 172.21.15.97:0/4026082554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94bc005330 con 0x7f94e4102fa0 2024-03-26T18:56:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.608+0000 7f94e8a00700 1 -- 172.21.15.97:0/4026082554 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f94d002a070 con 0x7f94e4102fa0 2024-03-26T18:56:49.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.610+0000 7f94e8a00700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f94c80a3190 0x7f94c80a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:49.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.611+0000 7f94e8a00700 1 -- 172.21.15.97:0/4026082554 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f94d00d0170 con 0x7f94e4102fa0 2024-03-26T18:56:49.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.611+0000 7f94e2a00700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f94c80a3190 0x7f94c80a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:49.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.612+0000 7f94e2a00700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f94c80a3190 0x7f94c80a5650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009c60 tx=0x7f94d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:49.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.613+0000 7f94e8a00700 1 -- 172.21.15.97:0/4026082554 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f94d0013070 con 0x7f94e4102fa0 2024-03-26T18:56:49.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.869+0000 7f94d9600700 1 -- 172.21.15.97:0/4026082554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94bc0058f0 con 0x7f94e4102fa0 2024-03-26T18:56:49.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.872+0000 7f94e8a00700 1 -- 172.21.15.97:0/4026082554 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22194 (secure 0 0 0) 0x7f94d009d000 con 0x7f94e4102fa0 2024-03-26T18:56:49.865 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:56:49.865 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:56:25.123057+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:56:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.875+0000 7f94d9600700 1 -- 172.21.15.97:0/4026082554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f94c80a3190 msgr2=0x7f94c80a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f94c80a3190 0x7f94c80a5650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009c60 tx=0x7f94d4009260 comp rx=0 tx=0).stop 2024-03-26T18:56:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 -- 172.21.15.97:0/4026082554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4102fa0 msgr2=0x7f94e419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4102fa0 0x7f94e419b370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94d0000c00 tx=0x7f94d00057c0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 -- 172.21.15.97:0/4026082554 shutdown_connections 2024-03-26T18:56:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f94c80a3190 0x7f94c80a5650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e41076d0 0x7f94e419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94e41025f0 0x7f94e419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 --2- 172.21.15.97:0/4026082554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4102fa0 0x7f94e419b370 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 -- 172.21.15.97:0/4026082554 >> 172.21.15.97:0/4026082554 conn(0x7f94e40fde60 msgr2=0x7f94e410da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 -- 172.21.15.97:0/4026082554 shutdown_connections 2024-03-26T18:56:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:49.876+0000 7f94d9600700 1 -- 172.21.15.97:0/4026082554 wait complete. 2024-03-26T18:56:50.262 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:49 smithi105 ceph-mon[36270]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-26T18:56:50.262 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:49 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/4026082554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:50.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:49 smithi097 ceph-mon[31955]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-26T18:56:50.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:49 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/4026082554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:50.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:49 smithi144 ceph-mon[36053]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-26T18:56:50.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:49 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/4026082554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:56:50.844 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-26T18:56:50.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-26T18:56:50.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-26T18:56:50.846 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-26T18:56:50.846 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-26T18:56:50.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-26T18:56:50.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-26T18:56:50.848 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-26T18:56:50.848 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-26T18:56:50.848 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-26T18:56:50.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-26T18:56:50.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-26T18:56:52.133 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:52.292 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:52.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:51 smithi097 ceph-mon[31955]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T18:56:52.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:51 smithi144 ceph-mon[36053]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T18:56:52.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:51 smithi105 ceph-mon[36270]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T18:56:53.789 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:53.803 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:53.840 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:53.846 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:56:54.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:53 smithi097 ceph-mon[31955]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-26T18:56:54.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:53 smithi144 ceph-mon[36053]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-26T18:56:54.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:53 smithi105 ceph-mon[36270]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-26T18:56:56.332 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:55 smithi097 ceph-mon[31955]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:56:56.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:55 smithi144 ceph-mon[36053]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:56:56.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:55 smithi105 ceph-mon[36270]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:56:58.242 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:56:58 smithi097 ceph-mon[31955]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:56:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:56:58 smithi144 ceph-mon[36053]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:56:58.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:56:58 smithi105 ceph-mon[36270]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:56:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.693+0000 7fd2ec000700 1 -- 172.21.15.97:0/1779466642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2e4102dc0 msgr2=0x7fd2e410f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.693+0000 7fd2ec000700 1 --2- 172.21.15.97:0/1779466642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2e4102dc0 0x7fd2e410f2c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009f90 tx=0x7fd2dc00b330 comp rx=0 tx=0).stop 2024-03-26T18:56:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.693+0000 7fd2ec000700 1 -- 172.21.15.97:0/1779466642 shutdown_connections 2024-03-26T18:56:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.693+0000 7fd2ec000700 1 --2- 172.21.15.97:0/1779466642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2e4102dc0 0x7fd2e410f2c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.693+0000 7fd2ec000700 1 --2- 172.21.15.97:0/1779466642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e4108480 0x7fd2e4108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.693+0000 7fd2ec000700 1 --2- 172.21.15.97:0/1779466642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4102400 0x7fd2e4102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.693+0000 7fd2ec000700 1 -- 172.21.15.97:0/1779466642 >> 172.21.15.97:0/1779466642 conn(0x7fd2e4079510 msgr2=0x7fd2e4079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.693+0000 7fd2ec000700 1 -- 172.21.15.97:0/1779466642 shutdown_connections 2024-03-26T18:56:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.693+0000 7fd2ec000700 1 -- 172.21.15.97:0/1779466642 wait complete. 2024-03-26T18:56:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.695+0000 7fd2ec000700 1 Processor -- start 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.695+0000 7fd2ec000700 1 -- start start 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.695+0000 7fd2ec000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4102400 0x7fd2e419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.695+0000 7fd2ec000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2e4102dc0 0x7fd2e419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.695+0000 7fd2ec000700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e4108480 0x7fd2e41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.695+0000 7fd2ec000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4112740 con 0x7fd2e4102400 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.695+0000 7fd2ec000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41125c0 con 0x7fd2e4102dc0 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.695+0000 7fd2ec000700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41128c0 con 0x7fd2e4108480 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2e8c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2e4102dc0 0x7fd2e419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2ea000700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e4108480 0x7fd2e41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2e9600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4102400 0x7fd2e419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2ea000700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e4108480 0x7fd2e41989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34262/0 (socket says 172.21.15.97:34262) 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2ea000700 1 -- 172.21.15.97:0/1846653532 learned_addr learned my addr 172.21.15.97:0/1846653532 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:56:59.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2e8c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2e4102dc0 0x7fd2e419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:41558/0 (socket says 172.21.15.97:41558) 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2ea000700 1 -- 172.21.15.97:0/1846653532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2e4102dc0 msgr2=0x7fd2e419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2ea000700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2e4102dc0 0x7fd2e419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2ea000700 1 -- 172.21.15.97:0/1846653532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4102400 msgr2=0x7fd2e419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2ea000700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4102400 0x7fd2e419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2ea000700 1 -- 172.21.15.97:0/1846653532 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e4199120 con 0x7fd2e4108480 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2e8c00700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2e4102dc0 0x7fd2e419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.696+0000 7fd2ea000700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e4108480 0x7fd2e41989c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009550 tx=0x7fd2dc00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.697+0000 7fd2e2000700 1 -- 172.21.15.97:0/1846653532 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc021070 con 0x7fd2e4108480 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.697+0000 7fd2e2000700 1 -- 172.21.15.97:0/1846653532 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc007ac0 con 0x7fd2e4108480 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.697+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e41993b0 con 0x7fd2e4108480 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.697+0000 7fd2e2000700 1 -- 172.21.15.97:0/1846653532 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc005040 con 0x7fd2e4108480 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.697+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e40745e0 con 0x7fd2e4108480 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.698+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2e4062ac0 con 0x7fd2e4108480 2024-03-26T18:56:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.701+0000 7fd2e2000700 1 -- 172.21.15.97:0/1846653532 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fd2dc024070 con 0x7fd2e4108480 2024-03-26T18:56:59.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.703+0000 7fd2e2000700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2cc0a30c0 0x7fd2cc0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:59.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.703+0000 7fd2e2000700 1 -- 172.21.15.97:0/1846653532 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd2dc00f020 con 0x7fd2e4108480 2024-03-26T18:56:59.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.703+0000 7fd2e9600700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2cc0a30c0 0x7fd2cc0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:59.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.703+0000 7fd2e2000700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] conn(0x7fd2cc0ac8a0 0x7fd2cc0aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:56:59.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.703+0000 7fd2e2000700 1 -- 172.21.15.97:0/1846653532 --> [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2dc00c850 con 0x7fd2cc0ac8a0 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.703+0000 7fd2e2000700 1 -- 172.21.15.97:0/1846653532 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd2dc0d7050 con 0x7fd2e4108480 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.703+0000 7fd2e8c00700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] conn(0x7fd2cc0ac8a0 0x7fd2cc0aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.704+0000 7fd2e9600700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2cc0a30c0 0x7fd2cc0a5580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009680 tx=0x7fd2d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.704+0000 7fd2e8c00700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] conn(0x7fd2cc0ac8a0 0x7fd2cc0aecc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.705+0000 7fd2e2000700 1 -- 172.21.15.97:0/1846653532 <== osd.2 v2:172.21.15.144:6800/837505140 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2dc00c850 con 0x7fd2cc0ac8a0 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.738+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 --> [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd2e4062ef0 con 0x7fd2cc0ac8a0 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.738+0000 7fd2e2000700 1 -- 172.21.15.97:0/1846653532 <== osd.2 v2:172.21.15.144:6800/837505140 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd2e4062ef0 con 0x7fd2cc0ac8a0 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 >> [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] conn(0x7fd2cc0ac8a0 msgr2=0x7fd2cc0aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ec000700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] conn(0x7fd2cc0ac8a0 0x7fd2cc0aecc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2cc0a30c0 msgr2=0x7fd2cc0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ec000700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2cc0a30c0 0x7fd2cc0a5580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009680 tx=0x7fd2d8009260 comp rx=0 tx=0).stop 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e4108480 msgr2=0x7fd2e41989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ec000700 1 --2- 172.21.15.97:0/1846653532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e4108480 0x7fd2e41989c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009550 tx=0x7fd2dc00cef0 comp rx=0 tx=0).stop 2024-03-26T18:56:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ea000700 1 -- 172.21.15.97:0/1846653532 reap_dead start 2024-03-26T18:56:59.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 shutdown_connections 2024-03-26T18:56:59.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 >> 172.21.15.97:0/1846653532 conn(0x7fd2e4079510 msgr2=0x7fd2e4100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:56:59.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 shutdown_connections 2024-03-26T18:56:59.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:56:57.739+0000 7fd2ec000700 1 -- 172.21.15.97:0/1846653532 wait complete. 2024-03-26T18:57:00.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:00 smithi097 ceph-mon[31955]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T18:57:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:00 smithi144 ceph-mon[36053]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T18:57:00.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:00 smithi105 ceph-mon[36270]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T18:57:00.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.876+0000 7f720a800700 1 -- 172.21.15.97:0/691050141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72040745e0 msgr2=0x7f72040749c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:00.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.876+0000 7f720a800700 1 --2- 172.21.15.97:0/691050141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72040745e0 0x7f72040749c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009990 tx=0x7f71f4009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:00.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.876+0000 7f720a800700 1 -- 172.21.15.97:0/691050141 shutdown_connections 2024-03-26T18:57:00.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.876+0000 7f720a800700 1 --2- 172.21.15.97:0/691050141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f720410edb0 0x7f72041111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:00.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.876+0000 7f720a800700 1 --2- 172.21.15.97:0/691050141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72040745e0 0x7f72040749c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:00.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.876+0000 7f720a800700 1 --2- 172.21.15.97:0/691050141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7204074f00 0x7f720410e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:00.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.876+0000 7f720a800700 1 -- 172.21.15.97:0/691050141 >> 172.21.15.97:0/691050141 conn(0x7f7204079510 msgr2=0x7f7204079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:00.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.876+0000 7f720a800700 1 -- 172.21.15.97:0/691050141 shutdown_connections 2024-03-26T18:57:00.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.876+0000 7f720a800700 1 -- 172.21.15.97:0/691050141 wait complete. 2024-03-26T18:57:00.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.878+0000 7f720a800700 1 Processor -- start 2024-03-26T18:57:00.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.878+0000 7f720a800700 1 -- start start 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.878+0000 7f720a800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72040745e0 0x7f7204073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.878+0000 7f720a800700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7204074f00 0x7f7204074000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f720a800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720410edb0 0x7f720406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f720a800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7204114990 con 0x7f720410edb0 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f720a800700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7204114810 con 0x7f72040745e0 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f720a800700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7204114b10 con 0x7f7204074f00 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f7203400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72040745e0 0x7f7204073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f7203400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72040745e0 0x7f7204073ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38176/0 (socket says 172.21.15.97:38176) 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f7203400700 1 -- 172.21.15.97:0/4004527938 learned_addr learned my addr 172.21.15.97:0/4004527938 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f71fbe00700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7204074f00 0x7f7204074000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:00.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f7203e00700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720410edb0 0x7f720406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:00.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f7203400700 1 -- 172.21.15.97:0/4004527938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7204074f00 msgr2=0x7f7204074000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:00.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f7203400700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7204074f00 0x7f7204074000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:00.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f7203400700 1 -- 172.21.15.97:0/4004527938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720410edb0 msgr2=0x7f720406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:00.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f7203400700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720410edb0 0x7f720406cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:00.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.879+0000 7f7203400700 1 -- 172.21.15.97:0/4004527938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720406d610 con 0x7f72040745e0 2024-03-26T18:57:00.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.880+0000 7f7203400700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72040745e0 0x7f7204073ac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f71f4005910 tx=0x7f71f400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:00.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.880+0000 7f7200c00700 1 -- 172.21.15.97:0/4004527938 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4027070 con 0x7f72040745e0 2024-03-26T18:57:00.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.880+0000 7f7200c00700 1 -- 172.21.15.97:0/4004527938 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f4003d70 con 0x7f72040745e0 2024-03-26T18:57:00.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.880+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720406d8a0 con 0x7f72040745e0 2024-03-26T18:57:00.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.880+0000 7f7200c00700 1 -- 172.21.15.97:0/4004527938 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f400f460 con 0x7f72040745e0 2024-03-26T18:57:00.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.880+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72040717c0 con 0x7f72040745e0 2024-03-26T18:57:00.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.881+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71e0001030 con 0x7f72040745e0 2024-03-26T18:57:00.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.883+0000 7f7200c00700 1 -- 172.21.15.97:0/4004527938 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f71f402a070 con 0x7f72040745e0 2024-03-26T18:57:00.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.885+0000 7f7200c00700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f71e80a30c0 0x7f71e80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:00.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.885+0000 7f7200c00700 1 -- 172.21.15.97:0/4004527938 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f71f40d0b40 con 0x7f72040745e0 2024-03-26T18:57:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.885+0000 7f71fbe00700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f71e80a30c0 0x7f71e80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.885+0000 7f7200c00700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] conn(0x7f71e80ac8a0 0x7f71e80aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.885+0000 7f7200c00700 1 -- 172.21.15.97:0/4004527938 --> [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71f40056d0 con 0x7f71e80ac8a0 2024-03-26T18:57:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.885+0000 7f7200c00700 1 -- 172.21.15.97:0/4004527938 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f71f40d0ef0 con 0x7f72040745e0 2024-03-26T18:57:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.886+0000 7f7203e00700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] conn(0x7f71e80ac8a0 0x7f71e80aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.886+0000 7f71fbe00700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f71e80a30c0 0x7f71e80a5580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f720406e310 tx=0x7f71ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:00.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.886+0000 7f7203e00700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] conn(0x7f71e80ac8a0 0x7f71e80aecc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:00.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.887+0000 7f7200c00700 1 -- 172.21.15.97:0/4004527938 <== osd.5 v2:172.21.15.144:6808/356775844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71f40056d0 con 0x7f71e80ac8a0 2024-03-26T18:57:00.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.905+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 --> [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71e0002e60 con 0x7f71e80ac8a0 2024-03-26T18:57:00.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.905+0000 7f7200c00700 1 -- 172.21.15.97:0/4004527938 <== osd.5 v2:172.21.15.144:6808/356775844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f71e0002e60 con 0x7f71e80ac8a0 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 >> [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] conn(0x7f71e80ac8a0 msgr2=0x7f71e80aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f720a800700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] conn(0x7f71e80ac8a0 0x7f71e80aecc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f71e80a30c0 msgr2=0x7f71e80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f720a800700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f71e80a30c0 0x7f71e80a5580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f720406e310 tx=0x7f71ec009260 comp rx=0 tx=0).stop 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72040745e0 msgr2=0x7f7204073ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f720a800700 1 --2- 172.21.15.97:0/4004527938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72040745e0 0x7f7204073ac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f71f4005910 tx=0x7f71f400bab0 comp rx=0 tx=0).stop 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f7203e00700 1 -- 172.21.15.97:0/4004527938 reap_dead start 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 shutdown_connections 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 >> 172.21.15.97:0/4004527938 conn(0x7f7204079510 msgr2=0x7f72040ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 shutdown_connections 2024-03-26T18:57:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:00.906+0000 7f720a800700 1 -- 172.21.15.97:0/4004527938 wait complete. 2024-03-26T18:57:02.102 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:02 smithi097 ceph-mon[31955]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-26T18:57:02.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:02 smithi144 ceph-mon[36053]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-26T18:57:02.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:02 smithi105 ceph-mon[36270]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-26T18:57:04.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:04 smithi097 ceph-mon[31955]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:04.378 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:04 smithi144 ceph-mon[36053]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:04.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:04 smithi105 ceph-mon[36270]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:06.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:06 smithi097 ceph-mon[31955]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:57:06.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:06 smithi144 ceph-mon[36053]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:57:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.355+0000 7f52f6200700 1 -- 172.21.15.97:0/256722838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f0103e30 msgr2=0x7f52f010c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.355+0000 7f52f6200700 1 --2- 172.21.15.97:0/256722838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f0103e30 0x7f52f010c320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009990 tx=0x7f52dc009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.355+0000 7f52f6200700 1 -- 172.21.15.97:0/256722838 shutdown_connections 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.355+0000 7f52f6200700 1 --2- 172.21.15.97:0/256722838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52f010c8f0 0x7f52f010ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.355+0000 7f52f6200700 1 --2- 172.21.15.97:0/256722838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52f0103510 0x7f52f01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.355+0000 7f52f6200700 1 --2- 172.21.15.97:0/256722838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f0103e30 0x7f52f010c320 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.355+0000 7f52f6200700 1 -- 172.21.15.97:0/256722838 >> 172.21.15.97:0/256722838 conn(0x7f52f00fcc80 msgr2=0x7f52f00ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.355+0000 7f52f6200700 1 -- 172.21.15.97:0/256722838 shutdown_connections 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.355+0000 7f52f6200700 1 -- 172.21.15.97:0/256722838 wait complete. 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.356+0000 7f52f6200700 1 Processor -- start 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.357+0000 7f52f6200700 1 -- start start 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.357+0000 7f52f6200700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f0103510 0x7f52f019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.357+0000 7f52f6200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52f0103e30 0x7f52f019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.357+0000 7f52f6200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52f010c8f0 0x7f52f019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.357+0000 7f52f6200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0112600 con 0x7f52f010c8f0 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.357+0000 7f52f6200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0112480 con 0x7f52f0103e30 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.357+0000 7f52f6200700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0112780 con 0x7f52f0103510 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52f4e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f0103510 0x7f52f019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52f4e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f0103510 0x7f52f019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:33868/0 (socket says 172.21.15.97:33868) 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52f4e00700 1 -- 172.21.15.97:0/2599749450 learned_addr learned my addr 172.21.15.97:0/2599749450 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52efe00700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52f0103e30 0x7f52f019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52f5800700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52f010c8f0 0x7f52f019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52f4e00700 1 -- 172.21.15.97:0/2599749450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52f0103e30 msgr2=0x7f52f019b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52f4e00700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52f0103e30 0x7f52f019b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52f4e00700 1 -- 172.21.15.97:0/2599749450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52f010c8f0 msgr2=0x7f52f019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52f4e00700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52f010c8f0 0x7f52f019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52f4e00700 1 -- 172.21.15.97:0/2599749450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f019fd20 con 0x7f52f0103510 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52efe00700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52f0103e30 0x7f52f019b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.358+0000 7f52f4e00700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f0103510 0x7f52f019ad10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52e400e9a0 tx=0x7f52e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.359+0000 7f52ed600700 1 -- 172.21.15.97:0/2599749450 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e400ca10 con 0x7f52f0103510 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.359+0000 7f52ed600700 1 -- 172.21.15.97:0/2599749450 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52e400cbb0 con 0x7f52f0103510 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.359+0000 7f52ed600700 1 -- 172.21.15.97:0/2599749450 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4018680 con 0x7f52f0103510 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.359+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f01a0010 con 0x7f52f0103510 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.359+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f01a7890 con 0x7f52f0103510 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.360+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52f0062b40 con 0x7f52f0103510 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.362+0000 7f52ed600700 1 -- 172.21.15.97:0/2599749450 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f52e4010430 con 0x7f52f0103510 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.363+0000 7f52ed600700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52d80a30c0 0x7f52d80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.363+0000 7f52ed600700 1 -- 172.21.15.97:0/2599749450 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f52e40cf680 con 0x7f52f0103510 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.364+0000 7f52efe00700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52d80a30c0 0x7f52d80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.364+0000 7f52ed600700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] conn(0x7f52d80ac8a0 0x7f52d80aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.364+0000 7f52ed600700 1 -- 172.21.15.97:0/2599749450 --> [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52e4015e80 con 0x7f52d80ac8a0 2024-03-26T18:57:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.364+0000 7f52ed600700 1 -- 172.21.15.97:0/2599749450 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f52e40d6050 con 0x7f52f0103510 2024-03-26T18:57:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.364+0000 7f52f5800700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] conn(0x7f52d80ac8a0 0x7f52d80aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.364+0000 7f52efe00700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52d80a30c0 0x7f52d80a5580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52f010e730 tx=0x7f52dc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.365+0000 7f52f5800700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] conn(0x7f52d80ac8a0 0x7f52d80aecc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.366+0000 7f52ed600700 1 -- 172.21.15.97:0/2599749450 <== osd.10 v2:172.21.15.105:6824/4281982959 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52e4015e80 con 0x7f52d80ac8a0 2024-03-26T18:57:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.387+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 --> [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52f0062f40 con 0x7f52d80ac8a0 2024-03-26T18:57:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.387+0000 7f52ed600700 1 -- 172.21.15.97:0/2599749450 <== osd.10 v2:172.21.15.105:6824/4281982959 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f52f0062f40 con 0x7f52d80ac8a0 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.388+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 >> [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] conn(0x7f52d80ac8a0 msgr2=0x7f52d80aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.388+0000 7f52f6200700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] conn(0x7f52d80ac8a0 0x7f52d80aecc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.388+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52d80a30c0 msgr2=0x7f52d80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.388+0000 7f52f6200700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52d80a30c0 0x7f52d80a5580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52f010e730 tx=0x7f52dc00b260 comp rx=0 tx=0).stop 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.388+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f0103510 msgr2=0x7f52f019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.388+0000 7f52f6200700 1 --2- 172.21.15.97:0/2599749450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f0103510 0x7f52f019ad10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52e400e9a0 tx=0x7f52e400ecb0 comp rx=0 tx=0).stop 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.388+0000 7f52f5800700 1 -- 172.21.15.97:0/2599749450 reap_dead start 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.388+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 shutdown_connections 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.388+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 >> 172.21.15.97:0/2599749450 conn(0x7f52f00fcc80 msgr2=0x7f52f00fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.389+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 shutdown_connections 2024-03-26T18:57:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:05.389+0000 7f52f6200700 1 -- 172.21.15.97:0/2599749450 wait complete. 2024-03-26T18:57:06.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:06 smithi105 ceph-mon[36270]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:57:06.906 INFO:teuthology.orchestra.run.smithi097.stdout:163208757304 2024-03-26T18:57:06.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-26T18:57:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.934+0000 7f4822400700 1 -- 172.21.15.97:0/2411540250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f481c06a060 msgr2=0x7f481c06a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.934+0000 7f4822400700 1 --2- 172.21.15.97:0/2411540250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f481c06a060 0x7f481c06a4c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4808009990 tx=0x7f4808009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.934+0000 7f4822400700 1 -- 172.21.15.97:0/2411540250 shutdown_connections 2024-03-26T18:57:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.934+0000 7f4822400700 1 --2- 172.21.15.97:0/2411540250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f481c06aa00 0x7f481c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.934+0000 7f4822400700 1 --2- 172.21.15.97:0/2411540250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c108400 0x7f481c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.934+0000 7f4822400700 1 --2- 172.21.15.97:0/2411540250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f481c06a060 0x7f481c06a4c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.934+0000 7f4822400700 1 -- 172.21.15.97:0/2411540250 >> 172.21.15.97:0/2411540250 conn(0x7f481c0781e0 msgr2=0x7f481c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.934+0000 7f4822400700 1 -- 172.21.15.97:0/2411540250 shutdown_connections 2024-03-26T18:57:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.934+0000 7f4822400700 1 -- 172.21.15.97:0/2411540250 wait complete. 2024-03-26T18:57:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.935+0000 7f4822400700 1 Processor -- start 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.936+0000 7f4822400700 1 -- start start 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.936+0000 7f4822400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c06a060 0x7f481c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.936+0000 7f4822400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f481c06aa00 0x7f481c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.936+0000 7f4822400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f481c108400 0x7f481c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.936+0000 7f4822400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1126b0 con 0x7f481c06a060 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.936+0000 7f4822400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f481c112530 con 0x7f481c06aa00 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.936+0000 7f4822400700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f481c112830 con 0x7f481c108400 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.936+0000 7f481b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c06a060 0x7f481c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.937+0000 7f481be00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f481c108400 0x7f481c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.937+0000 7f481b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c06a060 0x7f481c19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48998/0 (socket says 172.21.15.97:48998) 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.937+0000 7f481aa00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f481c06aa00 0x7f481c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.937+0000 7f481b400700 1 -- 172.21.15.97:0/3954304855 learned_addr learned my addr 172.21.15.97:0/3954304855 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.937+0000 7f481be00700 1 -- 172.21.15.97:0/3954304855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f481c06aa00 msgr2=0x7f481c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.937+0000 7f481be00700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f481c06aa00 0x7f481c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.937+0000 7f481be00700 1 -- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c06a060 msgr2=0x7f481c19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.937+0000 7f481be00700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c06a060 0x7f481c19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.937+0000 7f481be00700 1 -- 172.21.15.97:0/3954304855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c19fe00 con 0x7f481c108400 2024-03-26T18:57:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.937+0000 7f481be00700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f481c108400 0x7f481c19f6c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f481400efb0 tx=0x7f481400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.938+0000 7f4813e00700 1 -- 172.21.15.97:0/3954304855 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481400ca50 con 0x7f481c108400 2024-03-26T18:57:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.938+0000 7f4813e00700 1 -- 172.21.15.97:0/3954304855 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481400cbf0 con 0x7f481c108400 2024-03-26T18:57:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.938+0000 7f4813e00700 1 -- 172.21.15.97:0/3954304855 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140048b0 con 0x7f481c108400 2024-03-26T18:57:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.938+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c1a00f0 con 0x7f481c108400 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.938+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c1a7970 con 0x7f481c108400 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.939+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f481c100690 con 0x7f481c108400 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.942+0000 7f4813e00700 1 -- 172.21.15.97:0/3954304855 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f4814012050 con 0x7f481c108400 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.944+0000 7f4813e00700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48000a30c0 0x7f48000a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.944+0000 7f4813e00700 1 -- 172.21.15.97:0/3954304855 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f48140cee30 con 0x7f481c108400 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.944+0000 7f481b400700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48000a30c0 0x7f48000a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.944+0000 7f4813e00700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] conn(0x7f48000ac8a0 0x7f48000aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.944+0000 7f4813e00700 1 -- 172.21.15.97:0/3954304855 --> [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48000af370 con 0x7f48000ac8a0 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.944+0000 7f4813e00700 1 -- 172.21.15.97:0/3954304855 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f48140d2020 con 0x7f481c108400 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.945+0000 7f481aa00700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] conn(0x7f48000ac8a0 0x7f48000aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.945+0000 7f481b400700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48000a30c0 0x7f48000a5580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f480c009c90 tx=0x7f480c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.945+0000 7f481aa00700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] conn(0x7f48000ac8a0 0x7f48000aecc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:08.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.946+0000 7f4813e00700 1 -- 172.21.15.97:0/3954304855 <== osd.9 v2:172.21.15.97:6826/2932879599 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48000af370 con 0x7f48000ac8a0 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.973+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 --> [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f481c103ab0 con 0x7f48000ac8a0 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.973+0000 7f4813e00700 1 -- 172.21.15.97:0/3954304855 <== osd.9 v2:172.21.15.97:6826/2932879599 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f481c103ab0 con 0x7f48000ac8a0 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] conn(0x7f48000ac8a0 msgr2=0x7f48000aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f4822400700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] conn(0x7f48000ac8a0 0x7f48000aecc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48000a30c0 msgr2=0x7f48000a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f4822400700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f48000a30c0 0x7f48000a5580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f480c009c90 tx=0x7f480c009260 comp rx=0 tx=0).stop 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f481c108400 msgr2=0x7f481c19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f4822400700 1 --2- 172.21.15.97:0/3954304855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f481c108400 0x7f481c19f6c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f481400efb0 tx=0x7f481400c3a0 comp rx=0 tx=0).stop 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f481be00700 1 -- 172.21.15.97:0/3954304855 reap_dead start 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 shutdown_connections 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 >> 172.21.15.97:0/3954304855 conn(0x7f481c0781e0 msgr2=0x7f481c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 shutdown_connections 2024-03-26T18:57:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:07.974+0000 7f4822400700 1 -- 172.21.15.97:0/3954304855 wait complete. 2024-03-26T18:57:08.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:08 smithi097 ceph-mon[31955]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:08.375 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:08 smithi144 ceph-mon[36053]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:08.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:08 smithi105 ceph-mon[36270]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.224+0000 7f8f9f600700 1 -- 172.21.15.97:0/2105105473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f981025f0 msgr2=0x7f8f981029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.224+0000 7f8f9f600700 1 --2- 172.21.15.97:0/2105105473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f981025f0 0x7f8f981029d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0099c0 tx=0x7f8f8c009cd0 comp rx=0 tx=0).stop 2024-03-26T18:57:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.224+0000 7f8f9f600700 1 -- 172.21.15.97:0/2105105473 shutdown_connections 2024-03-26T18:57:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.224+0000 7f8f9f600700 1 --2- 172.21.15.97:0/2105105473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f981076d0 0x7f8f9810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.224+0000 7f8f9f600700 1 --2- 172.21.15.97:0/2105105473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f981025f0 0x7f8f981029d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.224+0000 7f8f9f600700 1 --2- 172.21.15.97:0/2105105473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f98102fa0 0x7f8f98106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.224+0000 7f8f9f600700 1 -- 172.21.15.97:0/2105105473 >> 172.21.15.97:0/2105105473 conn(0x7f8f980fde60 msgr2=0x7f8f98100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.225+0000 7f8f9f600700 1 -- 172.21.15.97:0/2105105473 shutdown_connections 2024-03-26T18:57:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.225+0000 7f8f9f600700 1 -- 172.21.15.97:0/2105105473 wait complete. 2024-03-26T18:57:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.226+0000 7f8f9f600700 1 Processor -- start 2024-03-26T18:57:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.226+0000 7f8f9f600700 1 -- start start 2024-03-26T18:57:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.226+0000 7f8f9f600700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f981025f0 0x7f8f9819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.226+0000 7f8f9f600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f98102fa0 0x7f8f9819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.226+0000 7f8f9f600700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f981076d0 0x7f8f9819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.226+0000 7f8f9f600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8f98112700 con 0x7f8f98102fa0 2024-03-26T18:57:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.226+0000 7f8f9f600700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8f98112580 con 0x7f8f981025f0 2024-03-26T18:57:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.226+0000 7f8f9f600700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f98112880 con 0x7f8f981076d0 2024-03-26T18:57:09.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.226+0000 7f8f9cc00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f981025f0 0x7f8f9819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:09.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f9cc00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f981025f0 0x7f8f9819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38260/0 (socket says 172.21.15.97:38260) 2024-03-26T18:57:09.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f9cc00700 1 -- 172.21.15.97:0/1293253972 learned_addr learned my addr 172.21.15.97:0/1293253972 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:09.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f9d600700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f981076d0 0x7f8f9819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:09.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f97e00700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f98102fa0 0x7f8f9819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:09.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f9d600700 1 -- 172.21.15.97:0/1293253972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f981025f0 msgr2=0x7f8f9819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:09.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f9d600700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f981025f0 0x7f8f9819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:09.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f9d600700 1 -- 172.21.15.97:0/1293253972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f98102fa0 msgr2=0x7f8f9819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:09.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f9d600700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f98102fa0 0x7f8f9819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:09.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f9d600700 1 -- 172.21.15.97:0/1293253972 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9819fd70 con 0x7f8f981076d0 2024-03-26T18:57:09.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f9d600700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f981076d0 0x7f8f9819f630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000efc0 tx=0x7f8f9000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:09.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.227+0000 7f8f95600700 1 -- 172.21.15.97:0/1293253972 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f900092e0 con 0x7f8f981076d0 2024-03-26T18:57:09.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.228+0000 7f8f95600700 1 -- 172.21.15.97:0/1293253972 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f90004bb0 con 0x7f8f981076d0 2024-03-26T18:57:09.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.228+0000 7f8f95600700 1 -- 172.21.15.97:0/1293253972 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90004280 con 0x7f8f981076d0 2024-03-26T18:57:09.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.228+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f981a0000 con 0x7f8f981076d0 2024-03-26T18:57:09.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.228+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f981a78e0 con 0x7f8f981076d0 2024-03-26T18:57:09.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.229+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f98105bb0 con 0x7f8f981076d0 2024-03-26T18:57:09.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.231+0000 7f8f95600700 1 -- 172.21.15.97:0/1293253972 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f8f9001b030 con 0x7f8f981076d0 2024-03-26T18:57:09.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.233+0000 7f8f95600700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8f800a30c0 0x7f8f800a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:09.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.233+0000 7f8f95600700 1 -- 172.21.15.97:0/1293253972 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f8f900ceec0 con 0x7f8f981076d0 2024-03-26T18:57:09.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.233+0000 7f8f9cc00700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8f800a30c0 0x7f8f800a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:09.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.234+0000 7f8f95600700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] conn(0x7f8f800ac8a0 0x7f8f800aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:09.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.234+0000 7f8f95600700 1 -- 172.21.15.97:0/1293253972 --> [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f900048a0 con 0x7f8f800ac8a0 2024-03-26T18:57:09.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.234+0000 7f8f95600700 1 -- 172.21.15.97:0/1293253972 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f8f900d14e0 con 0x7f8f981076d0 2024-03-26T18:57:09.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.234+0000 7f8f97e00700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] conn(0x7f8f800ac8a0 0x7f8f800aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:09.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.234+0000 7f8f9cc00700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8f800a30c0 0x7f8f800a5580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c000c00 tx=0x7f8f8c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:09.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.235+0000 7f8f97e00700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] conn(0x7f8f800ac8a0 0x7f8f800aecc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:09.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.236+0000 7f8f95600700 1 -- 172.21.15.97:0/1293253972 <== osd.7 v2:172.21.15.105:6816/1025195601 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f900048a0 con 0x7f8f800ac8a0 2024-03-26T18:57:09.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.263+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 --> [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f98103ad0 con 0x7f8f800ac8a0 2024-03-26T18:57:09.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.263+0000 7f8f95600700 1 -- 172.21.15.97:0/1293253972 <== osd.7 v2:172.21.15.105:6816/1025195601 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8f98103ad0 con 0x7f8f800ac8a0 2024-03-26T18:57:09.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 >> [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] conn(0x7f8f800ac8a0 msgr2=0x7f8f800aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9f600700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] conn(0x7f8f800ac8a0 0x7f8f800aecc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8f800a30c0 msgr2=0x7f8f800a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9f600700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8f800a30c0 0x7f8f800a5580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c000c00 tx=0x7f8f8c005fb0 comp rx=0 tx=0).stop 2024-03-26T18:57:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f981076d0 msgr2=0x7f8f9819f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9f600700 1 --2- 172.21.15.97:0/1293253972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f981076d0 0x7f8f9819f630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000efc0 tx=0x7f8f9000c3a0 comp rx=0 tx=0).stop 2024-03-26T18:57:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9d600700 1 -- 172.21.15.97:0/1293253972 reap_dead start 2024-03-26T18:57:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 shutdown_connections 2024-03-26T18:57:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 >> 172.21.15.97:0/1293253972 conn(0x7f8f980fde60 msgr2=0x7f8f98100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 shutdown_connections 2024-03-26T18:57:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:09.264+0000 7f8f9f600700 1 -- 172.21.15.97:0/1293253972 wait complete. 2024-03-26T18:57:10.296 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:10 smithi097 ceph-mon[31955]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-26T18:57:10.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:10 smithi144 ceph-mon[36053]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-26T18:57:10.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.155+0000 7f9937000700 1 -- 172.21.15.97:0/3739393024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f993010a650 msgr2=0x7f993010aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:10.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.155+0000 7f9937000700 1 --2- 172.21.15.97:0/3739393024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f993010a650 0x7f993010aa30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9920009990 tx=0x7f9920009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:10.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.155+0000 7f9937000700 1 -- 172.21.15.97:0/3739393024 shutdown_connections 2024-03-26T18:57:10.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.155+0000 7f9937000700 1 --2- 172.21.15.97:0/3739393024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9930104f80 0x7f9930111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.155+0000 7f9937000700 1 --2- 172.21.15.97:0/3739393024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f993010a650 0x7f993010aa30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.155+0000 7f9937000700 1 --2- 172.21.15.97:0/3739393024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99301045c0 0x7f9930104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.155+0000 7f9937000700 1 -- 172.21.15.97:0/3739393024 >> 172.21.15.97:0/3739393024 conn(0x7f99301000b0 msgr2=0x7f99301024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.156+0000 7f9937000700 1 -- 172.21.15.97:0/3739393024 shutdown_connections 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.156+0000 7f9937000700 1 -- 172.21.15.97:0/3739393024 wait complete. 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.157+0000 7f9937000700 1 Processor -- start 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.157+0000 7f9937000700 1 -- start start 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.157+0000 7f9937000700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99301045c0 0x7f993019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.157+0000 7f9937000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9930104f80 0x7f993019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.157+0000 7f9937000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f993010a650 0x7f993019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.157+0000 7f9937000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9930114940 con 0x7f9930104f80 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.157+0000 7f9937000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f99301147c0 con 0x7f993010a650 2024-03-26T18:57:10.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.157+0000 7f9937000700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9930114ac0 con 0x7f99301045c0 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f992fe00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99301045c0 0x7f993019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f992fe00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99301045c0 0x7f993019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:33890/0 (socket says 172.21.15.97:33890) 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f992fe00700 1 -- 172.21.15.97:0/2437190163 learned_addr learned my addr 172.21.15.97:0/2437190163 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f992f400700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9930104f80 0x7f993019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f9935000700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f993010a650 0x7f993019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f992fe00700 1 -- 172.21.15.97:0/2437190163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f993010a650 msgr2=0x7f993019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f992fe00700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f993010a650 0x7f993019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f992fe00700 1 -- 172.21.15.97:0/2437190163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9930104f80 msgr2=0x7f993019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f992fe00700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9930104f80 0x7f993019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f992fe00700 1 -- 172.21.15.97:0/2437190163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993019fdb0 con 0x7f99301045c0 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.158+0000 7f992fe00700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99301045c0 0x7f993019adf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9920000c00 tx=0x7f992000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.159+0000 7f992cc00700 1 -- 172.21.15.97:0/2437190163 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920026070 con 0x7f99301045c0 2024-03-26T18:57:10.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.159+0000 7f992cc00700 1 -- 172.21.15.97:0/2437190163 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9920003d70 con 0x7f99301045c0 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.159+0000 7f992cc00700 1 -- 172.21.15.97:0/2437190163 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992000fe20 con 0x7f99301045c0 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.159+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99301a0040 con 0x7f99301045c0 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.159+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99301a7920 con 0x7f99301045c0 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.160+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9930062b40 con 0x7f99301045c0 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.162+0000 7f992cc00700 1 -- 172.21.15.97:0/2437190163 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f9920009d10 con 0x7f99301045c0 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.164+0000 7f992cc00700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f99140a30c0 0x7f99140a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.164+0000 7f992cc00700 1 -- 172.21.15.97:0/2437190163 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f99200d05a0 con 0x7f99301045c0 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.164+0000 7f992f400700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f99140a30c0 0x7f99140a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.164+0000 7f992cc00700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] conn(0x7f99140ac770 0x7f99140aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.165+0000 7f992cc00700 1 -- 172.21.15.97:0/2437190163 --> [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99140af240 con 0x7f99140ac770 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.165+0000 7f992cc00700 1 -- 172.21.15.97:0/2437190163 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f99200d0950 con 0x7f99301045c0 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.165+0000 7f9935000700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] conn(0x7f99140ac770 0x7f99140aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:10.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.165+0000 7f992f400700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f99140a30c0 0x7f99140a5580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9930109d50 tx=0x7f991c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.165+0000 7f9935000700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] conn(0x7f99140ac770 0x7f99140aeb90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.166+0000 7f992cc00700 1 -- 172.21.15.97:0/2437190163 <== osd.11 v2:172.21.15.144:6824/2295652267 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99140af240 con 0x7f99140ac770 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.184+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 --> [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9930062fc0 con 0x7f99140ac770 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.184+0000 7f992cc00700 1 -- 172.21.15.97:0/2437190163 <== osd.11 v2:172.21.15.144:6824/2295652267 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9930062fc0 con 0x7f99140ac770 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 >> [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] conn(0x7f99140ac770 msgr2=0x7f99140aeb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9937000700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] conn(0x7f99140ac770 0x7f99140aeb90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f99140a30c0 msgr2=0x7f99140a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9937000700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f99140a30c0 0x7f99140a5580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9930109d50 tx=0x7f991c009260 comp rx=0 tx=0).stop 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99301045c0 msgr2=0x7f993019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9937000700 1 --2- 172.21.15.97:0/2437190163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99301045c0 0x7f993019adf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9920000c00 tx=0x7f992000bab0 comp rx=0 tx=0).stop 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9935000700 1 -- 172.21.15.97:0/2437190163 reap_dead start 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 shutdown_connections 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 >> 172.21.15.97:0/2437190163 conn(0x7f99301000b0 msgr2=0x7f9930101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 shutdown_connections 2024-03-26T18:57:10.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:06.185+0000 7f9937000700 1 -- 172.21.15.97:0/2437190163 wait complete. 2024-03-26T18:57:10.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:10 smithi105 ceph-mon[36270]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-26T18:57:10.899 INFO:teuthology.orchestra.run.smithi097.stdout:77309411399 2024-03-26T18:57:10.899 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.702+0000 7f7019200700 1 -- 172.21.15.97:0/188056262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7014102fa0 msgr2=0x7f7014106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.702+0000 7f7019200700 1 --2- 172.21.15.97:0/188056262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7014102fa0 0x7f7014106ff0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7000009990 tx=0x7f7000009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.702+0000 7f7019200700 1 -- 172.21.15.97:0/188056262 shutdown_connections 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.702+0000 7f7019200700 1 --2- 172.21.15.97:0/188056262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70141076d0 0x7f701410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.702+0000 7f7019200700 1 --2- 172.21.15.97:0/188056262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70141025f0 0x7f70141029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.702+0000 7f7019200700 1 --2- 172.21.15.97:0/188056262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7014102fa0 0x7f7014106ff0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.702+0000 7f7019200700 1 -- 172.21.15.97:0/188056262 >> 172.21.15.97:0/188056262 conn(0x7f70140fde60 msgr2=0x7f7014100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.702+0000 7f7019200700 1 -- 172.21.15.97:0/188056262 shutdown_connections 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.702+0000 7f7019200700 1 -- 172.21.15.97:0/188056262 wait complete. 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.704+0000 7f7019200700 1 Processor -- start 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.704+0000 7f7019200700 1 -- start start 2024-03-26T18:57:11.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7019200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70141025f0 0x7f70140746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7019200700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7014102fa0 0x7f7014074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7019200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70141076d0 0x7f7014075130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7019200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f70141126e0 con 0x7f70141076d0 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7019200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7014112560 con 0x7f70141025f0 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7019200700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7014112860 con 0x7f7014102fa0 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7011e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70141025f0 0x7f70140746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7011e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70141025f0 0x7f70140746b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53402/0 (socket says 172.21.15.97:53402) 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7011e00700 1 -- 172.21.15.97:0/3471266445 learned_addr learned my addr 172.21.15.97:0/3471266445 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7011400700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7014102fa0 0x7f7014074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7012800700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70141076d0 0x7f7014075130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7011e00700 1 -- 172.21.15.97:0/3471266445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7014102fa0 msgr2=0x7f7014074bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7011e00700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7014102fa0 0x7f7014074bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:11.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7011e00700 1 -- 172.21.15.97:0/3471266445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70141076d0 msgr2=0x7f7014075130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:11.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7011e00700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70141076d0 0x7f7014075130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:11.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.705+0000 7f7011e00700 1 -- 172.21.15.97:0/3471266445 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70141a7130 con 0x7f70141025f0 2024-03-26T18:57:11.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.706+0000 7f7012800700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70141076d0 0x7f7014075130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:57:11.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.706+0000 7f7011e00700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70141025f0 0x7f70140746b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f700400e9a0 tx=0x7f700400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:11.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.706+0000 7f700aa00700 1 -- 172.21.15.97:0/3471266445 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700400cbd0 con 0x7f70141025f0 2024-03-26T18:57:11.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.706+0000 7f700aa00700 1 -- 172.21.15.97:0/3471266445 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f700400cd70 con 0x7f70141025f0 2024-03-26T18:57:11.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.706+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70141a7420 con 0x7f70141025f0 2024-03-26T18:57:11.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.706+0000 7f700aa00700 1 -- 172.21.15.97:0/3471266445 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004018750 con 0x7f70141025f0 2024-03-26T18:57:11.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.706+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70141a7980 con 0x7f70141025f0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.707+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7014062ac0 con 0x7f70141025f0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.710+0000 7f700aa00700 1 -- 172.21.15.97:0/3471266445 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f7004010430 con 0x7f70141025f0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.712+0000 7f700aa00700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6ff80a30c0 0x7f6ff80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.712+0000 7f7011400700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6ff80a30c0 0x7f6ff80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.712+0000 7f700aa00700 1 -- 172.21.15.97:0/3471266445 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f70040cf710 con 0x7f70141025f0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.713+0000 7f700aa00700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] conn(0x7f6ff80ac8a0 0x7f6ff80aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.713+0000 7f700aa00700 1 -- 172.21.15.97:0/3471266445 --> [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7004024e50 con 0x7f6ff80ac8a0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.713+0000 7f700aa00700 1 -- 172.21.15.97:0/3471266445 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f70040d6050 con 0x7f70141025f0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.713+0000 7f7012800700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] conn(0x7f6ff80ac8a0 0x7f6ff80aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.713+0000 7f7011400700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6ff80a30c0 0x7f6ff80a5580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f70140760f0 tx=0x7f700000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.713+0000 7f7012800700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] conn(0x7f6ff80ac8a0 0x7f6ff80aecc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.715+0000 7f700aa00700 1 -- 172.21.15.97:0/3471266445 <== osd.8 v2:172.21.15.144:6816/3722681651 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7004024e50 con 0x7f6ff80ac8a0 2024-03-26T18:57:11.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.741+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 --> [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7014062ec0 con 0x7f6ff80ac8a0 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.741+0000 7f700aa00700 1 -- 172.21.15.97:0/3471266445 <== osd.8 v2:172.21.15.144:6816/3722681651 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7014062ec0 con 0x7f6ff80ac8a0 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.742+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 >> [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] conn(0x7f6ff80ac8a0 msgr2=0x7f6ff80aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.742+0000 7f7019200700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] conn(0x7f6ff80ac8a0 0x7f6ff80aecc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.743+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6ff80a30c0 msgr2=0x7f6ff80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.743+0000 7f7019200700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6ff80a30c0 0x7f6ff80a5580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f70140760f0 tx=0x7f700000b260 comp rx=0 tx=0).stop 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.743+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70141025f0 msgr2=0x7f70140746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.743+0000 7f7019200700 1 --2- 172.21.15.97:0/3471266445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70141025f0 0x7f70140746b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f700400e9a0 tx=0x7f700400ecb0 comp rx=0 tx=0).stop 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.743+0000 7f7012800700 1 -- 172.21.15.97:0/3471266445 reap_dead start 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.743+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 shutdown_connections 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.743+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 >> 172.21.15.97:0/3471266445 conn(0x7f70140fde60 msgr2=0x7f7014100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.743+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 shutdown_connections 2024-03-26T18:57:11.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:11.743+0000 7f7019200700 1 -- 172.21.15.97:0/3471266445 wait complete. 2024-03-26T18:57:12.243 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:12 smithi097 ceph-mon[31955]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:57:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:309237645339 2024-03-26T18:57:12.247 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-26T18:57:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:12 smithi144 ceph-mon[36053]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:57:12.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:12 smithi105 ceph-mon[36270]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:57:12.749 INFO:teuthology.orchestra.run.smithi097.stdout:253403070504 2024-03-26T18:57:12.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-26T18:57:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.357+0000 7fd8fe000700 1 -- 172.21.15.97:0/90182048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8f8104db0 msgr2=0x7fd8f8105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.357+0000 7fd8fe000700 1 --2- 172.21.15.97:0/90182048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8f8104db0 0x7fd8f8105240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009f90 tx=0x7fd8f000b330 comp rx=0 tx=0).stop 2024-03-26T18:57:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.357+0000 7fd8fe000700 1 -- 172.21.15.97:0/90182048 shutdown_connections 2024-03-26T18:57:13.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.357+0000 7fd8fe000700 1 --2- 172.21.15.97:0/90182048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8f8104db0 0x7fd8f8105240 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.357+0000 7fd8fe000700 1 --2- 172.21.15.97:0/90182048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f81072a0 0x7fd8f806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.357+0000 7fd8fe000700 1 --2- 172.21.15.97:0/90182048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f806a5a0 0x7fd8f806aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.357+0000 7fd8fe000700 1 -- 172.21.15.97:0/90182048 >> 172.21.15.97:0/90182048 conn(0x7fd8f80781e0 msgr2=0x7fd8f80785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.357+0000 7fd8fe000700 1 -- 172.21.15.97:0/90182048 shutdown_connections 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.357+0000 7fd8fe000700 1 -- 172.21.15.97:0/90182048 wait complete. 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.358+0000 7fd8fe000700 1 Processor -- start 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8fe000700 1 -- start start 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8fe000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f806a5a0 0x7fd8f819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8fe000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8f8104db0 0x7fd8f819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8fe000700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f81072a0 0x7fd8f81a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8fe000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8118c80 con 0x7fd8f806a5a0 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8fe000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8118b00 con 0x7fd8f8104db0 2024-03-26T18:57:13.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8fe000700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8118e00 con 0x7fd8f81072a0 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8f6a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8f8104db0 0x7fd8f819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8f6a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8f8104db0 0x7fd8f819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53418/0 (socket says 172.21.15.97:53418) 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8f6a00700 1 -- 172.21.15.97:0/3558855745 learned_addr learned my addr 172.21.15.97:0/3558855745 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.359+0000 7fd8f7e00700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f81072a0 0x7fd8f81a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8f6a00700 1 -- 172.21.15.97:0/3558855745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f81072a0 msgr2=0x7fd8f81a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8f6a00700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f81072a0 0x7fd8f81a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8f6a00700 1 -- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f806a5a0 msgr2=0x7fd8f819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8f6a00700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f806a5a0 0x7fd8f819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8f6a00700 1 -- 172.21.15.97:0/3558855745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f81a4120 con 0x7fd8f8104db0 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8f6a00700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8f8104db0 0x7fd8f819f650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400ec10 tx=0x7fd8e400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:13.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8efe00700 1 -- 172.21.15.97:0/3558855745 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e400cd40 con 0x7fd8f8104db0 2024-03-26T18:57:13.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8efe00700 1 -- 172.21.15.97:0/3558855745 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e4004500 con 0x7fd8f8104db0 2024-03-26T18:57:13.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f81a4410 con 0x7fd8f8104db0 2024-03-26T18:57:13.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8efe00700 1 -- 172.21.15.97:0/3558855745 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4010690 con 0x7fd8f8104db0 2024-03-26T18:57:13.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.360+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f81abc90 con 0x7fd8f8104db0 2024-03-26T18:57:13.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.361+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd8f81047f0 con 0x7fd8f8104db0 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.364+0000 7fd8efe00700 1 -- 172.21.15.97:0/3558855745 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fd8e4003f10 con 0x7fd8f8104db0 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.365+0000 7fd8efe00700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8dc0a30c0 0x7fd8dc0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.365+0000 7fd8efe00700 1 -- 172.21.15.97:0/3558855745 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd8e40cf9f0 con 0x7fd8f8104db0 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.365+0000 7fd8f7400700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8dc0a30c0 0x7fd8dc0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.366+0000 7fd8efe00700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] conn(0x7fd8dc0ac8a0 0x7fd8dc0aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.366+0000 7fd8efe00700 1 -- 172.21.15.97:0/3558855745 --> [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8e401ee50 con 0x7fd8dc0ac8a0 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.366+0000 7fd8efe00700 1 -- 172.21.15.97:0/3558855745 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd8e40cfda0 con 0x7fd8f8104db0 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.366+0000 7fd8f7e00700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] conn(0x7fd8dc0ac8a0 0x7fd8dc0aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.366+0000 7fd8f7400700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8dc0a30c0 0x7fd8dc0a5580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009b60 tx=0x7fd8e8009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.367+0000 7fd8f7e00700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] conn(0x7fd8dc0ac8a0 0x7fd8dc0aecc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.369+0000 7fd8efe00700 1 -- 172.21.15.97:0/3558855745 <== osd.3 v2:172.21.15.97:6810/1821717710 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8e401ee50 con 0x7fd8dc0ac8a0 2024-03-26T18:57:13.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.386+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 --> [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd8f8101890 con 0x7fd8dc0ac8a0 2024-03-26T18:57:13.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.386+0000 7fd8efe00700 1 -- 172.21.15.97:0/3558855745 <== osd.3 v2:172.21.15.97:6810/1821717710 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd8f8101890 con 0x7fd8dc0ac8a0 2024-03-26T18:57:13.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.387+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] conn(0x7fd8dc0ac8a0 msgr2=0x7fd8dc0aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.387+0000 7fd8fe000700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] conn(0x7fd8dc0ac8a0 0x7fd8dc0aecc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.387+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8dc0a30c0 msgr2=0x7fd8dc0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.387+0000 7fd8fe000700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8dc0a30c0 0x7fd8dc0a5580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009b60 tx=0x7fd8e8009620 comp rx=0 tx=0).stop 2024-03-26T18:57:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.387+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8f8104db0 msgr2=0x7fd8f819f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.387+0000 7fd8fe000700 1 --2- 172.21.15.97:0/3558855745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8f8104db0 0x7fd8f819f650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400ec10 tx=0x7fd8e400ef20 comp rx=0 tx=0).stop 2024-03-26T18:57:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.387+0000 7fd8f7e00700 1 -- 172.21.15.97:0/3558855745 reap_dead start 2024-03-26T18:57:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.387+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 shutdown_connections 2024-03-26T18:57:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.387+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 >> 172.21.15.97:0/3558855745 conn(0x7fd8f80781e0 msgr2=0x7fd8f8103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.387+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 shutdown_connections 2024-03-26T18:57:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:13.388+0000 7fd8fe000700 1 -- 172.21.15.97:0/3558855745 wait complete. 2024-03-26T18:57:13.940 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:14.104 INFO:teuthology.orchestra.run.smithi097.stdout:343597383701 2024-03-26T18:57:14.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-26T18:57:14.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:14 smithi097 ceph-mon[31955]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:14.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:14 smithi144 ceph-mon[36053]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:14.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:14 smithi105 ceph-mon[36270]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:15.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.407+0000 7fc07aa00700 1 -- 172.21.15.97:0/2995273050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc07410ebd0 msgr2=0x7fc074111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:15.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.407+0000 7fc07aa00700 1 --2- 172.21.15.97:0/2995273050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc07410ebd0 0x7fc074111040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00b0a0 tx=0x7fc06c00b3b0 comp rx=0 tx=0).stop 2024-03-26T18:57:15.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.407+0000 7fc07aa00700 1 -- 172.21.15.97:0/2995273050 shutdown_connections 2024-03-26T18:57:15.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.407+0000 7fc07aa00700 1 --2- 172.21.15.97:0/2995273050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc07410ebd0 0x7fc074111040 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:15.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.407+0000 7fc07aa00700 1 --2- 172.21.15.97:0/2995273050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0740747e0 0x7fc074074bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:15.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.407+0000 7fc07aa00700 1 --2- 172.21.15.97:0/2995273050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc074075100 0x7fc07410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:15.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.407+0000 7fc07aa00700 1 -- 172.21.15.97:0/2995273050 >> 172.21.15.97:0/2995273050 conn(0x7fc0740fde60 msgr2=0x7fc074100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:15.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.407+0000 7fc07aa00700 1 -- 172.21.15.97:0/2995273050 shutdown_connections 2024-03-26T18:57:15.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.407+0000 7fc07aa00700 1 -- 172.21.15.97:0/2995273050 wait complete. 2024-03-26T18:57:15.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.409+0000 7fc07aa00700 1 Processor -- start 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.409+0000 7fc07aa00700 1 -- start start 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.409+0000 7fc07aa00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0740747e0 0x7fc07410e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.409+0000 7fc07aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc074075100 0x7fc07410b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc07aa00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc07410ebd0 0x7fc07410bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc07aa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc074114900 con 0x7fc074075100 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc07aa00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc074114780 con 0x7fc07410ebd0 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc07aa00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc074114a80 con 0x7fc0740747e0 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc078a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc07410ebd0 0x7fc07410bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc078a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc07410ebd0 0x7fc07410bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53460/0 (socket says 172.21.15.97:53460) 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc078a00700 1 -- 172.21.15.97:0/4166302976 learned_addr learned my addr 172.21.15.97:0/4166302976 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc073400700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc074075100 0x7fc07410b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc078a00700 1 -- 172.21.15.97:0/4166302976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0740747e0 msgr2=0x7fc07410e4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T18:57:15.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc078a00700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0740747e0 0x7fc07410e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc078a00700 1 -- 172.21.15.97:0/4166302976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc074075100 msgr2=0x7fc07410b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc078a00700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc074075100 0x7fc07410b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc078a00700 1 -- 172.21.15.97:0/4166302976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07410c3b0 con 0x7fc07410ebd0 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc073400700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc074075100 0x7fc07410b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.410+0000 7fc078a00700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc07410ebd0 0x7fc07410bae0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00b070 tx=0x7fc06c004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.411+0000 7fc070c00700 1 -- 172.21.15.97:0/4166302976 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c021070 con 0x7fc07410ebd0 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.411+0000 7fc070c00700 1 -- 172.21.15.97:0/4166302976 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc06c007c00 con 0x7fc07410ebd0 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.411+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0741ab700 con 0x7fc07410ebd0 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.411+0000 7fc070c00700 1 -- 172.21.15.97:0/4166302976 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0048b0 con 0x7fc07410ebd0 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.411+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0741abc60 con 0x7fc07410ebd0 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.412+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc074062ac0 con 0x7fc07410ebd0 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.414+0000 7fc070c00700 1 -- 172.21.15.97:0/4166302976 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fc06c004000 con 0x7fc07410ebd0 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.415+0000 7fc070c00700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc0580a3140 0x7fc0580a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:15.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.415+0000 7fc070c00700 1 -- 172.21.15.97:0/4166302976 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc06c00f020 con 0x7fc07410ebd0 2024-03-26T18:57:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.415+0000 7fc073e00700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc0580a3140 0x7fc0580a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.416+0000 7fc070c00700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] conn(0x7fc0580ac920 0x7fc0580aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.416+0000 7fc070c00700 1 -- 172.21.15.97:0/4166302976 --> [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc06c01ae50 con 0x7fc0580ac920 2024-03-26T18:57:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.416+0000 7fc070c00700 1 -- 172.21.15.97:0/4166302976 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fc06c0d0e90 con 0x7fc07410ebd0 2024-03-26T18:57:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.416+0000 7fc073400700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] conn(0x7fc0580ac920 0x7fc0580aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.416+0000 7fc073e00700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc0580a3140 0x7fc0580a5600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc064009af0 tx=0x7fc064009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.416+0000 7fc073400700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] conn(0x7fc0580ac920 0x7fc0580aed40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.417+0000 7fc070c00700 1 -- 172.21.15.97:0/4166302976 <== osd.1 v2:172.21.15.105:6800/3472523969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc06c01ae50 con 0x7fc0580ac920 2024-03-26T18:57:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.443+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 --> [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc074062f40 con 0x7fc0580ac920 2024-03-26T18:57:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.444+0000 7fc070c00700 1 -- 172.21.15.97:0/4166302976 <== osd.1 v2:172.21.15.105:6800/3472523969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc074062f40 con 0x7fc0580ac920 2024-03-26T18:57:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.444+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 >> [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] conn(0x7fc0580ac920 msgr2=0x7fc0580aed40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.444+0000 7fc07aa00700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] conn(0x7fc0580ac920 0x7fc0580aed40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.445+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc0580a3140 msgr2=0x7fc0580a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.445+0000 7fc07aa00700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc0580a3140 0x7fc0580a5600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc064009af0 tx=0x7fc064009260 comp rx=0 tx=0).stop 2024-03-26T18:57:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.445+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc07410ebd0 msgr2=0x7fc07410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.445+0000 7fc07aa00700 1 --2- 172.21.15.97:0/4166302976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc07410ebd0 0x7fc07410bae0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00b070 tx=0x7fc06c004d10 comp rx=0 tx=0).stop 2024-03-26T18:57:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.445+0000 7fc078a00700 1 -- 172.21.15.97:0/4166302976 reap_dead start 2024-03-26T18:57:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.445+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 shutdown_connections 2024-03-26T18:57:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.445+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 >> 172.21.15.97:0/4166302976 conn(0x7fc0740fde60 msgr2=0x7fc0740fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.445+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 shutdown_connections 2024-03-26T18:57:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.445+0000 7fc07aa00700 1 -- 172.21.15.97:0/4166302976 wait complete. 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.299+0000 7fd756000700 1 -- 172.21.15.97:0/3559542971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd750103450 msgr2=0x7fd750103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.299+0000 7fd756000700 1 --2- 172.21.15.97:0/3559542971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd750103450 0x7fd750103830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd744009990 tx=0x7fd744009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.299+0000 7fd756000700 1 -- 172.21.15.97:0/3559542971 shutdown_connections 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.299+0000 7fd756000700 1 --2- 172.21.15.97:0/3559542971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd75010c880 0x7fd75010ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.299+0000 7fd756000700 1 --2- 172.21.15.97:0/3559542971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd750103450 0x7fd750103830 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.299+0000 7fd756000700 1 --2- 172.21.15.97:0/3559542971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750103d70 0x7fd75010c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.299+0000 7fd756000700 1 -- 172.21.15.97:0/3559542971 >> 172.21.15.97:0/3559542971 conn(0x7fd7500fcc80 msgr2=0x7fd7500ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.299+0000 7fd756000700 1 -- 172.21.15.97:0/3559542971 shutdown_connections 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.299+0000 7fd756000700 1 -- 172.21.15.97:0/3559542971 wait complete. 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.301+0000 7fd756000700 1 Processor -- start 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.301+0000 7fd756000700 1 -- start start 2024-03-26T18:57:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.301+0000 7fd756000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd750103450 0x7fd7501010b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.301+0000 7fd756000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750103d70 0x7fd7501015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.301+0000 7fd756000700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd75010c880 0x7fd750101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.301+0000 7fd756000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd750112610 con 0x7fd750103d70 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.301+0000 7fd756000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd750112490 con 0x7fd750103450 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.301+0000 7fd756000700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd750112790 con 0x7fd75010c880 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd754c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd750103450 0x7fd7501010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd754c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd750103450 0x7fd7501010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53438/0 (socket says 172.21.15.97:53438) 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd754c00700 1 -- 172.21.15.97:0/960138931 learned_addr learned my addr 172.21.15.97:0/960138931 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd74fe00700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750103d70 0x7fd7501015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd755600700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd75010c880 0x7fd750101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd754c00700 1 -- 172.21.15.97:0/960138931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd75010c880 msgr2=0x7fd750101b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd754c00700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd75010c880 0x7fd750101b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd754c00700 1 -- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750103d70 msgr2=0x7fd7501015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd754c00700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750103d70 0x7fd7501015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd754c00700 1 -- 172.21.15.97:0/960138931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7501a7010 con 0x7fd750103450 2024-03-26T18:57:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd74fe00700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750103d70 0x7fd7501015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:57:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.302+0000 7fd754c00700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd750103450 0x7fd7501010b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd744005910 tx=0x7fd74400bb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.303+0000 7fd74d600700 1 -- 172.21.15.97:0/960138931 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744027070 con 0x7fd750103450 2024-03-26T18:57:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.303+0000 7fd74d600700 1 -- 172.21.15.97:0/960138931 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd744003d70 con 0x7fd750103450 2024-03-26T18:57:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.303+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7501a72a0 con 0x7fd750103450 2024-03-26T18:57:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.303+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7501a7780 con 0x7fd750103450 2024-03-26T18:57:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.303+0000 7fd74d600700 1 -- 172.21.15.97:0/960138931 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74400f6c0 con 0x7fd750103450 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.304+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd750062ac0 con 0x7fd750103450 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.307+0000 7fd74d600700 1 -- 172.21.15.97:0/960138931 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fd74402a070 con 0x7fd750103450 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.309+0000 7fd74d600700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7380a3040 0x7fd7380a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.309+0000 7fd74d600700 1 -- 172.21.15.97:0/960138931 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd7440d0580 con 0x7fd750103450 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.309+0000 7fd74fe00700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7380a3040 0x7fd7380a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.310+0000 7fd74d600700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] conn(0x7fd7380ac6f0 0x7fd7380aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.310+0000 7fd74d600700 1 -- 172.21.15.97:0/960138931 --> [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7380af1c0 con 0x7fd7380ac6f0 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.310+0000 7fd74d600700 1 -- 172.21.15.97:0/960138931 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd7440d7050 con 0x7fd750103450 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.310+0000 7fd755600700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] conn(0x7fd7380ac6f0 0x7fd7380aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.310+0000 7fd74fe00700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7380a3040 0x7fd7380a5500 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd75010dc00 tx=0x7fd73c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.311+0000 7fd755600700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] conn(0x7fd7380ac6f0 0x7fd7380aeb10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.312+0000 7fd74d600700 1 -- 172.21.15.97:0/960138931 <== osd.0 v2:172.21.15.97:6802/932099107 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7380af1c0 con 0x7fd7380ac6f0 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.336+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 --> [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd750062f40 con 0x7fd7380ac6f0 2024-03-26T18:57:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.337+0000 7fd74d600700 1 -- 172.21.15.97:0/960138931 <== osd.0 v2:172.21.15.97:6802/932099107 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd750062f40 con 0x7fd7380ac6f0 2024-03-26T18:57:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] conn(0x7fd7380ac6f0 msgr2=0x7fd7380aeb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd756000700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] conn(0x7fd7380ac6f0 0x7fd7380aeb10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7380a3040 msgr2=0x7fd7380a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd756000700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7380a3040 0x7fd7380a5500 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd75010dc00 tx=0x7fd73c009260 comp rx=0 tx=0).stop 2024-03-26T18:57:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd750103450 msgr2=0x7fd7501010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd756000700 1 --2- 172.21.15.97:0/960138931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd750103450 0x7fd7501010b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd744005910 tx=0x7fd74400bb60 comp rx=0 tx=0).stop 2024-03-26T18:57:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd755600700 1 -- 172.21.15.97:0/960138931 reap_dead start 2024-03-26T18:57:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 shutdown_connections 2024-03-26T18:57:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 >> 172.21.15.97:0/960138931 conn(0x7fd7500fcc80 msgr2=0x7fd7500fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 shutdown_connections 2024-03-26T18:57:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:15.338+0000 7fd756000700 1 -- 172.21.15.97:0/960138931 wait complete. 2024-03-26T18:57:16.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:16 smithi097 ceph-mon[31955]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:57:16.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:16 smithi144 ceph-mon[36053]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:57:16.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:16 smithi105 ceph-mon[36270]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:57:17.173 INFO:teuthology.orchestra.run.smithi097.stdout:107374182468 2024-03-26T18:57:17.173 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-26T18:57:17.541 INFO:teuthology.orchestra.run.smithi097.stdout:287762808864 2024-03-26T18:57:17.541 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-26T18:57:18.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:18 smithi097 ceph-mon[31955]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:18.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:18 smithi144 ceph-mon[36053]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:18.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:18 smithi105 ceph-mon[36270]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:19.212 INFO:teuthology.orchestra.run.smithi097.stdout:55834574927 2024-03-26T18:57:19.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-26T18:57:19.791 INFO:teuthology.orchestra.run.smithi097.stdout:34359738453 2024-03-26T18:57:19.791 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.617+0000 7f451fe00700 1 -- 172.21.15.97:0/1981547576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518102400 msgr2=0x7f4518102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.617+0000 7f451fe00700 1 --2- 172.21.15.97:0/1981547576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518102400 0x7f4518102880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4504009990 tx=0x7f4504009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.617+0000 7f451fe00700 1 -- 172.21.15.97:0/1981547576 shutdown_connections 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.617+0000 7f451fe00700 1 --2- 172.21.15.97:0/1981547576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4518102dc0 0x7f451810f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.617+0000 7f451fe00700 1 --2- 172.21.15.97:0/1981547576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4518108480 0x7f4518108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.617+0000 7f451fe00700 1 --2- 172.21.15.97:0/1981547576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518102400 0x7f4518102880 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.617+0000 7f451fe00700 1 -- 172.21.15.97:0/1981547576 >> 172.21.15.97:0/1981547576 conn(0x7f4518079510 msgr2=0x7f4518079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.617+0000 7f451fe00700 1 -- 172.21.15.97:0/1981547576 shutdown_connections 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.617+0000 7f451fe00700 1 -- 172.21.15.97:0/1981547576 wait complete. 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.618+0000 7f451fe00700 1 Processor -- start 2024-03-26T18:57:19.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.618+0000 7f451fe00700 1 -- start start 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.618+0000 7f451fe00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4518102400 0x7f451819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.618+0000 7f451fe00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518102dc0 0x7f451819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.618+0000 7f451fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4518108480 0x7f451819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.618+0000 7f451fe00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4518112740 con 0x7f4518108480 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.618+0000 7f451fe00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f45181125c0 con 0x7f4518102400 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.618+0000 7f451fe00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45181128c0 con 0x7f4518102dc0 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451d400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4518102400 0x7f451819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451d400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4518102400 0x7f451819aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53480/0 (socket says 172.21.15.97:53480) 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451d400700 1 -- 172.21.15.97:0/2784445664 learned_addr learned my addr 172.21.15.97:0/2784445664 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451ca00700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518102dc0 0x7f451819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451de00700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4518108480 0x7f451819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451d400700 1 -- 172.21.15.97:0/2784445664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518102dc0 msgr2=0x7f451819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:19.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451d400700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518102dc0 0x7f451819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451d400700 1 -- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4518108480 msgr2=0x7f451819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451d400700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4518108480 0x7f451819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451d400700 1 -- 172.21.15.97:0/2784445664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451819fec0 con 0x7f4518102400 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.619+0000 7f451d400700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4518102400 0x7f451819aeb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f450c00e9a0 tx=0x7f450c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.620+0000 7f4516000700 1 -- 172.21.15.97:0/2784445664 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c00cbd0 con 0x7f4518102400 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.620+0000 7f4516000700 1 -- 172.21.15.97:0/2784445664 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f450c00cd70 con 0x7f4518102400 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.620+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45181a01b0 con 0x7f4518102400 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.620+0000 7f4516000700 1 -- 172.21.15.97:0/2784445664 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c018750 con 0x7f4518102400 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.620+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45181a7a30 con 0x7f4518102400 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.621+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4518100c50 con 0x7f4518102400 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.623+0000 7f4516000700 1 -- 172.21.15.97:0/2784445664 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f450c010430 con 0x7f4518102400 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.624+0000 7f4516000700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45000a30c0 0x7f45000a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.624+0000 7f4516000700 1 -- 172.21.15.97:0/2784445664 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f450c0cfea0 con 0x7f4518102400 2024-03-26T18:57:19.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.624+0000 7f451ca00700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45000a30c0 0x7f45000a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:19.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.624+0000 7f4516000700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] conn(0x7f45000ac8a0 0x7f45000aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:19.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.624+0000 7f4516000700 1 -- 172.21.15.97:0/2784445664 --> [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45000af370 con 0x7f45000ac8a0 2024-03-26T18:57:19.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.624+0000 7f4516000700 1 -- 172.21.15.97:0/2784445664 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f450c0d0250 con 0x7f4518102400 2024-03-26T18:57:19.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.625+0000 7f451de00700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] conn(0x7f45000ac8a0 0x7f45000aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:19.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.625+0000 7f451ca00700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45000a30c0 0x7f45000a5580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f451810f6c0 tx=0x7f450400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:19.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.625+0000 7f451de00700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] conn(0x7f45000ac8a0 0x7f45000aecc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:19.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.626+0000 7f4516000700 1 -- 172.21.15.97:0/2784445664 <== osd.6 v2:172.21.15.97:6818/35447762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45000af370 con 0x7f45000ac8a0 2024-03-26T18:57:19.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.644+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 --> [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4518103b30 con 0x7f45000ac8a0 2024-03-26T18:57:19.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.645+0000 7f4516000700 1 -- 172.21.15.97:0/2784445664 <== osd.6 v2:172.21.15.97:6818/35447762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4518103b30 con 0x7f45000ac8a0 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.645+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] conn(0x7f45000ac8a0 msgr2=0x7f45000aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.645+0000 7f451fe00700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] conn(0x7f45000ac8a0 0x7f45000aecc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.645+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45000a30c0 msgr2=0x7f45000a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.645+0000 7f451fe00700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45000a30c0 0x7f45000a5580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f451810f6c0 tx=0x7f450400b260 comp rx=0 tx=0).stop 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.645+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4518102400 msgr2=0x7f451819aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.645+0000 7f451fe00700 1 --2- 172.21.15.97:0/2784445664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4518102400 0x7f451819aeb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f450c00e9a0 tx=0x7f450c00ecb0 comp rx=0 tx=0).stop 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.645+0000 7f451de00700 1 -- 172.21.15.97:0/2784445664 reap_dead start 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.646+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 shutdown_connections 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.646+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 >> 172.21.15.97:0/2784445664 conn(0x7f4518079510 msgr2=0x7f4518100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.646+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 shutdown_connections 2024-03-26T18:57:19.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:19.646+0000 7f451fe00700 1 -- 172.21.15.97:0/2784445664 wait complete. 2024-03-26T18:57:20.346 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:20 smithi097 ceph-mon[31955]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:57:20.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:20 smithi144 ceph-mon[36053]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:57:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:20 smithi105 ceph-mon[36270]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:57:20.763 INFO:teuthology.orchestra.run.smithi097.stdout:197568495669 2024-03-26T18:57:20.764 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-26T18:57:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.391+0000 7fb072600700 1 -- 172.21.15.97:0/17663405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06c10ec30 msgr2=0x7fb06c111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.391+0000 7fb072600700 1 --2- 172.21.15.97:0/17663405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06c10ec30 0x7fb06c111040 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb064009f90 tx=0x7fb06400b330 comp rx=0 tx=0).stop 2024-03-26T18:57:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.391+0000 7fb072600700 1 -- 172.21.15.97:0/17663405 shutdown_connections 2024-03-26T18:57:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.391+0000 7fb072600700 1 --2- 172.21.15.97:0/17663405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06c10ec30 0x7fb06c111040 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.391+0000 7fb072600700 1 --2- 172.21.15.97:0/17663405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb06c074780 0x7fb06c074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.391+0000 7fb072600700 1 --2- 172.21.15.97:0/17663405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c0750a0 0x7fb06c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.391+0000 7fb072600700 1 -- 172.21.15.97:0/17663405 >> 172.21.15.97:0/17663405 conn(0x7fb06c0fde60 msgr2=0x7fb06c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.391+0000 7fb072600700 1 -- 172.21.15.97:0/17663405 shutdown_connections 2024-03-26T18:57:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.391+0000 7fb072600700 1 -- 172.21.15.97:0/17663405 wait complete. 2024-03-26T18:57:21.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.393+0000 7fb072600700 1 Processor -- start 2024-03-26T18:57:21.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.393+0000 7fb072600700 1 -- start start 2024-03-26T18:57:21.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.393+0000 7fb072600700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06c074780 0x7fb06c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:21.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.393+0000 7fb072600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c0750a0 0x7fb06c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:21.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.393+0000 7fb072600700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb06c10ec30 0x7fb06c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:21.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.393+0000 7fb072600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c114910 con 0x7fb06c0750a0 2024-03-26T18:57:21.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.393+0000 7fb072600700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c114790 con 0x7fb06c10ec30 2024-03-26T18:57:21.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.393+0000 7fb072600700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c114a90 con 0x7fb06c074780 2024-03-26T18:57:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06b400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06c074780 0x7fb06c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06b400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06c074780 0x7fb06c19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53020/0 (socket says 172.21.15.97:53020) 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06b400700 1 -- 172.21.15.97:0/55058182 learned_addr learned my addr 172.21.15.97:0/55058182 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb063e00700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c0750a0 0x7fb06c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06be00700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb06c10ec30 0x7fb06c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06b400700 1 -- 172.21.15.97:0/55058182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb06c10ec30 msgr2=0x7fb06c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06b400700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb06c10ec30 0x7fb06c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06b400700 1 -- 172.21.15.97:0/55058182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c0750a0 msgr2=0x7fb06c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06b400700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c0750a0 0x7fb06c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06b400700 1 -- 172.21.15.97:0/55058182 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c19fe00 con 0x7fb06c074780 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06be00700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb06c10ec30 0x7fb06c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.394+0000 7fb06b400700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06c074780 0x7fb06c19ae40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00e9f0 tx=0x7fb05c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.395+0000 7fb068c00700 1 -- 172.21.15.97:0/55058182 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c004a30 con 0x7fb06c074780 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.395+0000 7fb068c00700 1 -- 172.21.15.97:0/55058182 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05c00cca0 con 0x7fb06c074780 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.395+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c1a00f0 con 0x7fb06c074780 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.395+0000 7fb068c00700 1 -- 172.21.15.97:0/55058182 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0056d0 con 0x7fb06c074780 2024-03-26T18:57:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.395+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1a7970 con 0x7fb06c074780 2024-03-26T18:57:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.396+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb06c062ac0 con 0x7fb06c074780 2024-03-26T18:57:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.399+0000 7fb068c00700 1 -- 172.21.15.97:0/55058182 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fb05c00c7c0 con 0x7fb06c074780 2024-03-26T18:57:21.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.400+0000 7fb068c00700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb0500a3260 0x7fb0500a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:21.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.401+0000 7fb068c00700 1 -- 172.21.15.97:0/55058182 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb05c0cf820 con 0x7fb06c074780 2024-03-26T18:57:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.401+0000 7fb063e00700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb0500a3260 0x7fb0500a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.401+0000 7fb068c00700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] conn(0x7fb0500aca40 0x7fb0500aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.401+0000 7fb068c00700 1 -- 172.21.15.97:0/55058182 --> [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb05c011e50 con 0x7fb0500aca40 2024-03-26T18:57:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.401+0000 7fb068c00700 1 -- 172.21.15.97:0/55058182 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb05c0d42c0 con 0x7fb06c074780 2024-03-26T18:57:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.401+0000 7fb06be00700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] conn(0x7fb0500aca40 0x7fb0500aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.402+0000 7fb063e00700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb0500a3260 0x7fb0500a5720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb06c101890 tx=0x7fb054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:21.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.402+0000 7fb06be00700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] conn(0x7fb0500aca40 0x7fb0500aee60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.403+0000 7fb068c00700 1 -- 172.21.15.97:0/55058182 <== osd.4 v2:172.21.15.105:6808/4057159020 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb05c011e50 con 0x7fb0500aca40 2024-03-26T18:57:21.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.420+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 --> [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb06c062f40 con 0x7fb0500aca40 2024-03-26T18:57:21.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.421+0000 7fb068c00700 1 -- 172.21.15.97:0/55058182 <== osd.4 v2:172.21.15.105:6808/4057159020 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb06c062f40 con 0x7fb0500aca40 2024-03-26T18:57:21.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.421+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 >> [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] conn(0x7fb0500aca40 msgr2=0x7fb0500aee60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:21.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.421+0000 7fb072600700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] conn(0x7fb0500aca40 0x7fb0500aee60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:21.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.421+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb0500a3260 msgr2=0x7fb0500a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:21.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.421+0000 7fb072600700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb0500a3260 0x7fb0500a5720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb06c101890 tx=0x7fb054009260 comp rx=0 tx=0).stop 2024-03-26T18:57:21.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.421+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06c074780 msgr2=0x7fb06c19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:21.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.421+0000 7fb072600700 1 --2- 172.21.15.97:0/55058182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06c074780 0x7fb06c19ae40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00e9f0 tx=0x7fb05c00ed00 comp rx=0 tx=0).stop 2024-03-26T18:57:21.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.421+0000 7fb06be00700 1 -- 172.21.15.97:0/55058182 reap_dead start 2024-03-26T18:57:21.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.422+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 shutdown_connections 2024-03-26T18:57:21.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.422+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 >> 172.21.15.97:0/55058182 conn(0x7fb06c0fde60 msgr2=0x7fb06c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:21.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.422+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 shutdown_connections 2024-03-26T18:57:21.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:21.422+0000 7fb072600700 1 -- 172.21.15.97:0/55058182 wait complete. 2024-03-26T18:57:22.193 INFO:teuthology.orchestra.run.smithi097.stdout:219043332142 2024-03-26T18:57:22.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-26T18:57:22.194 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:22.213 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:22.213 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:22.331 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:22 smithi097 ceph-mon[31955]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-26T18:57:22.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:22 smithi144 ceph-mon[36053]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-26T18:57:22.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:22 smithi105 ceph-mon[36270]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-26T18:57:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:22.814 INFO:teuthology.orchestra.run.smithi097.stdout:141733920833 2024-03-26T18:57:22.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-26T18:57:22.818 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:22.827 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:22.831 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:23.454 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:23.464 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:24.276 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:24 smithi097 ceph-mon[31955]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:24.276 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:24 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:57:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:24 smithi144 ceph-mon[36053]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:24 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:57:24.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:24 smithi105 ceph-mon[36270]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:24.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:24 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:57:26.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:26 smithi097 ceph-mon[31955]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:57:26.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:26 smithi144 ceph-mon[36053]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:57:26.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:26 smithi105 ceph-mon[36270]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:57:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:26.626 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:28.148 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:28 smithi097 ceph-mon[31955]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:28 smithi144 ceph-mon[36053]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:28.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:28 smithi105 ceph-mon[36270]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.953+0000 7f787aa00700 1 -- 172.21.15.97:0/3457988872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7874102fa0 msgr2=0x7f7874106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.953+0000 7f787aa00700 1 --2- 172.21.15.97:0/3457988872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7874102fa0 0x7f7874106ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78600099c0 tx=0x7f7860009cd0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.954+0000 7f787aa00700 1 -- 172.21.15.97:0/3457988872 shutdown_connections 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.954+0000 7f787aa00700 1 --2- 172.21.15.97:0/3457988872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78741076d0 0x7f787410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.954+0000 7f787aa00700 1 --2- 172.21.15.97:0/3457988872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78741025f0 0x7f78741029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.954+0000 7f787aa00700 1 --2- 172.21.15.97:0/3457988872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7874102fa0 0x7f7874106ff0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.954+0000 7f787aa00700 1 -- 172.21.15.97:0/3457988872 >> 172.21.15.97:0/3457988872 conn(0x7f78740fde60 msgr2=0x7f7874100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.954+0000 7f787aa00700 1 -- 172.21.15.97:0/3457988872 shutdown_connections 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.954+0000 7f787aa00700 1 -- 172.21.15.97:0/3457988872 wait complete. 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.955+0000 7f787aa00700 1 Processor -- start 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.955+0000 7f787aa00700 1 -- start start 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f787aa00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78741025f0 0x7f787419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f787aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874102fa0 0x7f787419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f787aa00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78741076d0 0x7f787419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f787aa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7874112700 con 0x7f7874102fa0 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f787aa00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7874112580 con 0x7f78741025f0 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f787aa00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7874112880 con 0x7f78741076d0 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f7873e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78741025f0 0x7f787419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f7873e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78741025f0 0x7f787419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:43246/0 (socket says 172.21.15.97:43246) 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f7873e00700 1 -- 172.21.15.97:0/1024400460 learned_addr learned my addr 172.21.15.97:0/1024400460 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f7878a00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78741076d0 0x7f787419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f7873400700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874102fa0 0x7f787419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.956+0000 7f7873e00700 1 -- 172.21.15.97:0/1024400460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78741076d0 msgr2=0x7f787419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.957+0000 7f7873e00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78741076d0 0x7f787419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.957+0000 7f7873e00700 1 -- 172.21.15.97:0/1024400460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874102fa0 msgr2=0x7f787419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.957+0000 7f7873e00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874102fa0 0x7f787419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.957+0000 7f7873e00700 1 -- 172.21.15.97:0/1024400460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787419fda0 con 0x7f78741025f0 2024-03-26T18:57:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.957+0000 7f7873e00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78741025f0 0x7f787419ad90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f786400e9a0 tx=0x7f786400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.957+0000 7f7870c00700 1 -- 172.21.15.97:0/1024400460 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786400ca10 con 0x7f78741025f0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.957+0000 7f7870c00700 1 -- 172.21.15.97:0/1024400460 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f786400cbb0 con 0x7f78741025f0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.957+0000 7f787aa00700 1 -- 172.21.15.97:0/1024400460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78741a0090 con 0x7f78741025f0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.957+0000 7f7870c00700 1 -- 172.21.15.97:0/1024400460 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7864018660 con 0x7f78741025f0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.957+0000 7f787aa00700 1 -- 172.21.15.97:0/1024400460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78741a7910 con 0x7f78741025f0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.959+0000 7f787aa00700 1 -- 172.21.15.97:0/1024400460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7854005330 con 0x7f78741025f0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.962+0000 7f7870c00700 1 -- 172.21.15.97:0/1024400460 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f7864018800 con 0x7f78741025f0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.964+0000 7f7870c00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f78580a30c0 0x7f78580a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.964+0000 7f7870c00700 1 -- 172.21.15.97:0/1024400460 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f78640cf4f0 con 0x7f78741025f0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.964+0000 7f7873400700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f78580a30c0 0x7f78580a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.965+0000 7f7873400700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f78580a30c0 0x7f78580a5580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7860000c00 tx=0x7f7860005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:28.967+0000 7f7870c00700 1 -- 172.21.15.97:0/1024400460 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f786409c270 con 0x7f78741025f0 2024-03-26T18:57:29.244 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-03-26T18:57:29.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-26T18:57:29.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-26T18:57:29.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-03-26T18:57:29.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-26T18:57:29.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-26T18:57:29.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-26T18:57:29.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.269+0000 7f787aa00700 1 -- 172.21.15.97:0/1024400460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f78540058f0 con 0x7f78741025f0 2024-03-26T18:57:29.264 INFO:teuthology.orchestra.run.smithi097.stdout:309237645343 2024-03-26T18:57:29.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.269+0000 7f7870c00700 1 -- 172.21.15.97:0/1024400460 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f786409bb70 con 0x7f78741025f0 2024-03-26T18:57:29.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.273+0000 7f787aa00700 1 -- 172.21.15.97:0/1024400460 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f78580a30c0 msgr2=0x7f78580a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:29.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.273+0000 7f787aa00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f78580a30c0 0x7f78580a5580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7860000c00 tx=0x7f7860005fb0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.273+0000 7f787aa00700 1 -- 172.21.15.97:0/1024400460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78741025f0 msgr2=0x7f787419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:29.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.273+0000 7f787aa00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78741025f0 0x7f787419ad90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f786400e9a0 tx=0x7f786400ecb0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.274+0000 7f787aa00700 1 -- 172.21.15.97:0/1024400460 shutdown_connections 2024-03-26T18:57:29.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.274+0000 7f787aa00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f78580a30c0 0x7f78580a5580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.274+0000 7f787aa00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78741076d0 0x7f787419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.274+0000 7f787aa00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78741025f0 0x7f787419ad90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.274+0000 7f787aa00700 1 --2- 172.21.15.97:0/1024400460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874102fa0 0x7f787419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.274+0000 7f787aa00700 1 -- 172.21.15.97:0/1024400460 >> 172.21.15.97:0/1024400460 conn(0x7f78740fde60 msgr2=0x7f78740febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:29.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.274+0000 7f787aa00700 1 -- 172.21.15.97:0/1024400460 shutdown_connections 2024-03-26T18:57:29.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.274+0000 7f787aa00700 1 -- 172.21.15.97:0/1024400460 wait complete. 2024-03-26T18:57:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-03-26T18:57:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-26T18:57:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-26T18:57:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-03-26T18:57:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-26T18:57:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-26T18:57:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-26T18:57:29.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.442+0000 7f9b58a00700 1 -- 172.21.15.97:0/1732024803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b54102dc0 msgr2=0x7f9b5410f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:29.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.442+0000 7f9b58a00700 1 --2- 172.21.15.97:0/1732024803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b54102dc0 0x7f9b5410f2c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00b0a0 tx=0x7f9b4c00b3b0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.443+0000 7f9b58a00700 1 -- 172.21.15.97:0/1732024803 shutdown_connections 2024-03-26T18:57:29.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.443+0000 7f9b58a00700 1 --2- 172.21.15.97:0/1732024803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b54102dc0 0x7f9b5410f2c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.443+0000 7f9b58a00700 1 --2- 172.21.15.97:0/1732024803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b54108480 0x7f9b54108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.443+0000 7f9b58a00700 1 --2- 172.21.15.97:0/1732024803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54102400 0x7f9b54102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.443+0000 7f9b58a00700 1 -- 172.21.15.97:0/1732024803 >> 172.21.15.97:0/1732024803 conn(0x7f9b54079510 msgr2=0x7f9b54079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:29.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.443+0000 7f9b58a00700 1 -- 172.21.15.97:0/1732024803 shutdown_connections 2024-03-26T18:57:29.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.443+0000 7f9b58a00700 1 -- 172.21.15.97:0/1732024803 wait complete. 2024-03-26T18:57:29.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.444+0000 7f9b58a00700 1 Processor -- start 2024-03-26T18:57:29.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.444+0000 7f9b58a00700 1 -- start start 2024-03-26T18:57:29.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.444+0000 7f9b58a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b54102400 0x7f9b5419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:29.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.444+0000 7f9b58a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b54102dc0 0x7f9b5419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:29.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.444+0000 7f9b58a00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54108480 0x7f9b541989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.444+0000 7f9b58a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b54112740 con 0x7f9b54102dc0 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.444+0000 7f9b58a00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9b541125c0 con 0x7f9b54102400 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.444+0000 7f9b58a00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b541128c0 con 0x7f9b54108480 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b51e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b54102400 0x7f9b5419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b51e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b54102400 0x7f9b5419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:43270/0 (socket says 172.21.15.97:43270) 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b51e00700 1 -- 172.21.15.97:0/3837493205 learned_addr learned my addr 172.21.15.97:0/3837493205 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b4be00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b54102dc0 0x7f9b5419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b52800700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54108480 0x7f9b541989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b51e00700 1 -- 172.21.15.97:0/3837493205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54108480 msgr2=0x7f9b541989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b51e00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54108480 0x7f9b541989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b51e00700 1 -- 172.21.15.97:0/3837493205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b54102dc0 msgr2=0x7f9b5419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b51e00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b54102dc0 0x7f9b5419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b51e00700 1 -- 172.21.15.97:0/3837493205 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b54199100 con 0x7f9b54102400 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.445+0000 7f9b52800700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54108480 0x7f9b541989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:57:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.446+0000 7f9b51e00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b54102400 0x7f9b5419e7e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400e9a0 tx=0x7f9b4400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:29.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.446+0000 7f9b4aa00700 1 -- 172.21.15.97:0/3837493205 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4400cbd0 con 0x7f9b54102400 2024-03-26T18:57:29.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.446+0000 7f9b4aa00700 1 -- 172.21.15.97:0/3837493205 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b4400cd70 con 0x7f9b54102400 2024-03-26T18:57:29.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.446+0000 7f9b58a00700 1 -- 172.21.15.97:0/3837493205 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b541993f0 con 0x7f9b54102400 2024-03-26T18:57:29.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.446+0000 7f9b4aa00700 1 -- 172.21.15.97:0/3837493205 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44018750 con 0x7f9b54102400 2024-03-26T18:57:29.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.446+0000 7f9b58a00700 1 -- 172.21.15.97:0/3837493205 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b541a56b0 con 0x7f9b54102400 2024-03-26T18:57:29.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.448+0000 7f9b58a00700 1 -- 172.21.15.97:0/3837493205 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b54069070 con 0x7f9b54102400 2024-03-26T18:57:29.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.449+0000 7f9b4aa00700 1 -- 172.21.15.97:0/3837493205 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f9b44010430 con 0x7f9b54102400 2024-03-26T18:57:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.451+0000 7f9b4aa00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9b380a30c0 0x7f9b380a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.451+0000 7f9b4aa00700 1 -- 172.21.15.97:0/3837493205 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8182+0+0 (secure 0 0 0) 0x7f9b440cf790 con 0x7f9b54102400 2024-03-26T18:57:29.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.451+0000 7f9b4be00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9b380a30c0 0x7f9b380a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:29.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.452+0000 7f9b4be00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9b380a30c0 0x7f9b380a5580 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b54079a80 tx=0x7f9b3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.456+0000 7f9b4aa00700 1 -- 172.21.15.97:0/3837493205 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9b4409c430 con 0x7f9b54102400 2024-03-26T18:57:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-03-26T18:57:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-26T18:57:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-26T18:57:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-03-26T18:57:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-26T18:57:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-26T18:57:29.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-26T18:57:29.579 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:29.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.716+0000 7f9b58a00700 1 -- 172.21.15.97:0/3837493205 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f9b5402dc00 con 0x7f9b54102400 2024-03-26T18:57:29.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.716+0000 7f9b4aa00700 1 -- 172.21.15.97:0/3837493205 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f9b4409bd30 con 0x7f9b54102400 2024-03-26T18:57:29.710 INFO:teuthology.orchestra.run.smithi097.stdout:343597383706 2024-03-26T18:57:29.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 -- 172.21.15.97:0/3837493205 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9b380a30c0 msgr2=0x7f9b380a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9b380a30c0 0x7f9b380a5580 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b54079a80 tx=0x7f9b3c009260 comp rx=0 tx=0).stop 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 -- 172.21.15.97:0/3837493205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b54102400 msgr2=0x7f9b5419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b54102400 0x7f9b5419e7e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400e9a0 tx=0x7f9b4400ecb0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 -- 172.21.15.97:0/3837493205 shutdown_connections 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9b380a30c0 0x7f9b380a5580 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b54108480 0x7f9b541989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b54102400 0x7f9b5419e7e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 --2- 172.21.15.97:0/3837493205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b54102dc0 0x7f9b5419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 -- 172.21.15.97:0/3837493205 >> 172.21.15.97:0/3837493205 conn(0x7f9b54079510 msgr2=0x7f9b541003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 -- 172.21.15.97:0/3837493205 shutdown_connections 2024-03-26T18:57:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:29.720+0000 7f9b58a00700 1 -- 172.21.15.97:0/3837493205 wait complete. 2024-03-26T18:57:30.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:57:30.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-03-26T18:57:30.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-26T18:57:30.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-26T18:57:30.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-03-26T18:57:30.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-26T18:57:30.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-26T18:57:30.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-26T18:57:30.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T18:57:30.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1024400460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T18:57:30.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:30 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3837493205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T18:57:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:57:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-03-26T18:57:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-26T18:57:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-26T18:57:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-03-26T18:57:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-26T18:57:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-26T18:57:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-26T18:57:30.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T18:57:30.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1024400460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T18:57:30.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:30 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3837493205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T18:57:30.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:57:30.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-03-26T18:57:30.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-26T18:57:30.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-26T18:57:30.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-03-26T18:57:30.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-26T18:57:30.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-26T18:57:30.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-26T18:57:30.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T18:57:30.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1024400460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T18:57:30.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:30 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3837493205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T18:57:31.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:31 smithi097 ceph-mon[31955]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T18:57:31.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:31 smithi144 ceph-mon[36053]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T18:57:31.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:31 smithi105 ceph-mon[36270]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T18:57:32.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:32 smithi097 ceph-mon[31955]: pgmap v348: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-26T18:57:32.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:32 smithi097 ceph-mon[31955]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-26T18:57:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:32 smithi144 ceph-mon[36053]: pgmap v348: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-26T18:57:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:32 smithi144 ceph-mon[36053]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-26T18:57:32.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:32 smithi105 ceph-mon[36270]: pgmap v348: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-26T18:57:32.483 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:32 smithi105 ceph-mon[36270]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-26T18:57:32.484 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645339 got 309237645343 for osd.10 2024-03-26T18:57:32.485 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:34.334 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:34 smithi097 ceph-mon[31955]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 13 B/s, 1 objects/s recovering 2024-03-26T18:57:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.374+0000 7f22f2400700 1 -- 172.21.15.97:0/880782160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec102fa0 msgr2=0x7f22ec106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.374+0000 7f22f2400700 1 --2- 172.21.15.97:0/880782160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec102fa0 0x7f22ec106ff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009990 tx=0x7f22d8009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.374+0000 7f22f2400700 1 -- 172.21.15.97:0/880782160 shutdown_connections 2024-03-26T18:57:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.374+0000 7f22f2400700 1 --2- 172.21.15.97:0/880782160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec1076d0 0x7f22ec10f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.374+0000 7f22f2400700 1 --2- 172.21.15.97:0/880782160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ec1025f0 0x7f22ec1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.374+0000 7f22f2400700 1 --2- 172.21.15.97:0/880782160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec102fa0 0x7f22ec106ff0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.374+0000 7f22f2400700 1 -- 172.21.15.97:0/880782160 >> 172.21.15.97:0/880782160 conn(0x7f22ec0fde60 msgr2=0x7f22ec100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.374+0000 7f22f2400700 1 -- 172.21.15.97:0/880782160 shutdown_connections 2024-03-26T18:57:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.374+0000 7f22f2400700 1 -- 172.21.15.97:0/880782160 wait complete. 2024-03-26T18:57:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.376+0000 7f22f2400700 1 Processor -- start 2024-03-26T18:57:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.376+0000 7f22f2400700 1 -- start start 2024-03-26T18:57:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.376+0000 7f22f2400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec1025f0 0x7f22ec0746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.376+0000 7f22f2400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec102fa0 0x7f22ec074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.376+0000 7f22f2400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ec1076d0 0x7f22ec075130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.376+0000 7f22f2400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec112700 con 0x7f22ec102fa0 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22f2400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec112580 con 0x7f22ec1025f0 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22f2400700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec112880 con 0x7f22ec1076d0 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22eb400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec1025f0 0x7f22ec0746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22eaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec102fa0 0x7f22ec074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22eb400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec1025f0 0x7f22ec0746b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60000/0 (socket says 172.21.15.97:60000) 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22eb400700 1 -- 172.21.15.97:0/3621477553 learned_addr learned my addr 172.21.15.97:0/3621477553 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22eb400700 1 -- 172.21.15.97:0/3621477553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ec1076d0 msgr2=0x7f22ec075130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22eb400700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ec1076d0 0x7f22ec075130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22eb400700 1 -- 172.21.15.97:0/3621477553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec102fa0 msgr2=0x7f22ec074bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22eb400700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec102fa0 0x7f22ec074bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22eb400700 1 -- 172.21.15.97:0/3621477553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec1a7120 con 0x7f22ec1025f0 2024-03-26T18:57:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.377+0000 7f22eb400700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec1025f0 0x7f22ec0746b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00e9c0 tx=0x7f22dc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.378+0000 7f22e3e00700 1 -- 172.21.15.97:0/3621477553 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00cbd0 con 0x7f22ec1025f0 2024-03-26T18:57:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.378+0000 7f22e3e00700 1 -- 172.21.15.97:0/3621477553 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22dc00cd70 con 0x7f22ec1025f0 2024-03-26T18:57:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.378+0000 7f22f2400700 1 -- 172.21.15.97:0/3621477553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec1a7410 con 0x7f22ec1025f0 2024-03-26T18:57:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.378+0000 7f22e3e00700 1 -- 172.21.15.97:0/3621477553 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc005660 con 0x7f22ec1025f0 2024-03-26T18:57:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.378+0000 7f22f2400700 1 -- 172.21.15.97:0/3621477553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec1a7920 con 0x7f22ec1025f0 2024-03-26T18:57:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.379+0000 7f22f2400700 1 -- 172.21.15.97:0/3621477553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ec069070 con 0x7f22ec1025f0 2024-03-26T18:57:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.381+0000 7f22e3e00700 1 -- 172.21.15.97:0/3621477553 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f22dc004750 con 0x7f22ec1025f0 2024-03-26T18:57:34.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:34 smithi144 ceph-mon[36053]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 13 B/s, 1 objects/s recovering 2024-03-26T18:57:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.383+0000 7f22e3e00700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22d00a30c0 0x7f22d00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.383+0000 7f22e3e00700 1 -- 172.21.15.97:0/3621477553 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f22dc0cf770 con 0x7f22ec1025f0 2024-03-26T18:57:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.383+0000 7f22eaa00700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22d00a30c0 0x7f22d00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.384+0000 7f22eaa00700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22d00a30c0 0x7f22d00a5580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f22ec076140 tx=0x7f22d800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:34.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.386+0000 7f22e3e00700 1 -- 172.21.15.97:0/3621477553 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f22dc09c410 con 0x7f22ec1025f0 2024-03-26T18:57:34.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:34 smithi105 ceph-mon[36270]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 13 B/s, 1 objects/s recovering 2024-03-26T18:57:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.628+0000 7f22f2400700 1 -- 172.21.15.97:0/3621477553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f22ec075be0 con 0x7f22ec1025f0 2024-03-26T18:57:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.628+0000 7f22e3e00700 1 -- 172.21.15.97:0/3621477553 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f22dc09bd10 con 0x7f22ec1025f0 2024-03-26T18:57:34.623 INFO:teuthology.orchestra.run.smithi097.stdout:197568495671 2024-03-26T18:57:34.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 -- 172.21.15.97:0/3621477553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22d00a30c0 msgr2=0x7f22d00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:34.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22d00a30c0 0x7f22d00a5580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f22ec076140 tx=0x7f22d800b260 comp rx=0 tx=0).stop 2024-03-26T18:57:34.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 -- 172.21.15.97:0/3621477553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec1025f0 msgr2=0x7f22ec0746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:34.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec1025f0 0x7f22ec0746b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00e9c0 tx=0x7f22dc00ecd0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 -- 172.21.15.97:0/3621477553 shutdown_connections 2024-03-26T18:57:34.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f22d00a30c0 0x7f22d00a5580 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ec1076d0 0x7f22ec075130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec1025f0 0x7f22ec0746b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 --2- 172.21.15.97:0/3621477553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec102fa0 0x7f22ec074bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:34.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 -- 172.21.15.97:0/3621477553 >> 172.21.15.97:0/3621477553 conn(0x7f22ec0fde60 msgr2=0x7f22ec0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:34.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 -- 172.21.15.97:0/3621477553 shutdown_connections 2024-03-26T18:57:34.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:34.632+0000 7f22f2400700 1 -- 172.21.15.97:0/3621477553 wait complete. 2024-03-26T18:57:35.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:35 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3621477553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T18:57:35.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:35 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3621477553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T18:57:35.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:35 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3621477553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T18:57:36.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:36 smithi097 ceph-mon[31955]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-03-26T18:57:36.315 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:36 smithi097 ceph-mon[31955]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-26T18:57:36.315 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:36 smithi097 ceph-mon[31955]: Cluster is now healthy 2024-03-26T18:57:36.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:36 smithi144 ceph-mon[36053]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-03-26T18:57:36.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:36 smithi144 ceph-mon[36053]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-26T18:57:36.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:36 smithi144 ceph-mon[36053]: Cluster is now healthy 2024-03-26T18:57:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:36 smithi105 ceph-mon[36270]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-03-26T18:57:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:36 smithi105 ceph-mon[36270]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-26T18:57:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:36 smithi105 ceph-mon[36270]: Cluster is now healthy 2024-03-26T18:57:38.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:38 smithi097 ceph-mon[31955]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-03-26T18:57:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:38 smithi144 ceph-mon[36053]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-03-26T18:57:38.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:38 smithi105 ceph-mon[36270]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-03-26T18:57:38.595 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495669 got 197568495671 for osd.6 2024-03-26T18:57:38.595 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:40 smithi097 ceph-mon[31955]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-03-26T18:57:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:40 smithi144 ceph-mon[36053]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-03-26T18:57:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:40 smithi105 ceph-mon[36270]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-03-26T18:57:40.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.745+0000 7f5363600700 1 -- 172.21.15.97:0/3907598434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f535c1072a0 msgr2=0x7f535c06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:40.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.745+0000 7f5363600700 1 --2- 172.21.15.97:0/3907598434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f535c1072a0 0x7f535c06a060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5350009990 tx=0x7f5350009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:40.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.745+0000 7f5363600700 1 -- 172.21.15.97:0/3907598434 shutdown_connections 2024-03-26T18:57:40.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.745+0000 7f5363600700 1 --2- 172.21.15.97:0/3907598434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c104db0 0x7f535c105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:40.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.745+0000 7f5363600700 1 --2- 172.21.15.97:0/3907598434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f535c1072a0 0x7f535c06a060 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:40.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.745+0000 7f5363600700 1 --2- 172.21.15.97:0/3907598434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c06a5a0 0x7f535c06aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:40.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.745+0000 7f5363600700 1 -- 172.21.15.97:0/3907598434 >> 172.21.15.97:0/3907598434 conn(0x7f535c0781e0 msgr2=0x7f535c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:40.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.745+0000 7f5363600700 1 -- 172.21.15.97:0/3907598434 shutdown_connections 2024-03-26T18:57:40.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.746+0000 7f5363600700 1 -- 172.21.15.97:0/3907598434 wait complete. 2024-03-26T18:57:40.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.747+0000 7f5363600700 1 Processor -- start 2024-03-26T18:57:40.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.747+0000 7f5363600700 1 -- start start 2024-03-26T18:57:40.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f5363600700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c06a5a0 0x7f535c19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:40.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f5363600700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f535c104db0 0x7f535c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:40.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f5363600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1072a0 0x7f535c1a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:40.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f5363600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f535c118c80 con 0x7f535c1072a0 2024-03-26T18:57:40.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f5363600700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f535c118b00 con 0x7f535c104db0 2024-03-26T18:57:40.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f5363600700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f535c118e00 con 0x7f535c06a5a0 2024-03-26T18:57:40.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f535be00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f535c104db0 0x7f535c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:40.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f535be00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f535c104db0 0x7f535c19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60028/0 (socket says 172.21.15.97:60028) 2024-03-26T18:57:40.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f535be00700 1 -- 172.21.15.97:0/901339129 learned_addr learned my addr 172.21.15.97:0/901339129 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:40.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f5361600700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1072a0 0x7f535c1a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:40.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.748+0000 7f5360c00700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c06a5a0 0x7f535c19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:40.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f535be00700 1 -- 172.21.15.97:0/901339129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c06a5a0 msgr2=0x7f535c19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f535be00700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c06a5a0 0x7f535c19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f535be00700 1 -- 172.21.15.97:0/901339129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1072a0 msgr2=0x7f535c1a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f535be00700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1072a0 0x7f535c1a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f535be00700 1 -- 172.21.15.97:0/901339129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535c1a4160 con 0x7f535c104db0 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f5360c00700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c06a5a0 0x7f535c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f535be00700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f535c104db0 0x7f535c19f690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f534800eb30 tx=0x7f534800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f5359600700 1 -- 172.21.15.97:0/901339129 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534800cbd0 con 0x7f535c104db0 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f5359600700 1 -- 172.21.15.97:0/901339129 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5348004500 con 0x7f535c104db0 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f5363600700 1 -- 172.21.15.97:0/901339129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535c1a4450 con 0x7f535c104db0 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.749+0000 7f5359600700 1 -- 172.21.15.97:0/901339129 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348010430 con 0x7f535c104db0 2024-03-26T18:57:40.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.750+0000 7f5363600700 1 -- 172.21.15.97:0/901339129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535c1abcd0 con 0x7f535c104db0 2024-03-26T18:57:40.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.751+0000 7f5363600700 1 -- 172.21.15.97:0/901339129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f535c069070 con 0x7f535c104db0 2024-03-26T18:57:40.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.753+0000 7f5359600700 1 -- 172.21.15.97:0/901339129 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f5348043b30 con 0x7f535c104db0 2024-03-26T18:57:40.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.754+0000 7f5359600700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f53440a30c0 0x7f53440a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:40.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.754+0000 7f5359600700 1 -- 172.21.15.97:0/901339129 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f53480cf6f0 con 0x7f535c104db0 2024-03-26T18:57:40.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.754+0000 7f5360c00700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f53440a30c0 0x7f53440a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:40.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.755+0000 7f5360c00700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f53440a30c0 0x7f53440a5580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5350006010 tx=0x7f535000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:40.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.756+0000 7f5359600700 1 -- 172.21.15.97:0/901339129 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5348043e20 con 0x7f535c104db0 2024-03-26T18:57:41.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.070+0000 7f5363600700 1 -- 172.21.15.97:0/901339129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f535c1abf60 con 0x7f535c104db0 2024-03-26T18:57:41.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.071+0000 7f5359600700 1 -- 172.21.15.97:0/901339129 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f534809c390 con 0x7f535c104db0 2024-03-26T18:57:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:77309411407 2024-03-26T18:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.074+0000 7f5363600700 1 -- 172.21.15.97:0/901339129 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f53440a30c0 msgr2=0x7f53440a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f53440a30c0 0x7f53440a5580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5350006010 tx=0x7f535000b260 comp rx=0 tx=0).stop 2024-03-26T18:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 -- 172.21.15.97:0/901339129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f535c104db0 msgr2=0x7f535c19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f535c104db0 0x7f535c19f690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f534800eb30 tx=0x7f534800ee40 comp rx=0 tx=0).stop 2024-03-26T18:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 -- 172.21.15.97:0/901339129 shutdown_connections 2024-03-26T18:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1072a0 0x7f535c1a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f53440a30c0 0x7f53440a5580 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c06a5a0 0x7f535c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 --2- 172.21.15.97:0/901339129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f535c104db0 0x7f535c19f690 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 -- 172.21.15.97:0/901339129 >> 172.21.15.97:0/901339129 conn(0x7f535c0781e0 msgr2=0x7f535c1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 -- 172.21.15.97:0/901339129 shutdown_connections 2024-03-26T18:57:41.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.075+0000 7f5363600700 1 -- 172.21.15.97:0/901339129 wait complete. 2024-03-26T18:57:41.256 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:41 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/583686615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T18:57:41.256 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:41 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/901339129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T18:57:41.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:41 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/583686615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T18:57:41.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:41 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/901339129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T18:57:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:41 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/583686615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T18:57:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:41 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/901339129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T18:57:41.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.541+0000 7f1ce7800700 1 -- 172.21.15.97:0/340713149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce0102fa0 msgr2=0x7f1ce0106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:41.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.541+0000 7f1ce7800700 1 --2- 172.21.15.97:0/340713149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce0102fa0 0x7f1ce0106ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0099c0 tx=0x7f1ccc009cd0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.541+0000 7f1ce7800700 1 -- 172.21.15.97:0/340713149 shutdown_connections 2024-03-26T18:57:41.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.541+0000 7f1ce7800700 1 --2- 172.21.15.97:0/340713149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce01076d0 0x7f1ce010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.541+0000 7f1ce7800700 1 --2- 172.21.15.97:0/340713149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce01025f0 0x7f1ce01029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.541+0000 7f1ce7800700 1 --2- 172.21.15.97:0/340713149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce0102fa0 0x7f1ce0106ff0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.541+0000 7f1ce7800700 1 -- 172.21.15.97:0/340713149 >> 172.21.15.97:0/340713149 conn(0x7f1ce00fde60 msgr2=0x7f1ce0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:41.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.541+0000 7f1ce7800700 1 -- 172.21.15.97:0/340713149 shutdown_connections 2024-03-26T18:57:41.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.541+0000 7f1ce7800700 1 -- 172.21.15.97:0/340713149 wait complete. 2024-03-26T18:57:41.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.543+0000 7f1ce7800700 1 Processor -- start 2024-03-26T18:57:41.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.543+0000 7f1ce7800700 1 -- start start 2024-03-26T18:57:41.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.543+0000 7f1ce7800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce01025f0 0x7f1ce019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:41.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.543+0000 7f1ce7800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0102fa0 0x7f1ce019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:41.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce7800700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce01076d0 0x7f1ce01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:41.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce7800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce0112700 con 0x7f1ce0102fa0 2024-03-26T18:57:41.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce7800700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce0112580 con 0x7f1ce01025f0 2024-03-26T18:57:41.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce7800700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce0112880 con 0x7f1ce01076d0 2024-03-26T18:57:41.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce4e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce01025f0 0x7f1ce019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:41.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce4e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce01025f0 0x7f1ce019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60656/0 (socket says 172.21.15.97:60656) 2024-03-26T18:57:41.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce4e00700 1 -- 172.21.15.97:0/365450490 learned_addr learned my addr 172.21.15.97:0/365450490 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:41.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1cdfe00700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0102fa0 0x7f1ce019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:41.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce5800700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce01076d0 0x7f1ce01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce5800700 1 -- 172.21.15.97:0/365450490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce01025f0 msgr2=0x7f1ce019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce5800700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce01025f0 0x7f1ce019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce5800700 1 -- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0102fa0 msgr2=0x7f1ce019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce5800700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0102fa0 0x7f1ce019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1ce5800700 1 -- 172.21.15.97:0/365450490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce0199140 con 0x7f1ce01076d0 2024-03-26T18:57:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.544+0000 7f1cdfe00700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0102fa0 0x7f1ce019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:57:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.545+0000 7f1ce5800700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce01076d0 0x7f1ce01989e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800efc0 tx=0x7f1cd800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:41.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.545+0000 7f1cdd600700 1 -- 172.21.15.97:0/365450490 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd80092e0 con 0x7f1ce01076d0 2024-03-26T18:57:41.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.545+0000 7f1cdd600700 1 -- 172.21.15.97:0/365450490 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd8004bb0 con 0x7f1ce01076d0 2024-03-26T18:57:41.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.545+0000 7f1cdd600700 1 -- 172.21.15.97:0/365450490 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8004280 con 0x7f1ce01076d0 2024-03-26T18:57:41.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.545+0000 7f1ce7800700 1 -- 172.21.15.97:0/365450490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce01993d0 con 0x7f1ce01076d0 2024-03-26T18:57:41.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.545+0000 7f1ce7800700 1 -- 172.21.15.97:0/365450490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce01a56a0 con 0x7f1ce01076d0 2024-03-26T18:57:41.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.547+0000 7f1ce7800700 1 -- 172.21.15.97:0/365450490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce0103ad0 con 0x7f1ce01076d0 2024-03-26T18:57:41.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.549+0000 7f1cdd600700 1 -- 172.21.15.97:0/365450490 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f1cd8012050 con 0x7f1ce01076d0 2024-03-26T18:57:41.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.551+0000 7f1cdd600700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1cc80a30c0 0x7f1cc80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:41.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.551+0000 7f1cdd600700 1 -- 172.21.15.97:0/365450490 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1cd80ceb90 con 0x7f1ce01076d0 2024-03-26T18:57:41.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.551+0000 7f1ce4e00700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1cc80a30c0 0x7f1cc80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:41.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.552+0000 7f1ce4e00700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1cc80a30c0 0x7f1cc80a5580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00fe300 tx=0x7f1cd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:41.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.552+0000 7f1cdd600700 1 -- 172.21.15.97:0/365450490 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1cd8010040 con 0x7f1ce01076d0 2024-03-26T18:57:41.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.810+0000 7f1ce7800700 1 -- 172.21.15.97:0/365450490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1ce002dc00 con 0x7f1ce01076d0 2024-03-26T18:57:41.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.811+0000 7f1cdd600700 1 -- 172.21.15.97:0/365450490 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1cd809b830 con 0x7f1ce01076d0 2024-03-26T18:57:41.805 INFO:teuthology.orchestra.run.smithi097.stdout:163208757312 2024-03-26T18:57:41.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 -- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1cc80a30c0 msgr2=0x7f1cc80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:41.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1cc80a30c0 0x7f1cc80a5580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00fe300 tx=0x7f1cd4009260 comp rx=0 tx=0).stop 2024-03-26T18:57:41.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 -- 172.21.15.97:0/365450490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce01076d0 msgr2=0x7f1ce01989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:41.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce01076d0 0x7f1ce01989e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800efc0 tx=0x7f1cd800c3a0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 -- 172.21.15.97:0/365450490 shutdown_connections 2024-03-26T18:57:41.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1cc80a30c0 0x7f1cc80a5580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce01076d0 0x7f1ce01989e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce01025f0 0x7f1ce019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 --2- 172.21.15.97:0/365450490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0102fa0 0x7f1ce019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:41.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 -- 172.21.15.97:0/365450490 >> 172.21.15.97:0/365450490 conn(0x7f1ce00fde60 msgr2=0x7f1ce0100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:41.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 -- 172.21.15.97:0/365450490 shutdown_connections 2024-03-26T18:57:41.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.814+0000 7f1ce7800700 1 -- 172.21.15.97:0/365450490 wait complete. 2024-03-26T18:57:42.043 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383701 got 343597383706 for osd.11 2024-03-26T18:57:42.044 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.071+0000 7f758fa00700 1 -- 172.21.15.97:0/1972198359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75881072f0 msgr2=0x7f75881076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.071+0000 7f758fa00700 1 --2- 172.21.15.97:0/1972198359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75881072f0 0x7f75881076d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f757c009990 tx=0x7f757c009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.072+0000 7f758fa00700 1 -- 172.21.15.97:0/1972198359 shutdown_connections 2024-03-26T18:57:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.072+0000 7f758fa00700 1 --2- 172.21.15.97:0/1972198359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7588101c50 0x7f758810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.072+0000 7f758fa00700 1 --2- 172.21.15.97:0/1972198359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75881072f0 0x7f75881076d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.072+0000 7f758fa00700 1 --2- 172.21.15.97:0/1972198359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75881012b0 0x7f7588101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.072+0000 7f758fa00700 1 -- 172.21.15.97:0/1972198359 >> 172.21.15.97:0/1972198359 conn(0x7f75880781e0 msgr2=0x7f75880785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.073+0000 7f758fa00700 1 -- 172.21.15.97:0/1972198359 shutdown_connections 2024-03-26T18:57:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.073+0000 7f758fa00700 1 -- 172.21.15.97:0/1972198359 wait complete. 2024-03-26T18:57:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.074+0000 7f758fa00700 1 Processor -- start 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.074+0000 7f758fa00700 1 -- start start 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f758fa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75881012b0 0x7f7588073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f758fa00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588101c50 0x7f758806ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f758fa00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75881072f0 0x7f758806f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f758fa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f75881115a0 con 0x7f75881012b0 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f758fa00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7588111420 con 0x7f75881072f0 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f758fa00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7588111720 con 0x7f7588101c50 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f7587e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588101c50 0x7f758806ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f758d000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75881012b0 0x7f7588073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f7587e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588101c50 0x7f758806ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:41758/0 (socket says 172.21.15.97:41758) 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f7587e00700 1 -- 172.21.15.97:0/1994276984 learned_addr learned my addr 172.21.15.97:0/1994276984 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f758da00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75881072f0 0x7f758806f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f7587e00700 1 -- 172.21.15.97:0/1994276984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75881072f0 msgr2=0x7f758806f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f7587e00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75881072f0 0x7f758806f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f7587e00700 1 -- 172.21.15.97:0/1994276984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75881012b0 msgr2=0x7f7588073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.075+0000 7f7587e00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75881012b0 0x7f7588073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.076+0000 7f7587e00700 1 -- 172.21.15.97:0/1994276984 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758806f9d0 con 0x7f7588101c50 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.076+0000 7f758da00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75881072f0 0x7f758806f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.076+0000 7f7587e00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588101c50 0x7f758806ed60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f757400e9c0 tx=0x7f757400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.076+0000 7f7585600700 1 -- 172.21.15.97:0/1994276984 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757400cbd0 con 0x7f7588101c50 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.076+0000 7f7585600700 1 -- 172.21.15.97:0/1994276984 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757400cd70 con 0x7f7588101c50 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.076+0000 7f758fa00700 1 -- 172.21.15.97:0/1994276984 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75881adab0 con 0x7f7588101c50 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.076+0000 7f7585600700 1 -- 172.21.15.97:0/1994276984 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574010430 con 0x7f7588101c50 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.076+0000 7f758fa00700 1 -- 172.21.15.97:0/1994276984 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75881adf10 con 0x7f7588101c50 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.078+0000 7f758fa00700 1 -- 172.21.15.97:0/1994276984 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7588069070 con 0x7f7588101c50 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.080+0000 7f7585600700 1 -- 172.21.15.97:0/1994276984 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f7574004750 con 0x7f7588101c50 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.082+0000 7f7585600700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75700a30c0 0x7f75700a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.082+0000 7f7585600700 1 -- 172.21.15.97:0/1994276984 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f75740cf7a0 con 0x7f7588101c50 2024-03-26T18:57:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.082+0000 7f758d000700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75700a30c0 0x7f75700a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.083+0000 7f758d000700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75700a30c0 0x7f75700a5580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f757c00b2e0 tx=0x7f757c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.085+0000 7f7585600700 1 -- 172.21.15.97:0/1994276984 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f757409c440 con 0x7f7588101c50 2024-03-26T18:57:42.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.359+0000 7f758fa00700 1 -- 172.21.15.97:0/1994276984 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f758806a660 con 0x7f7588101c50 2024-03-26T18:57:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:42 smithi097 ceph-mon[31955]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-26T18:57:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:42 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/365450490' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T18:57:42.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.359+0000 7f7585600700 1 -- 172.21.15.97:0/1994276984 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f757409bd40 con 0x7f7588101c50 2024-03-26T18:57:42.353 INFO:teuthology.orchestra.run.smithi097.stdout:287762808871 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 -- 172.21.15.97:0/1994276984 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75700a30c0 msgr2=0x7f75700a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75700a30c0 0x7f75700a5580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f757c00b2e0 tx=0x7f757c005fd0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 -- 172.21.15.97:0/1994276984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588101c50 msgr2=0x7f758806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588101c50 0x7f758806ed60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f757400e9c0 tx=0x7f757400ecd0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 -- 172.21.15.97:0/1994276984 shutdown_connections 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75881072f0 0x7f758806f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75881012b0 0x7f7588073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75700a30c0 0x7f75700a5580 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 --2- 172.21.15.97:0/1994276984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588101c50 0x7f758806ed60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 -- 172.21.15.97:0/1994276984 >> 172.21.15.97:0/1994276984 conn(0x7f75880781e0 msgr2=0x7f75881000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 -- 172.21.15.97:0/1994276984 shutdown_connections 2024-03-26T18:57:42.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.363+0000 7f758fa00700 1 -- 172.21.15.97:0/1994276984 wait complete. 2024-03-26T18:57:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:42 smithi144 ceph-mon[36053]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-26T18:57:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:42 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/365450490' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T18:57:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:42 smithi105 ceph-mon[36270]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-26T18:57:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:42 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/365450490' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T18:57:42.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.846+0000 7f92ee000700 1 -- 172.21.15.97:0/3804787637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e80750a0 msgr2=0x7f92e810e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.846+0000 7f92ee000700 1 --2- 172.21.15.97:0/3804787637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e80750a0 0x7f92e810e730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009990 tx=0x7f92d4009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.846+0000 7f92ee000700 1 -- 172.21.15.97:0/3804787637 shutdown_connections 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.846+0000 7f92ee000700 1 --2- 172.21.15.97:0/3804787637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e810ec70 0x7f92e8111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.846+0000 7f92ee000700 1 --2- 172.21.15.97:0/3804787637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e8074780 0x7f92e8074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.846+0000 7f92ee000700 1 --2- 172.21.15.97:0/3804787637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e80750a0 0x7f92e810e730 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.846+0000 7f92ee000700 1 -- 172.21.15.97:0/3804787637 >> 172.21.15.97:0/3804787637 conn(0x7f92e80fde80 msgr2=0x7f92e81002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.846+0000 7f92ee000700 1 -- 172.21.15.97:0/3804787637 shutdown_connections 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.846+0000 7f92ee000700 1 -- 172.21.15.97:0/3804787637 wait complete. 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.847+0000 7f92ee000700 1 Processor -- start 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.847+0000 7f92ee000700 1 -- start start 2024-03-26T18:57:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92ee000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e8074780 0x7f92e819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92ee000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80750a0 0x7f92e819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92ee000700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e810ec70 0x7f92e819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92ee000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8114980 con 0x7f92e8074780 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92ee000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8114800 con 0x7f92e80750a0 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92ee000700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8114b00 con 0x7f92e810ec70 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92e7e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e810ec70 0x7f92e819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92e7e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e810ec70 0x7f92e819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53482/0 (socket says 172.21.15.97:53482) 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92e7e00700 1 -- 172.21.15.97:0/583686615 learned_addr learned my addr 172.21.15.97:0/583686615 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92e7400700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e8074780 0x7f92e819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.848+0000 7f92e6a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80750a0 0x7f92e819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.849+0000 7f92e7e00700 1 -- 172.21.15.97:0/583686615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80750a0 msgr2=0x7f92e819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.849+0000 7f92e7e00700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80750a0 0x7f92e819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.849+0000 7f92e7e00700 1 -- 172.21.15.97:0/583686615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e8074780 msgr2=0x7f92e819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.849+0000 7f92e7e00700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e8074780 0x7f92e819adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.849+0000 7f92e7e00700 1 -- 172.21.15.97:0/583686615 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e819fdc0 con 0x7f92e810ec70 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.849+0000 7f92e7e00700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e810ec70 0x7f92e819f680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f92e000ef90 tx=0x7f92e000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.849+0000 7f92dfe00700 1 -- 172.21.15.97:0/583686615 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e00092e0 con 0x7f92e810ec70 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.849+0000 7f92dfe00700 1 -- 172.21.15.97:0/583686615 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e0004bb0 con 0x7f92e810ec70 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.850+0000 7f92dfe00700 1 -- 172.21.15.97:0/583686615 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0004280 con 0x7f92e810ec70 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.850+0000 7f92ee000700 1 -- 172.21.15.97:0/583686615 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e81a0050 con 0x7f92e810ec70 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.850+0000 7f92ee000700 1 -- 172.21.15.97:0/583686615 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e81a7930 con 0x7f92e810ec70 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.851+0000 7f92ee000700 1 -- 172.21.15.97:0/583686615 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e8069070 con 0x7f92e810ec70 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.853+0000 7f92dfe00700 1 -- 172.21.15.97:0/583686615 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f92e001b030 con 0x7f92e810ec70 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.854+0000 7f92dfe00700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f92cc0a3040 0x7f92cc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.854+0000 7f92dfe00700 1 -- 172.21.15.97:0/583686615 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f92e00cf060 con 0x7f92e810ec70 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.854+0000 7f92e7400700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f92cc0a3040 0x7f92cc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.855+0000 7f92e7400700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f92cc0a3040 0x7f92cc0a5500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009680 tx=0x7f92d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:39.858+0000 7f92dfe00700 1 -- 172.21.15.97:0/583686615 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f92e0022020 con 0x7f92e810ec70 2024-03-26T18:57:42.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.144+0000 7f92ee000700 1 -- 172.21.15.97:0/583686615 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f92e81a7b00 con 0x7f92e810ec70 2024-03-26T18:57:42.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.145+0000 7f92dfe00700 1 -- 172.21.15.97:0/583686615 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f92e0010040 con 0x7f92e810ec70 2024-03-26T18:57:42.706 INFO:teuthology.orchestra.run.smithi097.stdout:107374182474 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.147+0000 7f92ee000700 1 -- 172.21.15.97:0/583686615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f92cc0a3040 msgr2=0x7f92cc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.147+0000 7f92ee000700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f92cc0a3040 0x7f92cc0a5500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009680 tx=0x7f92d8009260 comp rx=0 tx=0).stop 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.147+0000 7f92ee000700 1 -- 172.21.15.97:0/583686615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e810ec70 msgr2=0x7f92e819f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.147+0000 7f92ee000700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e810ec70 0x7f92e819f680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f92e000ef90 tx=0x7f92e000c3a0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.148+0000 7f92ee000700 1 -- 172.21.15.97:0/583686615 shutdown_connections 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.148+0000 7f92ee000700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f92cc0a3040 0x7f92cc0a5500 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.148+0000 7f92ee000700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e810ec70 0x7f92e819f680 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.148+0000 7f92ee000700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e8074780 0x7f92e819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.148+0000 7f92ee000700 1 --2- 172.21.15.97:0/583686615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80750a0 0x7f92e819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.148+0000 7f92ee000700 1 -- 172.21.15.97:0/583686615 >> 172.21.15.97:0/583686615 conn(0x7f92e80fde80 msgr2=0x7f92e8110e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.148+0000 7f92ee000700 1 -- 172.21.15.97:0/583686615 shutdown_connections 2024-03-26T18:57:42.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:40.148+0000 7f92ee000700 1 -- 172.21.15.97:0/583686615 wait complete. 2024-03-26T18:57:43.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:43 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2325501991' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T18:57:43.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:43 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1994276984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T18:57:43.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:43 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2325501991' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T18:57:43.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:43 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1994276984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T18:57:43.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:43 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2325501991' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T18:57:43.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:43 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1994276984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T18:57:43.693 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757304 got 163208757312 for osd.5 2024-03-26T18:57:43.693 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.031+0000 7fb36b200700 1 -- 172.21.15.97:0/1101082308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36406a980 msgr2=0x7fb36410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.031+0000 7fb36b200700 1 --2- 172.21.15.97:0/1101082308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36406a980 0x7fb36410e920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb350009990 tx=0x7fb350009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.031+0000 7fb36b200700 1 -- 172.21.15.97:0/1101082308 shutdown_connections 2024-03-26T18:57:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.031+0000 7fb36b200700 1 --2- 172.21.15.97:0/1101082308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb36410ee60 0x7fb364111250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.031+0000 7fb36b200700 1 --2- 172.21.15.97:0/1101082308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb36406a060 0x7fb36406a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.031+0000 7fb36b200700 1 --2- 172.21.15.97:0/1101082308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36406a980 0x7fb36410e920 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.031+0000 7fb36b200700 1 -- 172.21.15.97:0/1101082308 >> 172.21.15.97:0/1101082308 conn(0x7fb3640781e0 msgr2=0x7fb3640785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.031+0000 7fb36b200700 1 -- 172.21.15.97:0/1101082308 shutdown_connections 2024-03-26T18:57:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.031+0000 7fb36b200700 1 -- 172.21.15.97:0/1101082308 wait complete. 2024-03-26T18:57:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.032+0000 7fb36b200700 1 Processor -- start 2024-03-26T18:57:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.032+0000 7fb36b200700 1 -- start start 2024-03-26T18:57:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb36b200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36406a060 0x7fb36419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb36b200700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb36406a980 0x7fb36419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb36b200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb36410ee60 0x7fb36419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb36b200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3641148e0 con 0x7fb36410ee60 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb36b200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb364114760 con 0x7fb36406a060 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb36b200700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb364114a60 con 0x7fb36406a980 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb363e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36406a060 0x7fb36419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb363e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36406a060 0x7fb36419aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60714/0 (socket says 172.21.15.97:60714) 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb363e00700 1 -- 172.21.15.97:0/2741738815 learned_addr learned my addr 172.21.15.97:0/2741738815 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb363400700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb36406a980 0x7fb36419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb369200700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb36410ee60 0x7fb36419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb363e00700 1 -- 172.21.15.97:0/2741738815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb36406a980 msgr2=0x7fb36419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb363e00700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb36406a980 0x7fb36419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb363e00700 1 -- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb36410ee60 msgr2=0x7fb36419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb363e00700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb36410ee60 0x7fb36419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb363e00700 1 -- 172.21.15.97:0/2741738815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36419fef0 con 0x7fb36406a060 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.033+0000 7fb369200700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb36410ee60 0x7fb36419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:57:44.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.034+0000 7fb363e00700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36406a060 0x7fb36419aee0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb35400e9a0 tx=0x7fb35400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:44.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.034+0000 7fb360c00700 1 -- 172.21.15.97:0/2741738815 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35400cbd0 con 0x7fb36406a060 2024-03-26T18:57:44.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.034+0000 7fb360c00700 1 -- 172.21.15.97:0/2741738815 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35400cd70 con 0x7fb36406a060 2024-03-26T18:57:44.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.034+0000 7fb36b200700 1 -- 172.21.15.97:0/2741738815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3641a01e0 con 0x7fb36406a060 2024-03-26T18:57:44.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.034+0000 7fb360c00700 1 -- 172.21.15.97:0/2741738815 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354018750 con 0x7fb36406a060 2024-03-26T18:57:44.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.034+0000 7fb36b200700 1 -- 172.21.15.97:0/2741738815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3641a7a60 con 0x7fb36406a060 2024-03-26T18:57:44.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.035+0000 7fb36b200700 1 -- 172.21.15.97:0/2741738815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb364069070 con 0x7fb36406a060 2024-03-26T18:57:44.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.038+0000 7fb360c00700 1 -- 172.21.15.97:0/2741738815 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fb354010430 con 0x7fb36406a060 2024-03-26T18:57:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.040+0000 7fb360c00700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3480a30c0 0x7fb3480a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:44.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.040+0000 7fb360c00700 1 -- 172.21.15.97:0/2741738815 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb3540cf790 con 0x7fb36406a060 2024-03-26T18:57:44.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.040+0000 7fb363400700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3480a30c0 0x7fb3480a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:44.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.041+0000 7fb363400700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3480a30c0 0x7fb3480a5580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb364102730 tx=0x7fb35000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:44.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.044+0000 7fb360c00700 1 -- 172.21.15.97:0/2741738815 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb35409c490 con 0x7fb36406a060 2024-03-26T18:57:44.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:44 smithi097 ceph-mon[31955]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-03-26T18:57:44.311 INFO:teuthology.orchestra.run.smithi097.stdout:34359738459 2024-03-26T18:57:44.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.314+0000 7fb36b200700 1 -- 172.21.15.97:0/2741738815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb36402dc00 con 0x7fb36406a060 2024-03-26T18:57:44.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.315+0000 7fb360c00700 1 -- 172.21.15.97:0/2741738815 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb35409bd90 con 0x7fb36406a060 2024-03-26T18:57:44.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.318+0000 7fb36b200700 1 -- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3480a30c0 msgr2=0x7fb3480a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:44.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.318+0000 7fb36b200700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3480a30c0 0x7fb3480a5580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb364102730 tx=0x7fb35000b260 comp rx=0 tx=0).stop 2024-03-26T18:57:44.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.318+0000 7fb36b200700 1 -- 172.21.15.97:0/2741738815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36406a060 msgr2=0x7fb36419aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:44.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.318+0000 7fb36b200700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36406a060 0x7fb36419aee0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb35400e9a0 tx=0x7fb35400ecb0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.319+0000 7fb36b200700 1 -- 172.21.15.97:0/2741738815 shutdown_connections 2024-03-26T18:57:44.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.319+0000 7fb36b200700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb36410ee60 0x7fb36419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.319+0000 7fb36b200700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36406a060 0x7fb36419aee0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.319+0000 7fb36b200700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3480a30c0 0x7fb3480a5580 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.319+0000 7fb36b200700 1 --2- 172.21.15.97:0/2741738815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb36406a980 0x7fb36419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.319+0000 7fb36b200700 1 -- 172.21.15.97:0/2741738815 >> 172.21.15.97:0/2741738815 conn(0x7fb3640781e0 msgr2=0x7fb364110270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:44.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.319+0000 7fb36b200700 1 -- 172.21.15.97:0/2741738815 shutdown_connections 2024-03-26T18:57:44.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:44.319+0000 7fb36b200700 1 -- 172.21.15.97:0/2741738815 wait complete. 2024-03-26T18:57:44.357 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182468 got 107374182474 for osd.3 2024-03-26T18:57:44.358 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:44.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:44 smithi144 ceph-mon[36053]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-03-26T18:57:44.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:44 smithi105 ceph-mon[36270]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-03-26T18:57:44.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.973+0000 7f0f40a00700 1 -- 172.21.15.97:0/2945780919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c102dc0 msgr2=0x7f0f3c10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:44.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.973+0000 7f0f40a00700 1 --2- 172.21.15.97:0/2945780919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c102dc0 0x7f0f3c10f2c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400b0a0 tx=0x7f0f3400b3b0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.974+0000 7f0f40a00700 1 -- 172.21.15.97:0/2945780919 shutdown_connections 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.974+0000 7f0f40a00700 1 --2- 172.21.15.97:0/2945780919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c102dc0 0x7f0f3c10f2c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.974+0000 7f0f40a00700 1 --2- 172.21.15.97:0/2945780919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f3c108480 0x7f0f3c108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.974+0000 7f0f40a00700 1 --2- 172.21.15.97:0/2945780919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f3c102400 0x7f0f3c102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.974+0000 7f0f40a00700 1 -- 172.21.15.97:0/2945780919 >> 172.21.15.97:0/2945780919 conn(0x7f0f3c079510 msgr2=0x7f0f3c079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.974+0000 7f0f40a00700 1 -- 172.21.15.97:0/2945780919 shutdown_connections 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.974+0000 7f0f40a00700 1 -- 172.21.15.97:0/2945780919 wait complete. 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.975+0000 7f0f40a00700 1 Processor -- start 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.975+0000 7f0f40a00700 1 -- start start 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f40a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c102400 0x7f0f3c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f40a00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f3c102dc0 0x7f0f3c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f40a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f3c108480 0x7f0f3c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f40a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c112740 con 0x7f0f3c108480 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f40a00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1125c0 con 0x7f0f3c102400 2024-03-26T18:57:44.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f40a00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1128c0 con 0x7f0f3c102dc0 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f39e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c102400 0x7f0f3c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f39400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f3c102dc0 0x7f0f3c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f39e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c102400 0x7f0f3c19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60680/0 (socket says 172.21.15.97:60680) 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f39e00700 1 -- 172.21.15.97:0/2325501991 learned_addr learned my addr 172.21.15.97:0/2325501991 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f3a800700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f3c108480 0x7f0f3c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f39400700 1 -- 172.21.15.97:0/2325501991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c102400 msgr2=0x7f0f3c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f39400700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c102400 0x7f0f3c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f39400700 1 -- 172.21.15.97:0/2325501991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f3c108480 msgr2=0x7f0f3c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f39400700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f3c108480 0x7f0f3c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.976+0000 7f0f39400700 1 -- 172.21.15.97:0/2325501991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c199100 con 0x7f0f3c102dc0 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.977+0000 7f0f39400700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f3c102dc0 0x7f0f3c19ed20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0f2800e9f0 tx=0x7f0f2800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.977+0000 7f0f32a00700 1 -- 172.21.15.97:0/2325501991 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2800cb70 con 0x7f0f3c102dc0 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.977+0000 7f0f32a00700 1 -- 172.21.15.97:0/2325501991 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f28004500 con 0x7f0f3c102dc0 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.977+0000 7f0f32a00700 1 -- 172.21.15.97:0/2325501991 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28010430 con 0x7f0f3c102dc0 2024-03-26T18:57:44.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.977+0000 7f0f40a00700 1 -- 172.21.15.97:0/2325501991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c1993f0 con 0x7f0f3c102dc0 2024-03-26T18:57:44.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.977+0000 7f0f40a00700 1 -- 172.21.15.97:0/2325501991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c1a56b0 con 0x7f0f3c102dc0 2024-03-26T18:57:44.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.978+0000 7f0f40a00700 1 -- 172.21.15.97:0/2325501991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f3c069070 con 0x7f0f3c102dc0 2024-03-26T18:57:44.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.980+0000 7f0f32a00700 1 -- 172.21.15.97:0/2325501991 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f0f28003e90 con 0x7f0f3c102dc0 2024-03-26T18:57:44.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.981+0000 7f0f32a00700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f200a3040 0x7f0f200a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:44.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.982+0000 7f0f32a00700 1 -- 172.21.15.97:0/2325501991 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0f280cf880 con 0x7f0f3c102dc0 2024-03-26T18:57:44.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.982+0000 7f0f39e00700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f200a3040 0x7f0f200a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:44.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.983+0000 7f0f39e00700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f200a3040 0x7f0f200a5500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c199ae0 tx=0x7f0f2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:44.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:41.983+0000 7f0f32a00700 1 -- 172.21.15.97:0/2325501991 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0f2809c5b0 con 0x7f0f3c102dc0 2024-03-26T18:57:44.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.258+0000 7f0f40a00700 1 -- 172.21.15.97:0/2325501991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f0f3c199f00 con 0x7f0f3c102dc0 2024-03-26T18:57:44.921 INFO:teuthology.orchestra.run.smithi097.stdout:253403070510 2024-03-26T18:57:44.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.259+0000 7f0f32a00700 1 -- 172.21.15.97:0/2325501991 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0f3c199f00 con 0x7f0f3c102dc0 2024-03-26T18:57:44.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.261+0000 7f0f40a00700 1 -- 172.21.15.97:0/2325501991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f200a3040 msgr2=0x7f0f200a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:44.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.261+0000 7f0f40a00700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f200a3040 0x7f0f200a5500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c199ae0 tx=0x7f0f2c009260 comp rx=0 tx=0).stop 2024-03-26T18:57:44.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.261+0000 7f0f40a00700 1 -- 172.21.15.97:0/2325501991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f3c102dc0 msgr2=0x7f0f3c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:44.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.261+0000 7f0f40a00700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f3c102dc0 0x7f0f3c19ed20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0f2800e9f0 tx=0x7f0f2800ed00 comp rx=0 tx=0).stop 2024-03-26T18:57:44.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.262+0000 7f0f40a00700 1 -- 172.21.15.97:0/2325501991 shutdown_connections 2024-03-26T18:57:44.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.262+0000 7f0f40a00700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f3c108480 0x7f0f3c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.262+0000 7f0f40a00700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c102400 0x7f0f3c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.262+0000 7f0f40a00700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f200a3040 0x7f0f200a5500 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.262+0000 7f0f40a00700 1 --2- 172.21.15.97:0/2325501991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f3c102dc0 0x7f0f3c19ed20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:44.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.262+0000 7f0f40a00700 1 -- 172.21.15.97:0/2325501991 >> 172.21.15.97:0/2325501991 conn(0x7f0f3c079510 msgr2=0x7f0f3c1003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:44.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.262+0000 7f0f40a00700 1 -- 172.21.15.97:0/2325501991 shutdown_connections 2024-03-26T18:57:44.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:42.262+0000 7f0f40a00700 1 -- 172.21.15.97:0/2325501991 wait complete. 2024-03-26T18:57:45.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:45 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2741738815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T18:57:45.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:45 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2741738815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T18:57:45.514 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:45 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2741738815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T18:57:46.372 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:46 smithi097 ceph-mon[31955]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:57:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:46 smithi105 ceph-mon[36270]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:57:46.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070510 for osd.8 2024-03-26T18:57:46.592 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:46.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:46 smithi144 ceph-mon[36053]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:57:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.847+0000 7fba08600700 1 -- 172.21.15.97:0/159248358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba00102690 msgr2=0x7fba00102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.847+0000 7fba08600700 1 --2- 172.21.15.97:0/159248358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba00102690 0x7fba00102a70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009990 tx=0x7fb9f4009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.847+0000 7fba08600700 1 -- 172.21.15.97:0/159248358 shutdown_connections 2024-03-26T18:57:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.847+0000 7fba08600700 1 --2- 172.21.15.97:0/159248358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba00107740 0x7fba0010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.847+0000 7fba08600700 1 --2- 172.21.15.97:0/159248358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba00102690 0x7fba00102a70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.847+0000 7fba08600700 1 --2- 172.21.15.97:0/159248358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba00103040 0x7fba00107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.847+0000 7fba08600700 1 -- 172.21.15.97:0/159248358 >> 172.21.15.97:0/159248358 conn(0x7fba00079480 msgr2=0x7fba00079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.847+0000 7fba08600700 1 -- 172.21.15.97:0/159248358 shutdown_connections 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.847+0000 7fba08600700 1 -- 172.21.15.97:0/159248358 wait complete. 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.849+0000 7fba08600700 1 Processor -- start 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.849+0000 7fba08600700 1 -- start start 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba08600700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba00102690 0x7fba0019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba08600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba00103040 0x7fba0019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba08600700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba00107740 0x7fba0019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba08600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba00074d40 con 0x7fba00103040 2024-03-26T18:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba08600700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba00074bc0 con 0x7fba00102690 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba08600700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba00074ec0 con 0x7fba00107740 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba05c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba00102690 0x7fba0019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba05c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba00102690 0x7fba0019b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60734/0 (socket says 172.21.15.97:60734) 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba05c00700 1 -- 172.21.15.97:0/2362263529 learned_addr learned my addr 172.21.15.97:0/2362263529 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba06600700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba00107740 0x7fba0019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba05200700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba00103040 0x7fba0019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba05c00700 1 -- 172.21.15.97:0/2362263529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba00107740 msgr2=0x7fba0019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.850+0000 7fba05c00700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba00107740 0x7fba0019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.851+0000 7fba05c00700 1 -- 172.21.15.97:0/2362263529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba00103040 msgr2=0x7fba0019b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.851+0000 7fba05c00700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba00103040 0x7fba0019b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.851+0000 7fba05c00700 1 -- 172.21.15.97:0/2362263529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0019fc60 con 0x7fba00102690 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.851+0000 7fba05c00700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba00102690 0x7fba0019b240 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4005910 tx=0x7fb9f400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.851+0000 7fb9fe000700 1 -- 172.21.15.97:0/2362263529 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4027070 con 0x7fba00102690 2024-03-26T18:57:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.851+0000 7fb9fe000700 1 -- 172.21.15.97:0/2362263529 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f4004650 con 0x7fba00102690 2024-03-26T18:57:46.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.851+0000 7fb9fe000700 1 -- 172.21.15.97:0/2362263529 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400f460 con 0x7fba00102690 2024-03-26T18:57:46.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.851+0000 7fba08600700 1 -- 172.21.15.97:0/2362263529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0019fef0 con 0x7fba00102690 2024-03-26T18:57:46.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.851+0000 7fba08600700 1 -- 172.21.15.97:0/2362263529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba001a0380 con 0x7fba00102690 2024-03-26T18:57:46.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.852+0000 7fba08600700 1 -- 172.21.15.97:0/2362263529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba00069070 con 0x7fba00102690 2024-03-26T18:57:46.918 INFO:teuthology.orchestra.run.smithi097.stdout:55834574933 2024-03-26T18:57:46.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.854+0000 7fb9fe000700 1 -- 172.21.15.97:0/2362263529 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fb9f402a070 con 0x7fba00102690 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.856+0000 7fb9fe000700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb9e80a30c0 0x7fb9e80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.857+0000 7fb9fe000700 1 -- 172.21.15.97:0/2362263529 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb9f40d0850 con 0x7fba00102690 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.857+0000 7fba05200700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb9e80a30c0 0x7fb9e80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.858+0000 7fb9fe000700 1 -- 172.21.15.97:0/2362263529 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb9f4031070 con 0x7fba00102690 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:45.858+0000 7fba05200700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb9e80a30c0 0x7fb9e80a5580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba00079a80 tx=0x7fb9ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.175+0000 7fba08600700 1 -- 172.21.15.97:0/2362263529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fba001a78e0 con 0x7fba00102690 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.175+0000 7fb9fe000700 1 -- 172.21.15.97:0/2362263529 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb9f4013070 con 0x7fba00102690 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 -- 172.21.15.97:0/2362263529 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb9e80a30c0 msgr2=0x7fb9e80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb9e80a30c0 0x7fb9e80a5580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba00079a80 tx=0x7fb9ec009260 comp rx=0 tx=0).stop 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 -- 172.21.15.97:0/2362263529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba00102690 msgr2=0x7fba0019b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba00102690 0x7fba0019b240 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4005910 tx=0x7fb9f400bab0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 -- 172.21.15.97:0/2362263529 shutdown_connections 2024-03-26T18:57:46.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb9e80a30c0 0x7fb9e80a5580 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba00107740 0x7fba0019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba00102690 0x7fba0019b240 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 --2- 172.21.15.97:0/2362263529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba00103040 0x7fba0019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:46.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 -- 172.21.15.97:0/2362263529 >> 172.21.15.97:0/2362263529 conn(0x7fba00079480 msgr2=0x7fba00105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:46.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 -- 172.21.15.97:0/2362263529 shutdown_connections 2024-03-26T18:57:46.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:46.178+0000 7fba08600700 1 -- 172.21.15.97:0/2362263529 wait complete. 2024-03-26T18:57:47.277 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:47 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2362263529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T18:57:47.327 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411399 got 77309411407 for osd.2 2024-03-26T18:57:47.327 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:47.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:47 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2362263529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T18:57:47.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:47 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2362263529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T18:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:47.800 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808864 got 287762808871 for osd.9 2024-03-26T18:57:47.800 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:48.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:48 smithi097 ceph-mon[31955]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:48.440 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574933 for osd.1 2024-03-26T18:57:48.440 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:48 smithi105 ceph-mon[36270]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:48.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:48 smithi144 ceph-mon[36053]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:57:49.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.913+0000 7f272e800700 1 -- 172.21.15.97:0/2914165360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27280745e0 msgr2=0x7f27280749c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:49.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.913+0000 7f272e800700 1 --2- 172.21.15.97:0/2914165360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27280745e0 0x7f27280749c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2718009990 tx=0x7f2718009ca0 comp rx=0 tx=0).stop 2024-03-26T18:57:49.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.913+0000 7f272e800700 1 -- 172.21.15.97:0/2914165360 shutdown_connections 2024-03-26T18:57:49.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.913+0000 7f272e800700 1 --2- 172.21.15.97:0/2914165360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272810edb0 0x7f27281111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.913+0000 7f272e800700 1 --2- 172.21.15.97:0/2914165360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27280745e0 0x7f27280749c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.913+0000 7f272e800700 1 --2- 172.21.15.97:0/2914165360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2728074f00 0x7f272810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.913+0000 7f272e800700 1 -- 172.21.15.97:0/2914165360 >> 172.21.15.97:0/2914165360 conn(0x7f2728079510 msgr2=0x7f2728079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.913+0000 7f272e800700 1 -- 172.21.15.97:0/2914165360 shutdown_connections 2024-03-26T18:57:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.913+0000 7f272e800700 1 -- 172.21.15.97:0/2914165360 wait complete. 2024-03-26T18:57:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.914+0000 7f272e800700 1 Processor -- start 2024-03-26T18:57:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f272e800700 1 -- start start 2024-03-26T18:57:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f272e800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27280745e0 0x7f272819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f272e800700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2728074f00 0x7f272819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f272e800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272810edb0 0x7f272819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f272e800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27281149b0 con 0x7f27280745e0 2024-03-26T18:57:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f272e800700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2728114830 con 0x7f272810edb0 2024-03-26T18:57:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f272e800700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2728114b30 con 0x7f2728074f00 2024-03-26T18:57:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f2726a00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2728074f00 0x7f272819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f2726a00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2728074f00 0x7f272819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:41806/0 (socket says 172.21.15.97:41806) 2024-03-26T18:57:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f2726a00700 1 -- 172.21.15.97:0/3185078478 learned_addr learned my addr 172.21.15.97:0/3185078478 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.915+0000 7f2727e00700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272810edb0 0x7f272819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.916+0000 7f2727400700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27280745e0 0x7f272819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.916+0000 7f2726a00700 1 -- 172.21.15.97:0/3185078478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272810edb0 msgr2=0x7f272819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.916+0000 7f2726a00700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272810edb0 0x7f272819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.916+0000 7f2726a00700 1 -- 172.21.15.97:0/3185078478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27280745e0 msgr2=0x7f272819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.916+0000 7f2726a00700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27280745e0 0x7f272819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.916+0000 7f2726a00700 1 -- 172.21.15.97:0/3185078478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272819fed0 con 0x7f2728074f00 2024-03-26T18:57:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.916+0000 7f2727e00700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272810edb0 0x7f272819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:57:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.916+0000 7f2726a00700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2728074f00 0x7f272819b400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f271400e9a0 tx=0x7f271400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:49.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.917+0000 7f271fe00700 1 -- 172.21.15.97:0/3185078478 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271400ca10 con 0x7f2728074f00 2024-03-26T18:57:49.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.917+0000 7f271fe00700 1 -- 172.21.15.97:0/3185078478 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f271400cbb0 con 0x7f2728074f00 2024-03-26T18:57:49.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.917+0000 7f272e800700 1 -- 172.21.15.97:0/3185078478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27281a01c0 con 0x7f2728074f00 2024-03-26T18:57:49.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.917+0000 7f271fe00700 1 -- 172.21.15.97:0/3185078478 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714018680 con 0x7f2728074f00 2024-03-26T18:57:49.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.917+0000 7f272e800700 1 -- 172.21.15.97:0/3185078478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27281a7a40 con 0x7f2728074f00 2024-03-26T18:57:49.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.919+0000 7f272e800700 1 -- 172.21.15.97:0/3185078478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2728069070 con 0x7f2728074f00 2024-03-26T18:57:49.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.920+0000 7f271fe00700 1 -- 172.21.15.97:0/3185078478 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f2714010430 con 0x7f2728074f00 2024-03-26T18:57:49.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.922+0000 7f271fe00700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f270c0a30c0 0x7f270c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:49.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.922+0000 7f271fe00700 1 -- 172.21.15.97:0/3185078478 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f27140cf760 con 0x7f2728074f00 2024-03-26T18:57:49.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.922+0000 7f2727400700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f270c0a30c0 0x7f270c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:49.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.923+0000 7f2727400700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f270c0a30c0 0x7f270c0a5580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2718006010 tx=0x7f271800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:49.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:49.930+0000 7f271fe00700 1 -- 172.21.15.97:0/3185078478 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f271409c400 con 0x7f2728074f00 2024-03-26T18:57:50.067 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738459 for osd.0 2024-03-26T18:57:50.068 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:50.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.245+0000 7f272e800700 1 -- 172.21.15.97:0/3185078478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f27281a7c70 con 0x7f2728074f00 2024-03-26T18:57:50.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.246+0000 7f271fe00700 1 -- 172.21.15.97:0/3185078478 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f271409bd00 con 0x7f2728074f00 2024-03-26T18:57:50.239 INFO:teuthology.orchestra.run.smithi097.stdout:219043332149 2024-03-26T18:57:50.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.248+0000 7f272e800700 1 -- 172.21.15.97:0/3185078478 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f270c0a30c0 msgr2=0x7f270c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:50.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.248+0000 7f272e800700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f270c0a30c0 0x7f270c0a5580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2718006010 tx=0x7f271800b260 comp rx=0 tx=0).stop 2024-03-26T18:57:50.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.248+0000 7f272e800700 1 -- 172.21.15.97:0/3185078478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2728074f00 msgr2=0x7f272819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:50.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.248+0000 7f272e800700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2728074f00 0x7f272819b400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f271400e9a0 tx=0x7f271400ecb0 comp rx=0 tx=0).stop 2024-03-26T18:57:50.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.248+0000 7f272e800700 1 -- 172.21.15.97:0/3185078478 shutdown_connections 2024-03-26T18:57:50.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.248+0000 7f272e800700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272810edb0 0x7f272819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:50.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.248+0000 7f272e800700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27280745e0 0x7f272819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:50.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.248+0000 7f272e800700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f270c0a30c0 0x7f270c0a5580 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:50.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.248+0000 7f272e800700 1 --2- 172.21.15.97:0/3185078478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2728074f00 0x7f272819b400 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:50.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.249+0000 7f272e800700 1 -- 172.21.15.97:0/3185078478 >> 172.21.15.97:0/3185078478 conn(0x7f2728079510 msgr2=0x7f2728100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:50.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.249+0000 7f272e800700 1 -- 172.21.15.97:0/3185078478 shutdown_connections 2024-03-26T18:57:50.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:50.249+0000 7f272e800700 1 -- 172.21.15.97:0/3185078478 wait complete. 2024-03-26T18:57:50.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:50 smithi097 ceph-mon[31955]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:57:50.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:50 smithi105 ceph-mon[36270]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:57:50.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:50 smithi144 ceph-mon[36053]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:57:51.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:51 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3185078478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T18:57:51.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:51 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3185078478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T18:57:51.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:51 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3185078478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T18:57:51.739 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332142 got 219043332149 for osd.7 2024-03-26T18:57:51.739 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:52.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.199+0000 7fdf48200700 1 -- 172.21.15.97:0/4051731499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf40074f00 msgr2=0x7fdf40113040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:52.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.199+0000 7fdf48200700 1 --2- 172.21.15.97:0/4051731499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf40074f00 0x7fdf40113040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0098f0 tx=0x7fdf2c009c00 comp rx=0 tx=0).stop 2024-03-26T18:57:52.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.199+0000 7fdf48200700 1 -- 172.21.15.97:0/4051731499 shutdown_connections 2024-03-26T18:57:52.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.199+0000 7fdf48200700 1 --2- 172.21.15.97:0/4051731499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf40113580 0x7fdf40115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:52.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.199+0000 7fdf48200700 1 --2- 172.21.15.97:0/4051731499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf400745e0 0x7fdf400749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:52.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.199+0000 7fdf48200700 1 --2- 172.21.15.97:0/4051731499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf40074f00 0x7fdf40113040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:52.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.199+0000 7fdf48200700 1 -- 172.21.15.97:0/4051731499 >> 172.21.15.97:0/4051731499 conn(0x7fdf40079510 msgr2=0x7fdf40079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:52.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.199+0000 7fdf48200700 1 -- 172.21.15.97:0/4051731499 shutdown_connections 2024-03-26T18:57:52.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.199+0000 7fdf48200700 1 -- 172.21.15.97:0/4051731499 wait complete. 2024-03-26T18:57:52.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.200+0000 7fdf48200700 1 Processor -- start 2024-03-26T18:57:52.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf48200700 1 -- start start 2024-03-26T18:57:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf48200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf400745e0 0x7fdf4019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf48200700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf40074f00 0x7fdf4019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf48200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40113580 0x7fdf401a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf48200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdf40118dc0 con 0x7fdf40113580 2024-03-26T18:57:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf48200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf40118c40 con 0x7fdf400745e0 2024-03-26T18:57:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf48200700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf40118f40 con 0x7fdf40074f00 2024-03-26T18:57:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf44e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf40074f00 0x7fdf4019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf44e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf40074f00 0x7fdf4019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:47914/0 (socket says 172.21.15.97:47914) 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf44e00700 1 -- 172.21.15.97:0/892451511 learned_addr learned my addr 172.21.15.97:0/892451511 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf45800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf400745e0 0x7fdf4019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.201+0000 7fdf46200700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40113580 0x7fdf401a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.202+0000 7fdf44e00700 1 -- 172.21.15.97:0/892451511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf400745e0 msgr2=0x7fdf4019f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.202+0000 7fdf44e00700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf400745e0 0x7fdf4019f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.202+0000 7fdf44e00700 1 -- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40113580 msgr2=0x7fdf401a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.202+0000 7fdf44e00700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40113580 0x7fdf401a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.202+0000 7fdf44e00700 1 -- 172.21.15.97:0/892451511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf401a42c0 con 0x7fdf40074f00 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.202+0000 7fdf45800700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf400745e0 0x7fdf4019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.202+0000 7fdf46200700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40113580 0x7fdf401a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:57:52.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.202+0000 7fdf44e00700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf40074f00 0x7fdf4019f7f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009fd0 tx=0x7fdf2c0094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:52.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.203+0000 7fdf3e000700 1 -- 172.21.15.97:0/892451511 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c025070 con 0x7fdf40074f00 2024-03-26T18:57:52.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.203+0000 7fdf48200700 1 -- 172.21.15.97:0/892451511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf401a4550 con 0x7fdf40074f00 2024-03-26T18:57:52.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.203+0000 7fdf3e000700 1 -- 172.21.15.97:0/892451511 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c00ba40 con 0x7fdf40074f00 2024-03-26T18:57:52.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.203+0000 7fdf3e000700 1 -- 172.21.15.97:0/892451511 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c004b90 con 0x7fdf40074f00 2024-03-26T18:57:52.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.203+0000 7fdf48200700 1 -- 172.21.15.97:0/892451511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf401abe30 con 0x7fdf40074f00 2024-03-26T18:57:52.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.204+0000 7fdf48200700 1 -- 172.21.15.97:0/892451511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf40069070 con 0x7fdf40074f00 2024-03-26T18:57:52.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.206+0000 7fdf3e000700 1 -- 172.21.15.97:0/892451511 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fdf2c028070 con 0x7fdf40074f00 2024-03-26T18:57:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.208+0000 7fdf3e000700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdf280a3040 0x7fdf280a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:52.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.208+0000 7fdf3e000700 1 -- 172.21.15.97:0/892451511 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fdf2c0cfff0 con 0x7fdf40074f00 2024-03-26T18:57:52.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.208+0000 7fdf45800700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdf280a3040 0x7fdf280a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.209+0000 7fdf45800700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdf280a3040 0x7fdf280a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdf34005fd0 tx=0x7fdf34009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.209+0000 7fdf3e000700 1 -- 172.21.15.97:0/892451511 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdf2c013030 con 0x7fdf40074f00 2024-03-26T18:57:52.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:52 smithi097 ceph-mon[31955]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:57:52.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:52 smithi105 ceph-mon[36270]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:57:52.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.503+0000 7fdf48200700 1 -- 172.21.15.97:0/892451511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdf401ac000 con 0x7fdf40074f00 2024-03-26T18:57:52.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.503+0000 7fdf3e000700 1 -- 172.21.15.97:0/892451511 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdf2c09cd20 con 0x7fdf40074f00 2024-03-26T18:57:52.496 INFO:teuthology.orchestra.run.smithi097.stdout:141733920839 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 -- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdf280a3040 msgr2=0x7fdf280a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdf280a3040 0x7fdf280a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdf34005fd0 tx=0x7fdf34009310 comp rx=0 tx=0).stop 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 -- 172.21.15.97:0/892451511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf40074f00 msgr2=0x7fdf4019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf40074f00 0x7fdf4019f7f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009fd0 tx=0x7fdf2c0094b0 comp rx=0 tx=0).stop 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 -- 172.21.15.97:0/892451511 shutdown_connections 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40113580 0x7fdf401a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf400745e0 0x7fdf4019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdf280a3040 0x7fdf280a5500 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 --2- 172.21.15.97:0/892451511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf40074f00 0x7fdf4019f7f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 -- 172.21.15.97:0/892451511 >> 172.21.15.97:0/892451511 conn(0x7fdf40079510 msgr2=0x7fdf4010f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 -- 172.21.15.97:0/892451511 shutdown_connections 2024-03-26T18:57:52.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:52.506+0000 7fdf48200700 1 -- 172.21.15.97:0/892451511 wait complete. 2024-03-26T18:57:52.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:52 smithi144 ceph-mon[36053]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:57:53.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:53 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/892451511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T18:57:53.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:53 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/892451511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T18:57:53.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:53 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/892451511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T18:57:53.747 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920833 got 141733920839 for osd.4 2024-03-26T18:57:53.747 DEBUG:teuthology.parallel:result is None 2024-03-26T18:57:53.747 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-26T18:57:53.748 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T18:57:54.252 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:54.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:54 smithi105 ceph-mon[36270]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:54.479 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:54 smithi097 ceph-mon[31955]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:54.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:54 smithi144 ceph-mon[36053]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:57:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:56 smithi105 ceph-mon[36270]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-03-26T18:57:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:56 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:56 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:56.517 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:56 smithi097 ceph-mon[31955]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-03-26T18:57:56.517 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:56 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:56.517 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:56 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:56 smithi144 ceph-mon[36053]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-03-26T18:57:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:56 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:56 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.701+0000 7fb474a00700 1 -- 172.21.15.97:0/269393131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb470108400 msgr2=0x7fb4701087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.701+0000 7fb474a00700 1 --2- 172.21.15.97:0/269393131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb470108400 0x7fb4701087e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb4600099c0 tx=0x7fb460009cd0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.701+0000 7fb474a00700 1 -- 172.21.15.97:0/269393131 shutdown_connections 2024-03-26T18:57:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.701+0000 7fb474a00700 1 --2- 172.21.15.97:0/269393131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb47006aa00 0x7fb47010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.701+0000 7fb474a00700 1 --2- 172.21.15.97:0/269393131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb470108400 0x7fb4701087e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.701+0000 7fb474a00700 1 --2- 172.21.15.97:0/269393131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb47006a060 0x7fb47006a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.701+0000 7fb474a00700 1 -- 172.21.15.97:0/269393131 >> 172.21.15.97:0/269393131 conn(0x7fb4700781e0 msgr2=0x7fb4700785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.701+0000 7fb474a00700 1 -- 172.21.15.97:0/269393131 shutdown_connections 2024-03-26T18:57:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.702+0000 7fb474a00700 1 -- 172.21.15.97:0/269393131 wait complete. 2024-03-26T18:57:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.703+0000 7fb474a00700 1 Processor -- start 2024-03-26T18:57:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.703+0000 7fb474a00700 1 -- start start 2024-03-26T18:57:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.703+0000 7fb474a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb47006a060 0x7fb47019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:56.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.703+0000 7fb474a00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb47006aa00 0x7fb47019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:56.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.703+0000 7fb474a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb470108400 0x7fb47019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.703+0000 7fb474a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb470112710 con 0x7fb470108400 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.703+0000 7fb474a00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb470112590 con 0x7fb47006a060 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.703+0000 7fb474a00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb470112890 con 0x7fb47006aa00 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46de00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb47006a060 0x7fb47019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46de00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb47006a060 0x7fb47019ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:48334/0 (socket says 172.21.15.97:48334) 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46de00700 1 -- 172.21.15.97:0/3797518144 learned_addr learned my addr 172.21.15.97:0/3797518144 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46d400700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb47006aa00 0x7fb47019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46e800700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb470108400 0x7fb47019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46d400700 1 -- 172.21.15.97:0/3797518144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb47006a060 msgr2=0x7fb47019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46d400700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb47006a060 0x7fb47019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46d400700 1 -- 172.21.15.97:0/3797518144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb470108400 msgr2=0x7fb47019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46d400700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb470108400 0x7fb47019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46d400700 1 -- 172.21.15.97:0/3797518144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47019fe50 con 0x7fb47006aa00 2024-03-26T18:57:56.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.704+0000 7fb46d400700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb47006aa00 0x7fb47019b380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00e9f0 tx=0x7fb45c00edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:56.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.705+0000 7fb466a00700 1 -- 172.21.15.97:0/3797518144 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00cbd0 con 0x7fb47006aa00 2024-03-26T18:57:56.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.705+0000 7fb466a00700 1 -- 172.21.15.97:0/3797518144 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb45c00cd70 con 0x7fb47006aa00 2024-03-26T18:57:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.705+0000 7fb474a00700 1 -- 172.21.15.97:0/3797518144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4701a0140 con 0x7fb47006aa00 2024-03-26T18:57:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.705+0000 7fb474a00700 1 -- 172.21.15.97:0/3797518144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4701a79c0 con 0x7fb47006aa00 2024-03-26T18:57:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.705+0000 7fb466a00700 1 -- 172.21.15.97:0/3797518144 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c018840 con 0x7fb47006aa00 2024-03-26T18:57:56.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.706+0000 7fb474a00700 1 -- 172.21.15.97:0/3797518144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb470069070 con 0x7fb47006aa00 2024-03-26T18:57:56.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.709+0000 7fb466a00700 1 -- 172.21.15.97:0/3797518144 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fb45c0189e0 con 0x7fb47006aa00 2024-03-26T18:57:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.711+0000 7fb466a00700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4540a30c0 0x7fb4540a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:57:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.711+0000 7fb466a00700 1 -- 172.21.15.97:0/3797518144 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb45c0cf6e0 con 0x7fb47006aa00 2024-03-26T18:57:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.711+0000 7fb46de00700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4540a30c0 0x7fb4540a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:57:56.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.712+0000 7fb46de00700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4540a30c0 0x7fb4540a5580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb460000c00 tx=0x7fb46001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:57:56.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.715+0000 7fb466a00700 1 -- 172.21.15.97:0/3797518144 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb45c09c3e0 con 0x7fb47006aa00 2024-03-26T18:57:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.982+0000 7fb474a00700 1 -- 172.21.15.97:0/3797518144 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb47010dca0 con 0x7fb4540a30c0 2024-03-26T18:57:56.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.991+0000 7fb466a00700 1 -- 172.21.15.97:0/3797518144 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258995 (secure 0 0 0) 0x7fb47010dca0 con 0x7fb4540a30c0 2024-03-26T18:57:56.990 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.996+0000 7fb474a00700 1 -- 172.21.15.97:0/3797518144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4540a30c0 msgr2=0x7fb4540a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.996+0000 7fb474a00700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4540a30c0 0x7fb4540a5580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb460000c00 tx=0x7fb46001a040 comp rx=0 tx=0).stop 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.996+0000 7fb474a00700 1 -- 172.21.15.97:0/3797518144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb47006aa00 msgr2=0x7fb47019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.996+0000 7fb474a00700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb47006aa00 0x7fb47019b380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00e9f0 tx=0x7fb45c00edb0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.996+0000 7fb474a00700 1 -- 172.21.15.97:0/3797518144 shutdown_connections 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.996+0000 7fb474a00700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb470108400 0x7fb47019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.997+0000 7fb474a00700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb47006a060 0x7fb47019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.997+0000 7fb474a00700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4540a30c0 0x7fb4540a5580 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.997+0000 7fb474a00700 1 --2- 172.21.15.97:0/3797518144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb47006aa00 0x7fb47019b380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.997+0000 7fb474a00700 1 -- 172.21.15.97:0/3797518144 >> 172.21.15.97:0/3797518144 conn(0x7fb4700781e0 msgr2=0x7fb47010d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:57:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.997+0000 7fb474a00700 1 -- 172.21.15.97:0/3797518144 shutdown_connections 2024-03-26T18:57:56.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:57:56.997+0000 7fb474a00700 1 -- 172.21.15.97:0/3797518144 wait complete. 2024-03-26T18:57:56.994 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-26T18:57:57.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:57 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:57:57.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:57 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:57:57.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:57 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:57.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:57 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:57:57.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:57 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:57:57.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:57 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:57.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:57 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:57:57.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:57 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:57:57.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:57 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:57:57.915 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-03-26T18:57:56.936985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656754,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1006,"num_read_kb":621,"num_write":420,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360932,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580892,"statfs":{"total":1151940427776,"available":1151570833408,"internally_reserved":0,"allocated":11980800,"data_stored":7436502,"data_compressed":112698,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":64,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004085"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067144+0000","last_change":"2024-03-26T18:56:24.177714+0000","last_active":"2024-03-26T18:57:30.067144+0000","last_peered":"2024-03-26T18:57:30.067144+0000","last_clean":"2024-03-26T18:57:30.067144+0000","last_became_active":"2024-03-26T18:56:24.177420+0000","last_became_peered":"2024-03-26T18:56:24.177420+0000","last_unstale":"2024-03-26T18:57:30.067144+0000","last_undegraded":"2024-03-26T18:57:30.067144+0000","last_fullsized":"2024-03-26T18:57:30.067144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:21:23.738513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.072672+0000","last_change":"2024-03-26T18:56:24.160191+0000","last_active":"2024-03-26T18:57:30.072672+0000","last_peered":"2024-03-26T18:57:30.072672+0000","last_clean":"2024-03-26T18:57:30.072672+0000","last_became_active":"2024-03-26T18:56:24.159718+0000","last_became_peered":"2024-03-26T18:56:24.159718+0000","last_unstale":"2024-03-26T18:57:30.072672+0000","last_undegraded":"2024-03-26T18:57:30.072672+0000","last_fullsized":"2024-03-26T18:57:30.072672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:12:04.287352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.071083+0000","last_change":"2024-03-26T18:56:24.169795+0000","last_active":"2024-03-26T18:57:30.071083+0000","last_peered":"2024-03-26T18:57:30.071083+0000","last_clean":"2024-03-26T18:57:30.071083+0000","last_became_active":"2024-03-26T18:56:24.166044+0000","last_became_peered":"2024-03-26T18:56:24.166044+0000","last_unstale":"2024-03-26T18:57:30.071083+0000","last_undegraded":"2024-03-26T18:57:30.071083+0000","last_fullsized":"2024-03-26T18:57:30.071083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:21:58.951019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.074617+0000","last_change":"2024-03-26T18:57:30.072874+0000","last_active":"2024-03-26T18:57:30.074617+0000","last_peered":"2024-03-26T18:57:30.074617+0000","last_clean":"2024-03-26T18:57:30.074617+0000","last_became_active":"2024-03-26T18:57:30.072453+0000","last_became_peered":"2024-03-26T18:57:30.072453+0000","last_unstale":"2024-03-26T18:57:30.074617+0000","last_undegraded":"2024-03-26T18:57:30.074617+0000","last_fullsized":"2024-03-26T18:57:30.074617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:12:28.900636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065322+0000","last_change":"2024-03-26T18:56:24.175902+0000","last_active":"2024-03-26T18:57:30.065322+0000","last_peered":"2024-03-26T18:57:30.065322+0000","last_clean":"2024-03-26T18:57:30.065322+0000","last_became_active":"2024-03-26T18:56:24.175568+0000","last_became_peered":"2024-03-26T18:56:24.175568+0000","last_unstale":"2024-03-26T18:57:30.065322+0000","last_undegraded":"2024-03-26T18:57:30.065322+0000","last_fullsized":"2024-03-26T18:57:30.065322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:33:14.054102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421539+0000","last_change":"2024-03-26T18:56:24.145629+0000","last_active":"2024-03-26T18:57:30.421539+0000","last_peered":"2024-03-26T18:57:30.421539+0000","last_clean":"2024-03-26T18:57:30.421539+0000","last_became_active":"2024-03-26T18:56:24.145286+0000","last_became_peered":"2024-03-26T18:56:24.145286+0000","last_unstale":"2024-03-26T18:57:30.421539+0000","last_undegraded":"2024-03-26T18:57:30.421539+0000","last_fullsized":"2024-03-26T18:57:30.421539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:37:24.989479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066412+0000","last_change":"2024-03-26T18:56:24.150869+0000","last_active":"2024-03-26T18:57:30.066412+0000","last_peered":"2024-03-26T18:57:30.066412+0000","last_clean":"2024-03-26T18:57:30.066412+0000","last_became_active":"2024-03-26T18:56:24.150474+0000","last_became_peered":"2024-03-26T18:56:24.150474+0000","last_unstale":"2024-03-26T18:57:30.066412+0000","last_undegraded":"2024-03-26T18:57:30.066412+0000","last_fullsized":"2024-03-26T18:57:30.066412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:55.113389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073698+0000","last_change":"2024-03-26T18:56:24.170218+0000","last_active":"2024-03-26T18:57:30.073698+0000","last_peered":"2024-03-26T18:57:30.073698+0000","last_clean":"2024-03-26T18:57:30.073698+0000","last_became_active":"2024-03-26T18:56:24.169343+0000","last_became_peered":"2024-03-26T18:56:24.169343+0000","last_unstale":"2024-03-26T18:57:30.073698+0000","last_undegraded":"2024-03-26T18:57:30.073698+0000","last_fullsized":"2024-03-26T18:57:30.073698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:28:41.609616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421778+0000","last_change":"2024-03-26T18:56:24.146892+0000","last_active":"2024-03-26T18:57:30.421778+0000","last_peered":"2024-03-26T18:57:30.421778+0000","last_clean":"2024-03-26T18:57:30.421778+0000","last_became_active":"2024-03-26T18:56:24.146564+0000","last_became_peered":"2024-03-26T18:56:24.146564+0000","last_unstale":"2024-03-26T18:57:30.421778+0000","last_undegraded":"2024-03-26T18:57:30.421778+0000","last_fullsized":"2024-03-26T18:57:30.421778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:12:42.104605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.422021+0000","last_change":"2024-03-26T18:56:24.148110+0000","last_active":"2024-03-26T18:57:30.422021+0000","last_peered":"2024-03-26T18:57:30.422021+0000","last_clean":"2024-03-26T18:57:30.422021+0000","last_became_active":"2024-03-26T18:56:24.147763+0000","last_became_peered":"2024-03-26T18:56:24.147763+0000","last_unstale":"2024-03-26T18:57:30.422021+0000","last_undegraded":"2024-03-26T18:57:30.422021+0000","last_fullsized":"2024-03-26T18:57:30.422021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:23.199428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066839+0000","last_change":"2024-03-26T18:56:24.149064+0000","last_active":"2024-03-26T18:57:30.066839+0000","last_peered":"2024-03-26T18:57:30.066839+0000","last_clean":"2024-03-26T18:57:30.066839+0000","last_became_active":"2024-03-26T18:56:24.148400+0000","last_became_peered":"2024-03-26T18:56:24.148400+0000","last_unstale":"2024-03-26T18:57:30.066839+0000","last_undegraded":"2024-03-26T18:57:30.066839+0000","last_fullsized":"2024-03-26T18:57:30.066839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:08:14.141931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.425094+0000","last_change":"2024-03-26T18:57:30.424292+0000","last_active":"2024-03-26T18:57:30.425094+0000","last_peered":"2024-03-26T18:57:30.425094+0000","last_clean":"2024-03-26T18:57:30.425094+0000","last_became_active":"2024-03-26T18:57:30.423759+0000","last_became_peered":"2024-03-26T18:57:30.423759+0000","last_unstale":"2024-03-26T18:57:30.425094+0000","last_undegraded":"2024-03-26T18:57:30.425094+0000","last_fullsized":"2024-03-26T18:57:30.425094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:18:04.632643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.072441+0000","last_change":"2024-03-26T18:56:24.165819+0000","last_active":"2024-03-26T18:57:30.072441+0000","last_peered":"2024-03-26T18:57:30.072441+0000","last_clean":"2024-03-26T18:57:30.072441+0000","last_became_active":"2024-03-26T18:56:24.164401+0000","last_became_peered":"2024-03-26T18:56:24.164401+0000","last_unstale":"2024-03-26T18:57:30.072441+0000","last_undegraded":"2024-03-26T18:57:30.072441+0000","last_fullsized":"2024-03-26T18:57:30.072441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:08:32.301805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.071427+0000","last_change":"2024-03-26T18:56:24.174229+0000","last_active":"2024-03-26T18:57:30.071427+0000","last_peered":"2024-03-26T18:57:30.071427+0000","last_clean":"2024-03-26T18:57:30.071427+0000","last_became_active":"2024-03-26T18:56:24.172924+0000","last_became_peered":"2024-03-26T18:56:24.172924+0000","last_unstale":"2024-03-26T18:57:30.071427+0000","last_undegraded":"2024-03-26T18:57:30.071427+0000","last_fullsized":"2024-03-26T18:57:30.071427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:26:00.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.071084+0000","last_change":"2024-03-26T18:56:24.165029+0000","last_active":"2024-03-26T18:57:30.071084+0000","last_peered":"2024-03-26T18:57:30.071084+0000","last_clean":"2024-03-26T18:57:30.071084+0000","last_became_active":"2024-03-26T18:56:24.160238+0000","last_became_peered":"2024-03-26T18:56:24.160238+0000","last_unstale":"2024-03-26T18:57:30.071084+0000","last_undegraded":"2024-03-26T18:57:30.071084+0000","last_fullsized":"2024-03-26T18:57:30.071084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:56:14.509564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421231+0000","last_change":"2024-03-26T18:56:24.167668+0000","last_active":"2024-03-26T18:57:30.421231+0000","last_peered":"2024-03-26T18:57:30.421231+0000","last_clean":"2024-03-26T18:57:30.421231+0000","last_became_active":"2024-03-26T18:56:24.167267+0000","last_became_peered":"2024-03-26T18:56:24.167267+0000","last_unstale":"2024-03-26T18:57:30.421231+0000","last_undegraded":"2024-03-26T18:57:30.421231+0000","last_fullsized":"2024-03-26T18:57:30.421231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:13:00.503977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067366+0000","last_change":"2024-03-26T18:56:24.177623+0000","last_active":"2024-03-26T18:57:30.067366+0000","last_peered":"2024-03-26T18:57:30.067366+0000","last_clean":"2024-03-26T18:57:30.067366+0000","last_became_active":"2024-03-26T18:56:24.177372+0000","last_became_peered":"2024-03-26T18:56:24.177372+0000","last_unstale":"2024-03-26T18:57:30.067366+0000","last_undegraded":"2024-03-26T18:57:30.067366+0000","last_fullsized":"2024-03-26T18:57:30.067366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:02:20.650520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.070333+0000","last_change":"2024-03-26T18:56:24.152407+0000","last_active":"2024-03-26T18:57:30.070333+0000","last_peered":"2024-03-26T18:57:30.070333+0000","last_clean":"2024-03-26T18:57:30.070333+0000","last_became_active":"2024-03-26T18:56:24.151935+0000","last_became_peered":"2024-03-26T18:56:24.151935+0000","last_unstale":"2024-03-26T18:57:30.070333+0000","last_undegraded":"2024-03-26T18:57:30.070333+0000","last_fullsized":"2024-03-26T18:57:30.070333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:52:13.211124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.068971+0000","last_change":"2024-03-26T18:56:24.162582+0000","last_active":"2024-03-26T18:57:30.068971+0000","last_peered":"2024-03-26T18:57:30.068971+0000","last_clean":"2024-03-26T18:57:30.068971+0000","last_became_active":"2024-03-26T18:56:24.162228+0000","last_became_peered":"2024-03-26T18:56:24.162228+0000","last_unstale":"2024-03-26T18:57:30.068971+0000","last_undegraded":"2024-03-26T18:57:30.068971+0000","last_fullsized":"2024-03-26T18:57:30.068971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:16.841466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:51.982730+0000","last_change":"2024-03-26T18:56:23.149475+0000","last_active":"2024-03-26T18:57:51.982730+0000","last_peered":"2024-03-26T18:57:51.982730+0000","last_clean":"2024-03-26T18:57:51.982730+0000","last_became_active":"2024-03-26T18:56:23.148813+0000","last_became_peered":"2024-03-26T18:56:23.148813+0000","last_unstale":"2024-03-26T18:57:51.982730+0000","last_undegraded":"2024-03-26T18:57:51.982730+0000","last_fullsized":"2024-03-26T18:57:51.982730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:38.315414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073497+0000","last_change":"2024-03-26T18:56:24.173904+0000","last_active":"2024-03-26T18:57:30.073497+0000","last_peered":"2024-03-26T18:57:30.073497+0000","last_clean":"2024-03-26T18:57:30.073497+0000","last_became_active":"2024-03-26T18:56:24.173523+0000","last_became_peered":"2024-03-26T18:56:24.173523+0000","last_unstale":"2024-03-26T18:57:30.073497+0000","last_undegraded":"2024-03-26T18:57:30.073497+0000","last_fullsized":"2024-03-26T18:57:30.073497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:40:35.651821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073788+0000","last_change":"2024-03-26T18:56:23.173583+0000","last_active":"2024-03-26T18:57:30.073788+0000","last_peered":"2024-03-26T18:57:30.073788+0000","last_clean":"2024-03-26T18:57:30.073788+0000","last_became_active":"2024-03-26T18:56:23.172071+0000","last_became_peered":"2024-03-26T18:56:23.172071+0000","last_unstale":"2024-03-26T18:57:30.073788+0000","last_undegraded":"2024-03-26T18:57:30.073788+0000","last_fullsized":"2024-03-26T18:57:30.073788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:29:39.194411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073162+0000","last_change":"2024-03-26T18:56:24.175535+0000","last_active":"2024-03-26T18:57:30.073162+0000","last_peered":"2024-03-26T18:57:30.073162+0000","last_clean":"2024-03-26T18:57:30.073162+0000","last_became_active":"2024-03-26T18:56:24.173748+0000","last_became_peered":"2024-03-26T18:56:24.173748+0000","last_unstale":"2024-03-26T18:57:30.073162+0000","last_undegraded":"2024-03-26T18:57:30.073162+0000","last_fullsized":"2024-03-26T18:57:30.073162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:38.960606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.433759+0000","last_change":"2024-03-26T18:57:30.433759+0000","last_active":"2024-03-26T18:57:30.433759+0000","last_peered":"2024-03-26T18:57:30.433759+0000","last_clean":"2024-03-26T18:57:30.433759+0000","last_became_active":"2024-03-26T18:57:30.423507+0000","last_became_peered":"2024-03-26T18:57:30.423507+0000","last_unstale":"2024-03-26T18:57:30.433759+0000","last_undegraded":"2024-03-26T18:57:30.433759+0000","last_fullsized":"2024-03-26T18:57:30.433759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:31.051608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421113+0000","last_change":"2024-03-26T18:56:24.161351+0000","last_active":"2024-03-26T18:57:30.421113+0000","last_peered":"2024-03-26T18:57:30.421113+0000","last_clean":"2024-03-26T18:57:30.421113+0000","last_became_active":"2024-03-26T18:56:24.161013+0000","last_became_peered":"2024-03-26T18:56:24.161013+0000","last_unstale":"2024-03-26T18:57:30.421113+0000","last_undegraded":"2024-03-26T18:57:30.421113+0000","last_fullsized":"2024-03-26T18:57:30.421113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:24:25.620772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065706+0000","last_change":"2024-03-26T18:56:23.148866+0000","last_active":"2024-03-26T18:57:30.065706+0000","last_peered":"2024-03-26T18:57:30.065706+0000","last_clean":"2024-03-26T18:57:30.065706+0000","last_became_active":"2024-03-26T18:56:23.148434+0000","last_became_peered":"2024-03-26T18:56:23.148434+0000","last_unstale":"2024-03-26T18:57:30.065706+0000","last_undegraded":"2024-03-26T18:57:30.065706+0000","last_fullsized":"2024-03-26T18:57:30.065706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:35:35.180038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069795+0000","last_change":"2024-03-26T18:56:24.147429+0000","last_active":"2024-03-26T18:57:30.069795+0000","last_peered":"2024-03-26T18:57:30.069795+0000","last_clean":"2024-03-26T18:57:30.069795+0000","last_became_active":"2024-03-26T18:56:24.147100+0000","last_became_peered":"2024-03-26T18:56:24.147100+0000","last_unstale":"2024-03-26T18:57:30.069795+0000","last_undegraded":"2024-03-26T18:57:30.069795+0000","last_fullsized":"2024-03-26T18:57:30.069795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:08.884223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":110,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:51.900178+0000","last_change":"2024-03-26T18:56:23.149301+0000","last_active":"2024-03-26T18:57:51.900178+0000","last_peered":"2024-03-26T18:57:51.900178+0000","last_clean":"2024-03-26T18:57:51.900178+0000","last_became_active":"2024-03-26T18:56:23.148939+0000","last_became_peered":"2024-03-26T18:56:23.148939+0000","last_unstale":"2024-03-26T18:57:51.900178+0000","last_undegraded":"2024-03-26T18:57:51.900178+0000","last_fullsized":"2024-03-26T18:57:51.900178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:50:06.649474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065318+0000","last_change":"2024-03-26T18:56:24.157317+0000","last_active":"2024-03-26T18:57:30.065318+0000","last_peered":"2024-03-26T18:57:30.065318+0000","last_clean":"2024-03-26T18:57:30.065318+0000","last_became_active":"2024-03-26T18:56:24.156971+0000","last_became_peered":"2024-03-26T18:56:24.156971+0000","last_unstale":"2024-03-26T18:57:30.065318+0000","last_undegraded":"2024-03-26T18:57:30.065318+0000","last_fullsized":"2024-03-26T18:57:30.065318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:22:39.360404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421437+0000","last_change":"2024-03-26T18:56:23.162848+0000","last_active":"2024-03-26T18:57:30.421437+0000","last_peered":"2024-03-26T18:57:30.421437+0000","last_clean":"2024-03-26T18:57:30.421437+0000","last_became_active":"2024-03-26T18:56:23.162478+0000","last_became_peered":"2024-03-26T18:56:23.162478+0000","last_unstale":"2024-03-26T18:57:30.421437+0000","last_undegraded":"2024-03-26T18:57:30.421437+0000","last_fullsized":"2024-03-26T18:57:30.421437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:07:51.139538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069742+0000","last_change":"2024-03-26T18:56:24.156451+0000","last_active":"2024-03-26T18:57:30.069742+0000","last_peered":"2024-03-26T18:57:30.069742+0000","last_clean":"2024-03-26T18:57:30.069742+0000","last_became_active":"2024-03-26T18:56:24.156237+0000","last_became_peered":"2024-03-26T18:56:24.156237+0000","last_unstale":"2024-03-26T18:57:30.069742+0000","last_undegraded":"2024-03-26T18:57:30.069742+0000","last_fullsized":"2024-03-26T18:57:30.069742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:52:54.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067202+0000","last_change":"2024-03-26T18:56:23.172926+0000","last_active":"2024-03-26T18:57:30.067202+0000","last_peered":"2024-03-26T18:57:30.067202+0000","last_clean":"2024-03-26T18:57:30.067202+0000","last_became_active":"2024-03-26T18:56:23.170512+0000","last_became_peered":"2024-03-26T18:56:23.170512+0000","last_unstale":"2024-03-26T18:57:30.067202+0000","last_undegraded":"2024-03-26T18:57:30.067202+0000","last_fullsized":"2024-03-26T18:57:30.067202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:16:13.747853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073932+0000","last_change":"2024-03-26T18:56:24.173733+0000","last_active":"2024-03-26T18:57:30.073932+0000","last_peered":"2024-03-26T18:57:30.073932+0000","last_clean":"2024-03-26T18:57:30.073932+0000","last_became_active":"2024-03-26T18:56:24.171895+0000","last_became_peered":"2024-03-26T18:56:24.171895+0000","last_unstale":"2024-03-26T18:57:30.073932+0000","last_undegraded":"2024-03-26T18:57:30.073932+0000","last_fullsized":"2024-03-26T18:57:30.073932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:07:59.273853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069293+0000","last_change":"2024-03-26T18:56:23.150140+0000","last_active":"2024-03-26T18:57:30.069293+0000","last_peered":"2024-03-26T18:57:30.069293+0000","last_clean":"2024-03-26T18:57:30.069293+0000","last_became_active":"2024-03-26T18:56:23.149634+0000","last_became_peered":"2024-03-26T18:56:23.149634+0000","last_unstale":"2024-03-26T18:57:30.069293+0000","last_undegraded":"2024-03-26T18:57:30.069293+0000","last_fullsized":"2024-03-26T18:57:30.069293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:52:48.025868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069976+0000","last_change":"2024-03-26T18:56:24.157855+0000","last_active":"2024-03-26T18:57:30.069976+0000","last_peered":"2024-03-26T18:57:30.069976+0000","last_clean":"2024-03-26T18:57:30.069976+0000","last_became_active":"2024-03-26T18:56:24.157220+0000","last_became_peered":"2024-03-26T18:56:24.157220+0000","last_unstale":"2024-03-26T18:57:30.069976+0000","last_undegraded":"2024-03-26T18:57:30.069976+0000","last_fullsized":"2024-03-26T18:57:30.069976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:40:00.815178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":193,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:54.902815+0000","last_change":"2024-03-26T18:56:23.148341+0000","last_active":"2024-03-26T18:57:54.902815+0000","last_peered":"2024-03-26T18:57:54.902815+0000","last_clean":"2024-03-26T18:57:54.902815+0000","last_became_active":"2024-03-26T18:56:23.148021+0000","last_became_peered":"2024-03-26T18:56:23.148021+0000","last_unstale":"2024-03-26T18:57:54.902815+0000","last_undegraded":"2024-03-26T18:57:54.902815+0000","last_fullsized":"2024-03-26T18:57:54.902815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:42.577047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421726+0000","last_change":"2024-03-26T18:56:24.160017+0000","last_active":"2024-03-26T18:57:30.421726+0000","last_peered":"2024-03-26T18:57:30.421726+0000","last_clean":"2024-03-26T18:57:30.421726+0000","last_became_active":"2024-03-26T18:56:24.159686+0000","last_became_peered":"2024-03-26T18:56:24.159686+0000","last_unstale":"2024-03-26T18:57:30.421726+0000","last_undegraded":"2024-03-26T18:57:30.421726+0000","last_fullsized":"2024-03-26T18:57:30.421726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:33:32.693850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066656+0000","last_change":"2024-03-26T18:56:23.146487+0000","last_active":"2024-03-26T18:57:30.066656+0000","last_peered":"2024-03-26T18:57:30.066656+0000","last_clean":"2024-03-26T18:57:30.066656+0000","last_became_active":"2024-03-26T18:56:23.146155+0000","last_became_peered":"2024-03-26T18:56:23.146155+0000","last_unstale":"2024-03-26T18:57:30.066656+0000","last_undegraded":"2024-03-26T18:57:30.066656+0000","last_fullsized":"2024-03-26T18:57:30.066656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:09:51.836211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073126+0000","last_change":"2024-03-26T18:56:24.172652+0000","last_active":"2024-03-26T18:57:30.073126+0000","last_peered":"2024-03-26T18:57:30.073126+0000","last_clean":"2024-03-26T18:57:30.073126+0000","last_became_active":"2024-03-26T18:56:24.170778+0000","last_became_peered":"2024-03-26T18:56:24.170778+0000","last_unstale":"2024-03-26T18:57:30.073126+0000","last_undegraded":"2024-03-26T18:57:30.073126+0000","last_fullsized":"2024-03-26T18:57:30.073126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:47:16.946692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069815+0000","last_change":"2024-03-26T18:56:23.151669+0000","last_active":"2024-03-26T18:57:30.069815+0000","last_peered":"2024-03-26T18:57:30.069815+0000","last_clean":"2024-03-26T18:57:30.069815+0000","last_became_active":"2024-03-26T18:56:23.151276+0000","last_became_peered":"2024-03-26T18:56:23.151276+0000","last_unstale":"2024-03-26T18:57:30.069815+0000","last_undegraded":"2024-03-26T18:57:30.069815+0000","last_fullsized":"2024-03-26T18:57:30.069815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:28:37.827930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421291+0000","last_change":"2024-03-26T18:56:24.151007+0000","last_active":"2024-03-26T18:57:30.421291+0000","last_peered":"2024-03-26T18:57:30.421291+0000","last_clean":"2024-03-26T18:57:30.421291+0000","last_became_active":"2024-03-26T18:56:24.150668+0000","last_became_peered":"2024-03-26T18:56:24.150668+0000","last_unstale":"2024-03-26T18:57:30.421291+0000","last_undegraded":"2024-03-26T18:57:30.421291+0000","last_fullsized":"2024-03-26T18:57:30.421291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:07:04.778039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":229,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:54.902540+0000","last_change":"2024-03-26T18:56:23.144522+0000","last_active":"2024-03-26T18:57:54.902540+0000","last_peered":"2024-03-26T18:57:54.902540+0000","last_clean":"2024-03-26T18:57:54.902540+0000","last_became_active":"2024-03-26T18:56:23.144176+0000","last_became_peered":"2024-03-26T18:56:23.144176+0000","last_unstale":"2024-03-26T18:57:54.902540+0000","last_undegraded":"2024-03-26T18:57:54.902540+0000","last_fullsized":"2024-03-26T18:57:54.902540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:42:20.319478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59730,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":180,"num_read_kb":147,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065770+0000","last_change":"2024-03-26T18:56:24.148307+0000","last_active":"2024-03-26T18:57:30.065770+0000","last_peered":"2024-03-26T18:57:30.065770+0000","last_clean":"2024-03-26T18:57:30.065770+0000","last_became_active":"2024-03-26T18:56:24.147973+0000","last_became_peered":"2024-03-26T18:56:24.147973+0000","last_unstale":"2024-03-26T18:57:30.065770+0000","last_undegraded":"2024-03-26T18:57:30.065770+0000","last_fullsized":"2024-03-26T18:57:30.065770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:08:45.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.068804+0000","last_change":"2024-03-26T18:56:23.163757+0000","last_active":"2024-03-26T18:57:30.068804+0000","last_peered":"2024-03-26T18:57:30.068804+0000","last_clean":"2024-03-26T18:57:30.068804+0000","last_became_active":"2024-03-26T18:56:23.163418+0000","last_became_peered":"2024-03-26T18:56:23.163418+0000","last_unstale":"2024-03-26T18:57:30.068804+0000","last_undegraded":"2024-03-26T18:57:30.068804+0000","last_fullsized":"2024-03-26T18:57:30.068804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:04:09.178853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.068140+0000","last_change":"2024-03-26T18:56:24.167274+0000","last_active":"2024-03-26T18:57:30.068140+0000","last_peered":"2024-03-26T18:57:30.068140+0000","last_clean":"2024-03-26T18:57:30.068140+0000","last_became_active":"2024-03-26T18:56:24.166353+0000","last_became_peered":"2024-03-26T18:56:24.166353+0000","last_unstale":"2024-03-26T18:57:30.068140+0000","last_undegraded":"2024-03-26T18:57:30.068140+0000","last_fullsized":"2024-03-26T18:57:30.068140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:14:44.794112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069498+0000","last_change":"2024-03-26T18:56:23.162115+0000","last_active":"2024-03-26T18:57:30.069498+0000","last_peered":"2024-03-26T18:57:30.069498+0000","last_clean":"2024-03-26T18:57:30.069498+0000","last_became_active":"2024-03-26T18:56:23.161746+0000","last_became_peered":"2024-03-26T18:56:23.161746+0000","last_unstale":"2024-03-26T18:57:30.069498+0000","last_undegraded":"2024-03-26T18:57:30.069498+0000","last_fullsized":"2024-03-26T18:57:30.069498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:08:26.583281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066713+0000","last_change":"2024-03-26T18:56:24.166407+0000","last_active":"2024-03-26T18:57:30.066713+0000","last_peered":"2024-03-26T18:57:30.066713+0000","last_clean":"2024-03-26T18:57:30.066713+0000","last_became_active":"2024-03-26T18:56:24.166077+0000","last_became_peered":"2024-03-26T18:56:24.166077+0000","last_unstale":"2024-03-26T18:57:30.066713+0000","last_undegraded":"2024-03-26T18:57:30.066713+0000","last_fullsized":"2024-03-26T18:57:30.066713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:36:35.160247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":110,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:51.933142+0000","last_change":"2024-03-26T18:56:23.146908+0000","last_active":"2024-03-26T18:57:51.933142+0000","last_peered":"2024-03-26T18:57:51.933142+0000","last_clean":"2024-03-26T18:57:51.933142+0000","last_became_active":"2024-03-26T18:56:23.146576+0000","last_became_peered":"2024-03-26T18:56:23.146576+0000","last_unstale":"2024-03-26T18:57:51.933142+0000","last_undegraded":"2024-03-26T18:57:51.933142+0000","last_fullsized":"2024-03-26T18:57:51.933142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:00:55.708101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.423372+0000","last_change":"2024-03-26T18:57:30.423372+0000","last_active":"2024-03-26T18:57:30.423372+0000","last_peered":"2024-03-26T18:57:30.423372+0000","last_clean":"2024-03-26T18:57:30.423372+0000","last_became_active":"2024-03-26T18:57:30.423022+0000","last_became_peered":"2024-03-26T18:57:30.423022+0000","last_unstale":"2024-03-26T18:57:30.423372+0000","last_undegraded":"2024-03-26T18:57:30.423372+0000","last_fullsized":"2024-03-26T18:57:30.423372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:04:52.867481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.085892+0000","last_change":"2024-03-26T18:57:30.085458+0000","last_active":"2024-03-26T18:57:30.085892+0000","last_peered":"2024-03-26T18:57:30.085892+0000","last_clean":"2024-03-26T18:57:30.085892+0000","last_became_active":"2024-03-26T18:57:30.073182+0000","last_became_peered":"2024-03-26T18:57:30.073182+0000","last_unstale":"2024-03-26T18:57:30.085892+0000","last_undegraded":"2024-03-26T18:57:30.085892+0000","last_fullsized":"2024-03-26T18:57:30.085892+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:32:31.999866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069107+0000","last_change":"2024-03-26T18:56:24.146993+0000","last_active":"2024-03-26T18:57:30.069107+0000","last_peered":"2024-03-26T18:57:30.069107+0000","last_clean":"2024-03-26T18:57:30.069107+0000","last_became_active":"2024-03-26T18:56:24.146692+0000","last_became_peered":"2024-03-26T18:56:24.146692+0000","last_unstale":"2024-03-26T18:57:30.069107+0000","last_undegraded":"2024-03-26T18:57:30.069107+0000","last_fullsized":"2024-03-26T18:57:30.069107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:08:33.731716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065752+0000","last_change":"2024-03-26T18:56:23.148459+0000","last_active":"2024-03-26T18:57:30.065752+0000","last_peered":"2024-03-26T18:57:30.065752+0000","last_clean":"2024-03-26T18:57:30.065752+0000","last_became_active":"2024-03-26T18:56:23.148097+0000","last_became_peered":"2024-03-26T18:56:23.148097+0000","last_unstale":"2024-03-26T18:57:30.065752+0000","last_undegraded":"2024-03-26T18:57:30.065752+0000","last_fullsized":"2024-03-26T18:57:30.065752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:08:51.990705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066974+0000","last_change":"2024-03-26T18:56:24.146399+0000","last_active":"2024-03-26T18:57:30.066974+0000","last_peered":"2024-03-26T18:57:30.066974+0000","last_clean":"2024-03-26T18:57:30.066974+0000","last_became_active":"2024-03-26T18:56:24.146014+0000","last_became_peered":"2024-03-26T18:56:24.146014+0000","last_unstale":"2024-03-26T18:57:30.066974+0000","last_undegraded":"2024-03-26T18:57:30.066974+0000","last_fullsized":"2024-03-26T18:57:30.066974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:24:26.809602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069836+0000","last_change":"2024-03-26T18:56:23.148037+0000","last_active":"2024-03-26T18:57:30.069836+0000","last_peered":"2024-03-26T18:57:30.069836+0000","last_clean":"2024-03-26T18:57:30.069836+0000","last_became_active":"2024-03-26T18:56:23.147598+0000","last_became_peered":"2024-03-26T18:56:23.147598+0000","last_unstale":"2024-03-26T18:57:30.069836+0000","last_undegraded":"2024-03-26T18:57:30.069836+0000","last_fullsized":"2024-03-26T18:57:30.069836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:21:55.105299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421789+0000","last_change":"2024-03-26T18:56:24.167488+0000","last_active":"2024-03-26T18:57:30.421789+0000","last_peered":"2024-03-26T18:57:30.421789+0000","last_clean":"2024-03-26T18:57:30.421789+0000","last_became_active":"2024-03-26T18:56:24.167154+0000","last_became_peered":"2024-03-26T18:56:24.167154+0000","last_unstale":"2024-03-26T18:57:30.421789+0000","last_undegraded":"2024-03-26T18:57:30.421789+0000","last_fullsized":"2024-03-26T18:57:30.421789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:00:35.606573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:52.985144+0000","last_change":"2024-03-26T18:56:23.160694+0000","last_active":"2024-03-26T18:57:52.985144+0000","last_peered":"2024-03-26T18:57:52.985144+0000","last_clean":"2024-03-26T18:57:52.985144+0000","last_became_active":"2024-03-26T18:56:23.160416+0000","last_became_peered":"2024-03-26T18:56:23.160416+0000","last_unstale":"2024-03-26T18:57:52.985144+0000","last_undegraded":"2024-03-26T18:57:52.985144+0000","last_fullsized":"2024-03-26T18:57:52.985144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:31:15.616473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421544+0000","last_change":"2024-03-26T18:56:24.170503+0000","last_active":"2024-03-26T18:57:30.421544+0000","last_peered":"2024-03-26T18:57:30.421544+0000","last_clean":"2024-03-26T18:57:30.421544+0000","last_became_active":"2024-03-26T18:56:24.170113+0000","last_became_peered":"2024-03-26T18:56:24.170113+0000","last_unstale":"2024-03-26T18:57:30.421544+0000","last_undegraded":"2024-03-26T18:57:30.421544+0000","last_fullsized":"2024-03-26T18:57:30.421544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:44.076756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069474+0000","last_change":"2024-03-26T18:56:24.146856+0000","last_active":"2024-03-26T18:57:30.069474+0000","last_peered":"2024-03-26T18:57:30.069474+0000","last_clean":"2024-03-26T18:57:30.069474+0000","last_became_active":"2024-03-26T18:56:24.146545+0000","last_became_peered":"2024-03-26T18:56:24.146545+0000","last_unstale":"2024-03-26T18:57:30.069474+0000","last_undegraded":"2024-03-26T18:57:30.069474+0000","last_fullsized":"2024-03-26T18:57:30.069474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:58:26.876614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:51.932584+0000","last_change":"2024-03-26T18:56:23.173650+0000","last_active":"2024-03-26T18:57:51.932584+0000","last_peered":"2024-03-26T18:57:51.932584+0000","last_clean":"2024-03-26T18:57:51.932584+0000","last_became_active":"2024-03-26T18:56:23.173343+0000","last_became_peered":"2024-03-26T18:56:23.173343+0000","last_unstale":"2024-03-26T18:57:51.932584+0000","last_undegraded":"2024-03-26T18:57:51.932584+0000","last_fullsized":"2024-03-26T18:57:51.932584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:42:51.138892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073926+0000","last_change":"2024-03-26T18:56:24.175697+0000","last_active":"2024-03-26T18:57:30.073926+0000","last_peered":"2024-03-26T18:57:30.073926+0000","last_clean":"2024-03-26T18:57:30.073926+0000","last_became_active":"2024-03-26T18:56:24.175159+0000","last_became_peered":"2024-03-26T18:56:24.175159+0000","last_unstale":"2024-03-26T18:57:30.073926+0000","last_undegraded":"2024-03-26T18:57:30.073926+0000","last_fullsized":"2024-03-26T18:57:30.073926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:02:35.581432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.422165+0000","last_change":"2024-03-26T18:56:23.154766+0000","last_active":"2024-03-26T18:57:30.422165+0000","last_peered":"2024-03-26T18:57:30.422165+0000","last_clean":"2024-03-26T18:57:30.422165+0000","last_became_active":"2024-03-26T18:56:23.154475+0000","last_became_peered":"2024-03-26T18:56:23.154475+0000","last_unstale":"2024-03-26T18:57:30.422165+0000","last_undegraded":"2024-03-26T18:57:30.422165+0000","last_fullsized":"2024-03-26T18:57:30.422165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:29:38.484201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066918+0000","last_change":"2024-03-26T18:55:03.797254+0000","last_active":"2024-03-26T18:57:30.066918+0000","last_peered":"2024-03-26T18:57:30.066918+0000","last_clean":"2024-03-26T18:57:30.066918+0000","last_became_active":"2024-03-26T18:55:03.796795+0000","last_became_peered":"2024-03-26T18:55:03.796795+0000","last_unstale":"2024-03-26T18:57:30.066918+0000","last_undegraded":"2024-03-26T18:57:30.066918+0000","last_fullsized":"2024-03-26T18:57:30.066918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:51:11.590736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:51:11.590736+0000","last_clean_scrub_stamp":"2024-03-26T18:51:11.590736+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:48.215695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066872+0000","last_change":"2024-03-26T18:56:24.165552+0000","last_active":"2024-03-26T18:57:30.066872+0000","last_peered":"2024-03-26T18:57:30.066872+0000","last_clean":"2024-03-26T18:57:30.066872+0000","last_became_active":"2024-03-26T18:56:24.164230+0000","last_became_peered":"2024-03-26T18:56:24.164230+0000","last_unstale":"2024-03-26T18:57:30.066872+0000","last_undegraded":"2024-03-26T18:57:30.066872+0000","last_fullsized":"2024-03-26T18:57:30.066872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:36:19.750451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.426540+0000","last_change":"2024-03-26T18:57:30.426540+0000","last_active":"2024-03-26T18:57:30.426540+0000","last_peered":"2024-03-26T18:57:30.426540+0000","last_clean":"2024-03-26T18:57:30.426540+0000","last_became_active":"2024-03-26T18:57:30.423235+0000","last_became_peered":"2024-03-26T18:57:30.423235+0000","last_unstale":"2024-03-26T18:57:30.426540+0000","last_undegraded":"2024-03-26T18:57:30.426540+0000","last_fullsized":"2024-03-26T18:57:30.426540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:36:08.904800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065868+0000","last_change":"2024-03-26T18:56:23.154066+0000","last_active":"2024-03-26T18:57:30.065868+0000","last_peered":"2024-03-26T18:57:30.065868+0000","last_clean":"2024-03-26T18:57:30.065868+0000","last_became_active":"2024-03-26T18:56:23.153813+0000","last_became_peered":"2024-03-26T18:56:23.153813+0000","last_unstale":"2024-03-26T18:57:30.065868+0000","last_undegraded":"2024-03-26T18:57:30.065868+0000","last_fullsized":"2024-03-26T18:57:30.065868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:25:07.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067811+0000","last_change":"2024-03-26T18:56:24.162166+0000","last_active":"2024-03-26T18:57:30.067811+0000","last_peered":"2024-03-26T18:57:30.067811+0000","last_clean":"2024-03-26T18:57:30.067811+0000","last_became_active":"2024-03-26T18:56:24.161623+0000","last_became_peered":"2024-03-26T18:56:24.161623+0000","last_unstale":"2024-03-26T18:57:30.067811+0000","last_undegraded":"2024-03-26T18:57:30.067811+0000","last_fullsized":"2024-03-26T18:57:30.067811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:42.588275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'13","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.070450+0000","last_change":"2024-03-26T18:56:23.167462+0000","last_active":"2024-03-26T18:57:30.070450+0000","last_peered":"2024-03-26T18:57:30.070450+0000","last_clean":"2024-03-26T18:57:30.070450+0000","last_became_active":"2024-03-26T18:56:23.167124+0000","last_became_peered":"2024-03-26T18:56:23.167124+0000","last_unstale":"2024-03-26T18:57:30.070450+0000","last_undegraded":"2024-03-26T18:57:30.070450+0000","last_fullsized":"2024-03-26T18:57:30.070450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:38.550421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.074654+0000","last_change":"2024-03-26T18:56:24.173164+0000","last_active":"2024-03-26T18:57:30.074654+0000","last_peered":"2024-03-26T18:57:30.074654+0000","last_clean":"2024-03-26T18:57:30.074654+0000","last_became_active":"2024-03-26T18:56:24.170083+0000","last_became_peered":"2024-03-26T18:56:24.170083+0000","last_unstale":"2024-03-26T18:57:30.074654+0000","last_undegraded":"2024-03-26T18:57:30.074654+0000","last_fullsized":"2024-03-26T18:57:30.074654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:26.696625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.422308+0000","last_change":"2024-03-26T18:56:23.162835+0000","last_active":"2024-03-26T18:57:30.422308+0000","last_peered":"2024-03-26T18:57:30.422308+0000","last_clean":"2024-03-26T18:57:30.422308+0000","last_became_active":"2024-03-26T18:56:23.162497+0000","last_became_peered":"2024-03-26T18:56:23.162497+0000","last_unstale":"2024-03-26T18:57:30.422308+0000","last_undegraded":"2024-03-26T18:57:30.422308+0000","last_fullsized":"2024-03-26T18:57:30.422308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:29:47.903712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.071309+0000","last_change":"2024-03-26T18:56:24.178140+0000","last_active":"2024-03-26T18:57:30.071309+0000","last_peered":"2024-03-26T18:57:30.071309+0000","last_clean":"2024-03-26T18:57:30.071309+0000","last_became_active":"2024-03-26T18:56:24.177769+0000","last_became_peered":"2024-03-26T18:56:24.177769+0000","last_unstale":"2024-03-26T18:57:30.071309+0000","last_undegraded":"2024-03-26T18:57:30.071309+0000","last_fullsized":"2024-03-26T18:57:30.071309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:39.714335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421323+0000","last_change":"2024-03-26T18:56:23.143972+0000","last_active":"2024-03-26T18:57:30.421323+0000","last_peered":"2024-03-26T18:57:30.421323+0000","last_clean":"2024-03-26T18:57:30.421323+0000","last_became_active":"2024-03-26T18:56:23.143395+0000","last_became_peered":"2024-03-26T18:56:23.143395+0000","last_unstale":"2024-03-26T18:57:30.421323+0000","last_undegraded":"2024-03-26T18:57:30.421323+0000","last_fullsized":"2024-03-26T18:57:30.421323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:46:32.548546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065137+0000","last_change":"2024-03-26T18:56:24.144839+0000","last_active":"2024-03-26T18:57:30.065137+0000","last_peered":"2024-03-26T18:57:30.065137+0000","last_clean":"2024-03-26T18:57:30.065137+0000","last_became_active":"2024-03-26T18:56:24.144519+0000","last_became_peered":"2024-03-26T18:56:24.144519+0000","last_unstale":"2024-03-26T18:57:30.065137+0000","last_undegraded":"2024-03-26T18:57:30.065137+0000","last_fullsized":"2024-03-26T18:57:30.065137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:54:59.787475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073699+0000","last_change":"2024-03-26T18:56:23.161380+0000","last_active":"2024-03-26T18:57:30.073699+0000","last_peered":"2024-03-26T18:57:30.073699+0000","last_clean":"2024-03-26T18:57:30.073699+0000","last_became_active":"2024-03-26T18:56:23.160376+0000","last_became_peered":"2024-03-26T18:56:23.160376+0000","last_unstale":"2024-03-26T18:57:30.073699+0000","last_undegraded":"2024-03-26T18:57:30.073699+0000","last_fullsized":"2024-03-26T18:57:30.073699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:49.610002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067094+0000","last_change":"2024-03-26T18:56:24.170284+0000","last_active":"2024-03-26T18:57:30.067094+0000","last_peered":"2024-03-26T18:57:30.067094+0000","last_clean":"2024-03-26T18:57:30.067094+0000","last_became_active":"2024-03-26T18:56:24.169954+0000","last_became_peered":"2024-03-26T18:56:24.169954+0000","last_unstale":"2024-03-26T18:57:30.067094+0000","last_undegraded":"2024-03-26T18:57:30.067094+0000","last_fullsized":"2024-03-26T18:57:30.067094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:05:21.383976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.088798+0000","last_change":"2024-03-26T18:57:30.088435+0000","last_active":"2024-03-26T18:57:30.088798+0000","last_peered":"2024-03-26T18:57:30.088798+0000","last_clean":"2024-03-26T18:57:30.088798+0000","last_became_active":"2024-03-26T18:57:30.075160+0000","last_became_peered":"2024-03-26T18:57:30.075160+0000","last_unstale":"2024-03-26T18:57:30.088798+0000","last_undegraded":"2024-03-26T18:57:30.088798+0000","last_fullsized":"2024-03-26T18:57:30.088798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:24:58.325454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.074292+0000","last_change":"2024-03-26T18:56:24.176940+0000","last_active":"2024-03-26T18:57:30.074292+0000","last_peered":"2024-03-26T18:57:30.074292+0000","last_clean":"2024-03-26T18:57:30.074292+0000","last_became_active":"2024-03-26T18:56:24.176396+0000","last_became_peered":"2024-03-26T18:56:24.176396+0000","last_unstale":"2024-03-26T18:57:30.074292+0000","last_undegraded":"2024-03-26T18:57:30.074292+0000","last_fullsized":"2024-03-26T18:57:30.074292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:53:06.994055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065440+0000","last_change":"2024-03-26T18:56:23.149738+0000","last_active":"2024-03-26T18:57:30.065440+0000","last_peered":"2024-03-26T18:57:30.065440+0000","last_clean":"2024-03-26T18:57:30.065440+0000","last_became_active":"2024-03-26T18:56:23.149399+0000","last_became_peered":"2024-03-26T18:56:23.149399+0000","last_unstale":"2024-03-26T18:57:30.065440+0000","last_undegraded":"2024-03-26T18:57:30.065440+0000","last_fullsized":"2024-03-26T18:57:30.065440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:44:50.666181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066645+0000","last_change":"2024-03-26T18:56:24.161505+0000","last_active":"2024-03-26T18:57:30.066645+0000","last_peered":"2024-03-26T18:57:30.066645+0000","last_clean":"2024-03-26T18:57:30.066645+0000","last_became_active":"2024-03-26T18:56:24.161035+0000","last_became_peered":"2024-03-26T18:56:24.161035+0000","last_unstale":"2024-03-26T18:57:30.066645+0000","last_undegraded":"2024-03-26T18:57:30.066645+0000","last_fullsized":"2024-03-26T18:57:30.066645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:22:30.062398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":110,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:53.935474+0000","last_change":"2024-03-26T18:56:23.150900+0000","last_active":"2024-03-26T18:57:53.935474+0000","last_peered":"2024-03-26T18:57:53.935474+0000","last_clean":"2024-03-26T18:57:53.935474+0000","last_became_active":"2024-03-26T18:56:23.150578+0000","last_became_peered":"2024-03-26T18:56:23.150578+0000","last_unstale":"2024-03-26T18:57:53.935474+0000","last_undegraded":"2024-03-26T18:57:53.935474+0000","last_fullsized":"2024-03-26T18:57:53.935474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:18:09.039791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067009+0000","last_change":"2024-03-26T18:56:24.148744+0000","last_active":"2024-03-26T18:57:30.067009+0000","last_peered":"2024-03-26T18:57:30.067009+0000","last_clean":"2024-03-26T18:57:30.067009+0000","last_became_active":"2024-03-26T18:56:24.147857+0000","last_became_peered":"2024-03-26T18:56:24.147857+0000","last_unstale":"2024-03-26T18:57:30.067009+0000","last_undegraded":"2024-03-26T18:57:30.067009+0000","last_fullsized":"2024-03-26T18:57:30.067009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:59:15.890399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069711+0000","last_change":"2024-03-26T18:56:23.165228+0000","last_active":"2024-03-26T18:57:30.069711+0000","last_peered":"2024-03-26T18:57:30.069711+0000","last_clean":"2024-03-26T18:57:30.069711+0000","last_became_active":"2024-03-26T18:56:23.164852+0000","last_became_peered":"2024-03-26T18:56:23.164852+0000","last_unstale":"2024-03-26T18:57:30.069711+0000","last_undegraded":"2024-03-26T18:57:30.069711+0000","last_fullsized":"2024-03-26T18:57:30.069711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:16:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066655+0000","last_change":"2024-03-26T18:56:24.154235+0000","last_active":"2024-03-26T18:57:30.066655+0000","last_peered":"2024-03-26T18:57:30.066655+0000","last_clean":"2024-03-26T18:57:30.066655+0000","last_became_active":"2024-03-26T18:56:24.153864+0000","last_became_peered":"2024-03-26T18:56:24.153864+0000","last_unstale":"2024-03-26T18:57:30.066655+0000","last_undegraded":"2024-03-26T18:57:30.066655+0000","last_fullsized":"2024-03-26T18:57:30.066655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:30:36.577695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067042+0000","last_change":"2024-03-26T18:56:23.175807+0000","last_active":"2024-03-26T18:57:30.067042+0000","last_peered":"2024-03-26T18:57:30.067042+0000","last_clean":"2024-03-26T18:57:30.067042+0000","last_became_active":"2024-03-26T18:56:23.175477+0000","last_became_peered":"2024-03-26T18:56:23.175477+0000","last_unstale":"2024-03-26T18:57:30.067042+0000","last_undegraded":"2024-03-26T18:57:30.067042+0000","last_fullsized":"2024-03-26T18:57:30.067042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:45:28.494823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067015+0000","last_change":"2024-03-26T18:56:24.145201+0000","last_active":"2024-03-26T18:57:30.067015+0000","last_peered":"2024-03-26T18:57:30.067015+0000","last_clean":"2024-03-26T18:57:30.067015+0000","last_became_active":"2024-03-26T18:56:24.144793+0000","last_became_peered":"2024-03-26T18:56:24.144793+0000","last_unstale":"2024-03-26T18:57:30.067015+0000","last_undegraded":"2024-03-26T18:57:30.067015+0000","last_fullsized":"2024-03-26T18:57:30.067015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:13:55.476075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067210+0000","last_change":"2024-03-26T18:56:24.145445+0000","last_active":"2024-03-26T18:57:30.067210+0000","last_peered":"2024-03-26T18:57:30.067210+0000","last_clean":"2024-03-26T18:57:30.067210+0000","last_became_active":"2024-03-26T18:56:24.145115+0000","last_became_peered":"2024-03-26T18:56:24.145115+0000","last_unstale":"2024-03-26T18:57:30.067210+0000","last_undegraded":"2024-03-26T18:57:30.067210+0000","last_fullsized":"2024-03-26T18:57:30.067210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:30:45.728966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.068815+0000","last_change":"2024-03-26T18:56:24.148307+0000","last_active":"2024-03-26T18:57:30.068815+0000","last_peered":"2024-03-26T18:57:30.068815+0000","last_clean":"2024-03-26T18:57:30.068815+0000","last_became_active":"2024-03-26T18:56:24.147973+0000","last_became_peered":"2024-03-26T18:56:24.147973+0000","last_unstale":"2024-03-26T18:57:30.068815+0000","last_undegraded":"2024-03-26T18:57:30.068815+0000","last_fullsized":"2024-03-26T18:57:30.068815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:36:23.330139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.068567+0000","last_change":"2024-03-26T18:56:24.175594+0000","last_active":"2024-03-26T18:57:30.068567+0000","last_peered":"2024-03-26T18:57:30.068567+0000","last_clean":"2024-03-26T18:57:30.068567+0000","last_became_active":"2024-03-26T18:56:24.175261+0000","last_became_peered":"2024-03-26T18:56:24.175261+0000","last_unstale":"2024-03-26T18:57:30.068567+0000","last_undegraded":"2024-03-26T18:57:30.068567+0000","last_fullsized":"2024-03-26T18:57:30.068567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:12:02.744475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.072014+0000","last_change":"2024-03-26T18:56:24.177648+0000","last_active":"2024-03-26T18:57:30.072014+0000","last_peered":"2024-03-26T18:57:30.072014+0000","last_clean":"2024-03-26T18:57:30.072014+0000","last_became_active":"2024-03-26T18:56:24.176768+0000","last_became_peered":"2024-03-26T18:56:24.176768+0000","last_unstale":"2024-03-26T18:57:30.072014+0000","last_undegraded":"2024-03-26T18:57:30.072014+0000","last_fullsized":"2024-03-26T18:57:30.072014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:24:10.207407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.068087+0000","last_change":"2024-03-26T18:56:24.166746+0000","last_active":"2024-03-26T18:57:30.068087+0000","last_peered":"2024-03-26T18:57:30.068087+0000","last_clean":"2024-03-26T18:57:30.068087+0000","last_became_active":"2024-03-26T18:56:24.166220+0000","last_became_peered":"2024-03-26T18:56:24.166220+0000","last_unstale":"2024-03-26T18:57:30.068087+0000","last_undegraded":"2024-03-26T18:57:30.068087+0000","last_fullsized":"2024-03-26T18:57:30.068087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:16:54.411740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066856+0000","last_change":"2024-03-26T18:56:24.145511+0000","last_active":"2024-03-26T18:57:30.066856+0000","last_peered":"2024-03-26T18:57:30.066856+0000","last_clean":"2024-03-26T18:57:30.066856+0000","last_became_active":"2024-03-26T18:56:24.145164+0000","last_became_peered":"2024-03-26T18:56:24.145164+0000","last_unstale":"2024-03-26T18:57:30.066856+0000","last_undegraded":"2024-03-26T18:57:30.066856+0000","last_fullsized":"2024-03-26T18:57:30.066856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:25:05.239952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069284+0000","last_change":"2024-03-26T18:56:24.145367+0000","last_active":"2024-03-26T18:57:30.069284+0000","last_peered":"2024-03-26T18:57:30.069284+0000","last_clean":"2024-03-26T18:57:30.069284+0000","last_became_active":"2024-03-26T18:56:24.144058+0000","last_became_peered":"2024-03-26T18:56:24.144058+0000","last_unstale":"2024-03-26T18:57:30.069284+0000","last_undegraded":"2024-03-26T18:57:30.069284+0000","last_fullsized":"2024-03-26T18:57:30.069284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:10.440142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.074415+0000","last_change":"2024-03-26T18:56:24.174209+0000","last_active":"2024-03-26T18:57:30.074415+0000","last_peered":"2024-03-26T18:57:30.074415+0000","last_clean":"2024-03-26T18:57:30.074415+0000","last_became_active":"2024-03-26T18:56:24.173873+0000","last_became_peered":"2024-03-26T18:56:24.173873+0000","last_unstale":"2024-03-26T18:57:30.074415+0000","last_undegraded":"2024-03-26T18:57:30.074415+0000","last_fullsized":"2024-03-26T18:57:30.074415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:21:34.317699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069391+0000","last_change":"2024-03-26T18:56:24.166633+0000","last_active":"2024-03-26T18:57:30.069391+0000","last_peered":"2024-03-26T18:57:30.069391+0000","last_clean":"2024-03-26T18:57:30.069391+0000","last_became_active":"2024-03-26T18:56:24.166372+0000","last_became_peered":"2024-03-26T18:56:24.166372+0000","last_unstale":"2024-03-26T18:57:30.069391+0000","last_undegraded":"2024-03-26T18:57:30.069391+0000","last_fullsized":"2024-03-26T18:57:30.069391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:38:18.005223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073449+0000","last_change":"2024-03-26T18:56:24.179979+0000","last_active":"2024-03-26T18:57:30.073449+0000","last_peered":"2024-03-26T18:57:30.073449+0000","last_clean":"2024-03-26T18:57:30.073449+0000","last_became_active":"2024-03-26T18:56:24.179533+0000","last_became_peered":"2024-03-26T18:56:24.179533+0000","last_unstale":"2024-03-26T18:57:30.073449+0000","last_undegraded":"2024-03-26T18:57:30.073449+0000","last_fullsized":"2024-03-26T18:57:30.073449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:19:26.104430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421466+0000","last_change":"2024-03-26T18:56:24.175802+0000","last_active":"2024-03-26T18:57:30.421466+0000","last_peered":"2024-03-26T18:57:30.421466+0000","last_clean":"2024-03-26T18:57:30.421466+0000","last_became_active":"2024-03-26T18:56:24.175451+0000","last_became_peered":"2024-03-26T18:56:24.175451+0000","last_unstale":"2024-03-26T18:57:30.421466+0000","last_undegraded":"2024-03-26T18:57:30.421466+0000","last_fullsized":"2024-03-26T18:57:30.421466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:48:07.444891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.072336+0000","last_change":"2024-03-26T18:56:24.167708+0000","last_active":"2024-03-26T18:57:30.072336+0000","last_peered":"2024-03-26T18:57:30.072336+0000","last_clean":"2024-03-26T18:57:30.072336+0000","last_became_active":"2024-03-26T18:56:24.166985+0000","last_became_peered":"2024-03-26T18:56:24.166985+0000","last_unstale":"2024-03-26T18:57:30.072336+0000","last_undegraded":"2024-03-26T18:57:30.072336+0000","last_fullsized":"2024-03-26T18:57:30.072336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:13:35.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066573+0000","last_change":"2024-03-26T18:56:24.151474+0000","last_active":"2024-03-26T18:57:30.066573+0000","last_peered":"2024-03-26T18:57:30.066573+0000","last_clean":"2024-03-26T18:57:30.066573+0000","last_became_active":"2024-03-26T18:56:24.151151+0000","last_became_peered":"2024-03-26T18:56:24.151151+0000","last_unstale":"2024-03-26T18:57:30.066573+0000","last_undegraded":"2024-03-26T18:57:30.066573+0000","last_fullsized":"2024-03-26T18:57:30.066573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:34.926612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280636,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66152,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":880,"num_read_kb":512,"num_write":237,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":198456,"data_compressed":5610,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383711,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1107471,"data_compressed":37566,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":456642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":456908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1049147,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":517002,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":457698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":517222,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":456184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1048825,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61482,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61702,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61466,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T18:57:57.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T18:57:58.380 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:57:58.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:58 smithi105 ceph-mon[36270]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-03-26T18:57:58.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:57:58 smithi105 ceph-mon[36270]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:57:58.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:58 smithi097 ceph-mon[31955]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-03-26T18:57:58.532 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:57:58 smithi097 ceph-mon[31955]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:57:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:58 smithi144 ceph-mon[36053]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-03-26T18:57:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:57:58 smithi144 ceph-mon[36053]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:58:00.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:00 smithi105 ceph-mon[36270]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-26T18:58:00.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:00 smithi105 ceph-mon[36270]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-26T18:58:00.486 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:00 smithi097 ceph-mon[31955]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-26T18:58:00.486 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:00 smithi097 ceph-mon[31955]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-26T18:58:00.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:00 smithi144 ceph-mon[36053]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-26T18:58:00.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:00 smithi144 ceph-mon[36053]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-26T18:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.668+0000 7fa9da400700 1 -- 172.21.15.97:0/3525986623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9d41024f0 msgr2=0x7fa9d41028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.668+0000 7fa9da400700 1 --2- 172.21.15.97:0/3525986623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9d41024f0 0x7fa9d41028d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009990 tx=0x7fa9c8009ca0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.668+0000 7fa9da400700 1 -- 172.21.15.97:0/3525986623 shutdown_connections 2024-03-26T18:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.668+0000 7fa9da400700 1 --2- 172.21.15.97:0/3525986623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d41075d0 0x7fa9d410f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.668+0000 7fa9da400700 1 --2- 172.21.15.97:0/3525986623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9d41024f0 0x7fa9d41028d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.668+0000 7fa9da400700 1 --2- 172.21.15.97:0/3525986623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9d4102ea0 0x7fa9d4106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.668+0000 7fa9da400700 1 -- 172.21.15.97:0/3525986623 >> 172.21.15.97:0/3525986623 conn(0x7fa9d40fdd80 msgr2=0x7fa9d41001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.668+0000 7fa9da400700 1 -- 172.21.15.97:0/3525986623 shutdown_connections 2024-03-26T18:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.668+0000 7fa9da400700 1 -- 172.21.15.97:0/3525986623 wait complete. 2024-03-26T18:58:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.670+0000 7fa9da400700 1 Processor -- start 2024-03-26T18:58:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.670+0000 7fa9da400700 1 -- start start 2024-03-26T18:58:00.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.670+0000 7fa9da400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9d41024f0 0x7fa9d41a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:00.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.670+0000 7fa9da400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9d4102ea0 0x7fa9d41a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:00.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.670+0000 7fa9da400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d41075d0 0x7fa9d419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:00.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.670+0000 7fa9da400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41125e0 con 0x7fa9d4102ea0 2024-03-26T18:58:00.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.670+0000 7fa9da400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4112460 con 0x7fa9d41024f0 2024-03-26T18:58:00.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.670+0000 7fa9da400700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4112760 con 0x7fa9d41075d0 2024-03-26T18:58:00.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.670+0000 7fa9d9000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9d41024f0 0x7fa9d41a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.670+0000 7fa9d9000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9d41024f0 0x7fa9d41a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:48352/0 (socket says 172.21.15.97:48352) 2024-03-26T18:58:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.671+0000 7fa9d9000700 1 -- 172.21.15.97:0/33572936 learned_addr learned my addr 172.21.15.97:0/33572936 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.671+0000 7fa9d9a00700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d41075d0 0x7fa9d419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.671+0000 7fa9d3e00700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9d4102ea0 0x7fa9d41a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.671+0000 7fa9d9000700 1 -- 172.21.15.97:0/33572936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d41075d0 msgr2=0x7fa9d419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.671+0000 7fa9d9000700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d41075d0 0x7fa9d419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.671+0000 7fa9d9000700 1 -- 172.21.15.97:0/33572936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9d4102ea0 msgr2=0x7fa9d41a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.671+0000 7fa9d9000700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9d4102ea0 0x7fa9d41a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.671+0000 7fa9d9000700 1 -- 172.21.15.97:0/33572936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d419b1a0 con 0x7fa9d41024f0 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.671+0000 7fa9d9a00700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d41075d0 0x7fa9d419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.671+0000 7fa9d9000700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9d41024f0 0x7fa9d41a0880 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8000c00 tx=0x7fa9c800b6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.672+0000 7fa9d1600700 1 -- 172.21.15.97:0/33572936 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8027070 con 0x7fa9d41024f0 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.672+0000 7fa9d1600700 1 -- 172.21.15.97:0/33572936 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c8022470 con 0x7fa9d41024f0 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.672+0000 7fa9da400700 1 -- 172.21.15.97:0/33572936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d419b430 con 0x7fa9d41024f0 2024-03-26T18:58:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.672+0000 7fa9d1600700 1 -- 172.21.15.97:0/33572936 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8017630 con 0x7fa9d41024f0 2024-03-26T18:58:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.672+0000 7fa9da400700 1 -- 172.21.15.97:0/33572936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d41a7750 con 0x7fa9d41024f0 2024-03-26T18:58:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.673+0000 7fa9da400700 1 -- 172.21.15.97:0/33572936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d4069070 con 0x7fa9d41024f0 2024-03-26T18:58:00.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.677+0000 7fa9d1600700 1 -- 172.21.15.97:0/33572936 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fa9c802a070 con 0x7fa9d41024f0 2024-03-26T18:58:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.679+0000 7fa9d1600700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9bc0a3030 0x7fa9bc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.679+0000 7fa9d1600700 1 -- 172.21.15.97:0/33572936 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa9c80d01c0 con 0x7fa9d41024f0 2024-03-26T18:58:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.679+0000 7fa9d3e00700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9bc0a3030 0x7fa9bc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.680+0000 7fa9d3e00700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9bc0a3030 0x7fa9bc0a54f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40ff660 tx=0x7fa9c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:00.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.685+0000 7fa9d1600700 1 -- 172.21.15.97:0/33572936 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa9c8013070 con 0x7fa9d41024f0 2024-03-26T18:58:00.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.960+0000 7fa9da400700 1 -- 172.21.15.97:0/33572936 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9d410eb80 con 0x7fa9bc0a3030 2024-03-26T18:58:00.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.969+0000 7fa9d1600700 1 -- 172.21.15.97:0/33572936 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259101 (secure 0 0 0) 0x7fa9d410eb80 con 0x7fa9bc0a3030 2024-03-26T18:58:00.967 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.974+0000 7fa9da400700 1 -- 172.21.15.97:0/33572936 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9bc0a3030 msgr2=0x7fa9bc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.974+0000 7fa9da400700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9bc0a3030 0x7fa9bc0a54f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40ff660 tx=0x7fa9c0009260 comp rx=0 tx=0).stop 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.974+0000 7fa9da400700 1 -- 172.21.15.97:0/33572936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9d41024f0 msgr2=0x7fa9d41a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.974+0000 7fa9da400700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9d41024f0 0x7fa9d41a0880 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8000c00 tx=0x7fa9c800b6c0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.975+0000 7fa9da400700 1 -- 172.21.15.97:0/33572936 shutdown_connections 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.975+0000 7fa9da400700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa9bc0a3030 0x7fa9bc0a54f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.975+0000 7fa9da400700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d41075d0 0x7fa9d419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.975+0000 7fa9da400700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9d41024f0 0x7fa9d41a0880 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.975+0000 7fa9da400700 1 --2- 172.21.15.97:0/33572936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9d4102ea0 0x7fa9d41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.975+0000 7fa9da400700 1 -- 172.21.15.97:0/33572936 >> 172.21.15.97:0/33572936 conn(0x7fa9d40fdd80 msgr2=0x7fa9d410e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.975+0000 7fa9da400700 1 -- 172.21.15.97:0/33572936 shutdown_connections 2024-03-26T18:58:00.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:00.975+0000 7fa9da400700 1 -- 172.21.15.97:0/33572936 wait complete. 2024-03-26T18:58:00.970 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-26T18:58:01.916 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-03-26T18:58:00.937859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657003,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1058,"num_read_kb":648,"num_write":423,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":536,"ondisk_log_size":536,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365060,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576764,"statfs":{"total":1151940427776,"available":1151566606336,"internally_reserved":0,"allocated":11980800,"data_stored":7437000,"data_compressed":112698,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":71,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003794"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067144+0000","last_change":"2024-03-26T18:56:24.177714+0000","last_active":"2024-03-26T18:57:30.067144+0000","last_peered":"2024-03-26T18:57:30.067144+0000","last_clean":"2024-03-26T18:57:30.067144+0000","last_became_active":"2024-03-26T18:56:24.177420+0000","last_became_peered":"2024-03-26T18:56:24.177420+0000","last_unstale":"2024-03-26T18:57:30.067144+0000","last_undegraded":"2024-03-26T18:57:30.067144+0000","last_fullsized":"2024-03-26T18:57:30.067144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:21:23.738513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.072672+0000","last_change":"2024-03-26T18:56:24.160191+0000","last_active":"2024-03-26T18:57:30.072672+0000","last_peered":"2024-03-26T18:57:30.072672+0000","last_clean":"2024-03-26T18:57:30.072672+0000","last_became_active":"2024-03-26T18:56:24.159718+0000","last_became_peered":"2024-03-26T18:56:24.159718+0000","last_unstale":"2024-03-26T18:57:30.072672+0000","last_undegraded":"2024-03-26T18:57:30.072672+0000","last_fullsized":"2024-03-26T18:57:30.072672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:12:04.287352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.071083+0000","last_change":"2024-03-26T18:56:24.169795+0000","last_active":"2024-03-26T18:57:30.071083+0000","last_peered":"2024-03-26T18:57:30.071083+0000","last_clean":"2024-03-26T18:57:30.071083+0000","last_became_active":"2024-03-26T18:56:24.166044+0000","last_became_peered":"2024-03-26T18:56:24.166044+0000","last_unstale":"2024-03-26T18:57:30.071083+0000","last_undegraded":"2024-03-26T18:57:30.071083+0000","last_fullsized":"2024-03-26T18:57:30.071083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:21:58.951019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.897913+0000","last_change":"2024-03-26T18:57:30.072874+0000","last_active":"2024-03-26T18:57:58.897913+0000","last_peered":"2024-03-26T18:57:58.897913+0000","last_clean":"2024-03-26T18:57:58.897913+0000","last_became_active":"2024-03-26T18:57:30.072453+0000","last_became_peered":"2024-03-26T18:57:30.072453+0000","last_unstale":"2024-03-26T18:57:58.897913+0000","last_undegraded":"2024-03-26T18:57:58.897913+0000","last_fullsized":"2024-03-26T18:57:58.897913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:12:28.900636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.916304+0000","last_change":"2024-03-26T18:56:24.175902+0000","last_active":"2024-03-26T18:57:58.916304+0000","last_peered":"2024-03-26T18:57:58.916304+0000","last_clean":"2024-03-26T18:57:58.916304+0000","last_became_active":"2024-03-26T18:56:24.175568+0000","last_became_peered":"2024-03-26T18:56:24.175568+0000","last_unstale":"2024-03-26T18:57:58.916304+0000","last_undegraded":"2024-03-26T18:57:58.916304+0000","last_fullsized":"2024-03-26T18:57:58.916304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:33:14.054102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.905181+0000","last_change":"2024-03-26T18:56:24.145629+0000","last_active":"2024-03-26T18:57:58.905181+0000","last_peered":"2024-03-26T18:57:58.905181+0000","last_clean":"2024-03-26T18:57:58.905181+0000","last_became_active":"2024-03-26T18:56:24.145286+0000","last_became_peered":"2024-03-26T18:56:24.145286+0000","last_unstale":"2024-03-26T18:57:58.905181+0000","last_undegraded":"2024-03-26T18:57:58.905181+0000","last_fullsized":"2024-03-26T18:57:58.905181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:37:24.989479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.895723+0000","last_change":"2024-03-26T18:56:24.150869+0000","last_active":"2024-03-26T18:57:58.895723+0000","last_peered":"2024-03-26T18:57:58.895723+0000","last_clean":"2024-03-26T18:57:58.895723+0000","last_became_active":"2024-03-26T18:56:24.150474+0000","last_became_peered":"2024-03-26T18:56:24.150474+0000","last_unstale":"2024-03-26T18:57:58.895723+0000","last_undegraded":"2024-03-26T18:57:58.895723+0000","last_fullsized":"2024-03-26T18:57:58.895723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:55.113389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073698+0000","last_change":"2024-03-26T18:56:24.170218+0000","last_active":"2024-03-26T18:57:30.073698+0000","last_peered":"2024-03-26T18:57:30.073698+0000","last_clean":"2024-03-26T18:57:30.073698+0000","last_became_active":"2024-03-26T18:56:24.169343+0000","last_became_peered":"2024-03-26T18:56:24.169343+0000","last_unstale":"2024-03-26T18:57:30.073698+0000","last_undegraded":"2024-03-26T18:57:30.073698+0000","last_fullsized":"2024-03-26T18:57:30.073698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:28:41.609616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421778+0000","last_change":"2024-03-26T18:56:24.146892+0000","last_active":"2024-03-26T18:57:30.421778+0000","last_peered":"2024-03-26T18:57:30.421778+0000","last_clean":"2024-03-26T18:57:30.421778+0000","last_became_active":"2024-03-26T18:56:24.146564+0000","last_became_peered":"2024-03-26T18:56:24.146564+0000","last_unstale":"2024-03-26T18:57:30.421778+0000","last_undegraded":"2024-03-26T18:57:30.421778+0000","last_fullsized":"2024-03-26T18:57:30.421778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:12:42.104605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.908455+0000","last_change":"2024-03-26T18:56:24.148110+0000","last_active":"2024-03-26T18:57:58.908455+0000","last_peered":"2024-03-26T18:57:58.908455+0000","last_clean":"2024-03-26T18:57:58.908455+0000","last_became_active":"2024-03-26T18:56:24.147763+0000","last_became_peered":"2024-03-26T18:56:24.147763+0000","last_unstale":"2024-03-26T18:57:58.908455+0000","last_undegraded":"2024-03-26T18:57:58.908455+0000","last_fullsized":"2024-03-26T18:57:58.908455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:23.199428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.911179+0000","last_change":"2024-03-26T18:56:24.149064+0000","last_active":"2024-03-26T18:57:58.911179+0000","last_peered":"2024-03-26T18:57:58.911179+0000","last_clean":"2024-03-26T18:57:58.911179+0000","last_became_active":"2024-03-26T18:56:24.148400+0000","last_became_peered":"2024-03-26T18:56:24.148400+0000","last_unstale":"2024-03-26T18:57:58.911179+0000","last_undegraded":"2024-03-26T18:57:58.911179+0000","last_fullsized":"2024-03-26T18:57:58.911179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:08:14.141931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.905282+0000","last_change":"2024-03-26T18:57:30.424292+0000","last_active":"2024-03-26T18:57:58.905282+0000","last_peered":"2024-03-26T18:57:58.905282+0000","last_clean":"2024-03-26T18:57:58.905282+0000","last_became_active":"2024-03-26T18:57:30.423759+0000","last_became_peered":"2024-03-26T18:57:30.423759+0000","last_unstale":"2024-03-26T18:57:58.905282+0000","last_undegraded":"2024-03-26T18:57:58.905282+0000","last_fullsized":"2024-03-26T18:57:58.905282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:18:04.632643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.072441+0000","last_change":"2024-03-26T18:56:24.165819+0000","last_active":"2024-03-26T18:57:30.072441+0000","last_peered":"2024-03-26T18:57:30.072441+0000","last_clean":"2024-03-26T18:57:30.072441+0000","last_became_active":"2024-03-26T18:56:24.164401+0000","last_became_peered":"2024-03-26T18:56:24.164401+0000","last_unstale":"2024-03-26T18:57:30.072441+0000","last_undegraded":"2024-03-26T18:57:30.072441+0000","last_fullsized":"2024-03-26T18:57:30.072441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:08:32.301805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.071427+0000","last_change":"2024-03-26T18:56:24.174229+0000","last_active":"2024-03-26T18:57:30.071427+0000","last_peered":"2024-03-26T18:57:30.071427+0000","last_clean":"2024-03-26T18:57:30.071427+0000","last_became_active":"2024-03-26T18:56:24.172924+0000","last_became_peered":"2024-03-26T18:56:24.172924+0000","last_unstale":"2024-03-26T18:57:30.071427+0000","last_undegraded":"2024-03-26T18:57:30.071427+0000","last_fullsized":"2024-03-26T18:57:30.071427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:26:00.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.071084+0000","last_change":"2024-03-26T18:56:24.165029+0000","last_active":"2024-03-26T18:57:30.071084+0000","last_peered":"2024-03-26T18:57:30.071084+0000","last_clean":"2024-03-26T18:57:30.071084+0000","last_became_active":"2024-03-26T18:56:24.160238+0000","last_became_peered":"2024-03-26T18:56:24.160238+0000","last_unstale":"2024-03-26T18:57:30.071084+0000","last_undegraded":"2024-03-26T18:57:30.071084+0000","last_fullsized":"2024-03-26T18:57:30.071084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:56:14.509564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421231+0000","last_change":"2024-03-26T18:56:24.167668+0000","last_active":"2024-03-26T18:57:30.421231+0000","last_peered":"2024-03-26T18:57:30.421231+0000","last_clean":"2024-03-26T18:57:30.421231+0000","last_became_active":"2024-03-26T18:56:24.167267+0000","last_became_peered":"2024-03-26T18:56:24.167267+0000","last_unstale":"2024-03-26T18:57:30.421231+0000","last_undegraded":"2024-03-26T18:57:30.421231+0000","last_fullsized":"2024-03-26T18:57:30.421231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:13:00.503977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067366+0000","last_change":"2024-03-26T18:56:24.177623+0000","last_active":"2024-03-26T18:57:30.067366+0000","last_peered":"2024-03-26T18:57:30.067366+0000","last_clean":"2024-03-26T18:57:30.067366+0000","last_became_active":"2024-03-26T18:56:24.177372+0000","last_became_peered":"2024-03-26T18:56:24.177372+0000","last_unstale":"2024-03-26T18:57:30.067366+0000","last_undegraded":"2024-03-26T18:57:30.067366+0000","last_fullsized":"2024-03-26T18:57:30.067366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:02:20.650520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.916516+0000","last_change":"2024-03-26T18:56:24.152407+0000","last_active":"2024-03-26T18:57:58.916516+0000","last_peered":"2024-03-26T18:57:58.916516+0000","last_clean":"2024-03-26T18:57:58.916516+0000","last_became_active":"2024-03-26T18:56:24.151935+0000","last_became_peered":"2024-03-26T18:56:24.151935+0000","last_unstale":"2024-03-26T18:57:58.916516+0000","last_undegraded":"2024-03-26T18:57:58.916516+0000","last_fullsized":"2024-03-26T18:57:58.916516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:52:13.211124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.912014+0000","last_change":"2024-03-26T18:56:24.162582+0000","last_active":"2024-03-26T18:57:58.912014+0000","last_peered":"2024-03-26T18:57:58.912014+0000","last_clean":"2024-03-26T18:57:58.912014+0000","last_became_active":"2024-03-26T18:56:24.162228+0000","last_became_peered":"2024-03-26T18:56:24.162228+0000","last_unstale":"2024-03-26T18:57:58.912014+0000","last_undegraded":"2024-03-26T18:57:58.912014+0000","last_fullsized":"2024-03-26T18:57:58.912014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:16.841466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":120,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:56.989869+0000","last_change":"2024-03-26T18:56:23.149475+0000","last_active":"2024-03-26T18:57:56.989869+0000","last_peered":"2024-03-26T18:57:56.989869+0000","last_clean":"2024-03-26T18:57:56.989869+0000","last_became_active":"2024-03-26T18:56:23.148813+0000","last_became_peered":"2024-03-26T18:56:23.148813+0000","last_unstale":"2024-03-26T18:57:56.989869+0000","last_undegraded":"2024-03-26T18:57:56.989869+0000","last_fullsized":"2024-03-26T18:57:56.989869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:38.315414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073497+0000","last_change":"2024-03-26T18:56:24.173904+0000","last_active":"2024-03-26T18:57:30.073497+0000","last_peered":"2024-03-26T18:57:30.073497+0000","last_clean":"2024-03-26T18:57:30.073497+0000","last_became_active":"2024-03-26T18:56:24.173523+0000","last_became_peered":"2024-03-26T18:56:24.173523+0000","last_unstale":"2024-03-26T18:57:30.073497+0000","last_undegraded":"2024-03-26T18:57:30.073497+0000","last_fullsized":"2024-03-26T18:57:30.073497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:40:35.651821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073788+0000","last_change":"2024-03-26T18:56:23.173583+0000","last_active":"2024-03-26T18:57:30.073788+0000","last_peered":"2024-03-26T18:57:30.073788+0000","last_clean":"2024-03-26T18:57:30.073788+0000","last_became_active":"2024-03-26T18:56:23.172071+0000","last_became_peered":"2024-03-26T18:56:23.172071+0000","last_unstale":"2024-03-26T18:57:30.073788+0000","last_undegraded":"2024-03-26T18:57:30.073788+0000","last_fullsized":"2024-03-26T18:57:30.073788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:29:39.194411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073162+0000","last_change":"2024-03-26T18:56:24.175535+0000","last_active":"2024-03-26T18:57:30.073162+0000","last_peered":"2024-03-26T18:57:30.073162+0000","last_clean":"2024-03-26T18:57:30.073162+0000","last_became_active":"2024-03-26T18:56:24.173748+0000","last_became_peered":"2024-03-26T18:56:24.173748+0000","last_unstale":"2024-03-26T18:57:30.073162+0000","last_undegraded":"2024-03-26T18:57:30.073162+0000","last_fullsized":"2024-03-26T18:57:30.073162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:38.960606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.433759+0000","last_change":"2024-03-26T18:57:30.433759+0000","last_active":"2024-03-26T18:57:30.433759+0000","last_peered":"2024-03-26T18:57:30.433759+0000","last_clean":"2024-03-26T18:57:30.433759+0000","last_became_active":"2024-03-26T18:57:30.423507+0000","last_became_peered":"2024-03-26T18:57:30.423507+0000","last_unstale":"2024-03-26T18:57:30.433759+0000","last_undegraded":"2024-03-26T18:57:30.433759+0000","last_fullsized":"2024-03-26T18:57:30.433759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:31.051608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421113+0000","last_change":"2024-03-26T18:56:24.161351+0000","last_active":"2024-03-26T18:57:30.421113+0000","last_peered":"2024-03-26T18:57:30.421113+0000","last_clean":"2024-03-26T18:57:30.421113+0000","last_became_active":"2024-03-26T18:56:24.161013+0000","last_became_peered":"2024-03-26T18:56:24.161013+0000","last_unstale":"2024-03-26T18:57:30.421113+0000","last_undegraded":"2024-03-26T18:57:30.421113+0000","last_fullsized":"2024-03-26T18:57:30.421113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:24:25.620772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065706+0000","last_change":"2024-03-26T18:56:23.148866+0000","last_active":"2024-03-26T18:57:30.065706+0000","last_peered":"2024-03-26T18:57:30.065706+0000","last_clean":"2024-03-26T18:57:30.065706+0000","last_became_active":"2024-03-26T18:56:23.148434+0000","last_became_peered":"2024-03-26T18:56:23.148434+0000","last_unstale":"2024-03-26T18:57:30.065706+0000","last_undegraded":"2024-03-26T18:57:30.065706+0000","last_fullsized":"2024-03-26T18:57:30.065706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:35:35.180038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069795+0000","last_change":"2024-03-26T18:56:24.147429+0000","last_active":"2024-03-26T18:57:30.069795+0000","last_peered":"2024-03-26T18:57:30.069795+0000","last_clean":"2024-03-26T18:57:30.069795+0000","last_became_active":"2024-03-26T18:56:24.147100+0000","last_became_peered":"2024-03-26T18:56:24.147100+0000","last_unstale":"2024-03-26T18:57:30.069795+0000","last_undegraded":"2024-03-26T18:57:30.069795+0000","last_fullsized":"2024-03-26T18:57:30.069795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:08.884223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:56.904940+0000","last_change":"2024-03-26T18:56:23.149301+0000","last_active":"2024-03-26T18:57:56.904940+0000","last_peered":"2024-03-26T18:57:56.904940+0000","last_clean":"2024-03-26T18:57:56.904940+0000","last_became_active":"2024-03-26T18:56:23.148939+0000","last_became_peered":"2024-03-26T18:56:23.148939+0000","last_unstale":"2024-03-26T18:57:56.904940+0000","last_undegraded":"2024-03-26T18:57:56.904940+0000","last_fullsized":"2024-03-26T18:57:56.904940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:50:06.649474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.900307+0000","last_change":"2024-03-26T18:56:24.157317+0000","last_active":"2024-03-26T18:57:58.900307+0000","last_peered":"2024-03-26T18:57:58.900307+0000","last_clean":"2024-03-26T18:57:58.900307+0000","last_became_active":"2024-03-26T18:56:24.156971+0000","last_became_peered":"2024-03-26T18:56:24.156971+0000","last_unstale":"2024-03-26T18:57:58.900307+0000","last_undegraded":"2024-03-26T18:57:58.900307+0000","last_fullsized":"2024-03-26T18:57:58.900307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:22:39.360404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421437+0000","last_change":"2024-03-26T18:56:23.162848+0000","last_active":"2024-03-26T18:57:30.421437+0000","last_peered":"2024-03-26T18:57:30.421437+0000","last_clean":"2024-03-26T18:57:30.421437+0000","last_became_active":"2024-03-26T18:56:23.162478+0000","last_became_peered":"2024-03-26T18:56:23.162478+0000","last_unstale":"2024-03-26T18:57:30.421437+0000","last_undegraded":"2024-03-26T18:57:30.421437+0000","last_fullsized":"2024-03-26T18:57:30.421437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:07:51.139538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069742+0000","last_change":"2024-03-26T18:56:24.156451+0000","last_active":"2024-03-26T18:57:30.069742+0000","last_peered":"2024-03-26T18:57:30.069742+0000","last_clean":"2024-03-26T18:57:30.069742+0000","last_became_active":"2024-03-26T18:56:24.156237+0000","last_became_peered":"2024-03-26T18:56:24.156237+0000","last_unstale":"2024-03-26T18:57:30.069742+0000","last_undegraded":"2024-03-26T18:57:30.069742+0000","last_fullsized":"2024-03-26T18:57:30.069742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:52:54.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067202+0000","last_change":"2024-03-26T18:56:23.172926+0000","last_active":"2024-03-26T18:57:30.067202+0000","last_peered":"2024-03-26T18:57:30.067202+0000","last_clean":"2024-03-26T18:57:30.067202+0000","last_became_active":"2024-03-26T18:56:23.170512+0000","last_became_peered":"2024-03-26T18:56:23.170512+0000","last_unstale":"2024-03-26T18:57:30.067202+0000","last_undegraded":"2024-03-26T18:57:30.067202+0000","last_fullsized":"2024-03-26T18:57:30.067202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:16:13.747853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073932+0000","last_change":"2024-03-26T18:56:24.173733+0000","last_active":"2024-03-26T18:57:30.073932+0000","last_peered":"2024-03-26T18:57:30.073932+0000","last_clean":"2024-03-26T18:57:30.073932+0000","last_became_active":"2024-03-26T18:56:24.171895+0000","last_became_peered":"2024-03-26T18:56:24.171895+0000","last_unstale":"2024-03-26T18:57:30.073932+0000","last_undegraded":"2024-03-26T18:57:30.073932+0000","last_fullsized":"2024-03-26T18:57:30.073932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:07:59.273853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069293+0000","last_change":"2024-03-26T18:56:23.150140+0000","last_active":"2024-03-26T18:57:30.069293+0000","last_peered":"2024-03-26T18:57:30.069293+0000","last_clean":"2024-03-26T18:57:30.069293+0000","last_became_active":"2024-03-26T18:56:23.149634+0000","last_became_peered":"2024-03-26T18:56:23.149634+0000","last_unstale":"2024-03-26T18:57:30.069293+0000","last_undegraded":"2024-03-26T18:57:30.069293+0000","last_fullsized":"2024-03-26T18:57:30.069293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:52:48.025868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.912930+0000","last_change":"2024-03-26T18:56:24.157855+0000","last_active":"2024-03-26T18:57:58.912930+0000","last_peered":"2024-03-26T18:57:58.912930+0000","last_clean":"2024-03-26T18:57:58.912930+0000","last_became_active":"2024-03-26T18:56:24.157220+0000","last_became_peered":"2024-03-26T18:56:24.157220+0000","last_unstale":"2024-03-26T18:57:58.912930+0000","last_undegraded":"2024-03-26T18:57:58.912930+0000","last_fullsized":"2024-03-26T18:57:58.912930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:40:00.815178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":203,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:59.907427+0000","last_change":"2024-03-26T18:56:23.148341+0000","last_active":"2024-03-26T18:57:59.907427+0000","last_peered":"2024-03-26T18:57:59.907427+0000","last_clean":"2024-03-26T18:57:59.907427+0000","last_became_active":"2024-03-26T18:56:23.148021+0000","last_became_peered":"2024-03-26T18:56:23.148021+0000","last_unstale":"2024-03-26T18:57:59.907427+0000","last_undegraded":"2024-03-26T18:57:59.907427+0000","last_fullsized":"2024-03-26T18:57:59.907427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:42.577047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":180,"num_read_kb":180,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421726+0000","last_change":"2024-03-26T18:56:24.160017+0000","last_active":"2024-03-26T18:57:30.421726+0000","last_peered":"2024-03-26T18:57:30.421726+0000","last_clean":"2024-03-26T18:57:30.421726+0000","last_became_active":"2024-03-26T18:56:24.159686+0000","last_became_peered":"2024-03-26T18:56:24.159686+0000","last_unstale":"2024-03-26T18:57:30.421726+0000","last_undegraded":"2024-03-26T18:57:30.421726+0000","last_fullsized":"2024-03-26T18:57:30.421726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:33:32.693850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066656+0000","last_change":"2024-03-26T18:56:23.146487+0000","last_active":"2024-03-26T18:57:30.066656+0000","last_peered":"2024-03-26T18:57:30.066656+0000","last_clean":"2024-03-26T18:57:30.066656+0000","last_became_active":"2024-03-26T18:56:23.146155+0000","last_became_peered":"2024-03-26T18:56:23.146155+0000","last_unstale":"2024-03-26T18:57:30.066656+0000","last_undegraded":"2024-03-26T18:57:30.066656+0000","last_fullsized":"2024-03-26T18:57:30.066656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:09:51.836211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073126+0000","last_change":"2024-03-26T18:56:24.172652+0000","last_active":"2024-03-26T18:57:30.073126+0000","last_peered":"2024-03-26T18:57:30.073126+0000","last_clean":"2024-03-26T18:57:30.073126+0000","last_became_active":"2024-03-26T18:56:24.170778+0000","last_became_peered":"2024-03-26T18:56:24.170778+0000","last_unstale":"2024-03-26T18:57:30.073126+0000","last_undegraded":"2024-03-26T18:57:30.073126+0000","last_fullsized":"2024-03-26T18:57:30.073126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:47:16.946692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069815+0000","last_change":"2024-03-26T18:56:23.151669+0000","last_active":"2024-03-26T18:57:30.069815+0000","last_peered":"2024-03-26T18:57:30.069815+0000","last_clean":"2024-03-26T18:57:30.069815+0000","last_became_active":"2024-03-26T18:56:23.151276+0000","last_became_peered":"2024-03-26T18:56:23.151276+0000","last_unstale":"2024-03-26T18:57:30.069815+0000","last_undegraded":"2024-03-26T18:57:30.069815+0000","last_fullsized":"2024-03-26T18:57:30.069815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:28:37.827930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421291+0000","last_change":"2024-03-26T18:56:24.151007+0000","last_active":"2024-03-26T18:57:30.421291+0000","last_peered":"2024-03-26T18:57:30.421291+0000","last_clean":"2024-03-26T18:57:30.421291+0000","last_became_active":"2024-03-26T18:56:24.150668+0000","last_became_peered":"2024-03-26T18:56:24.150668+0000","last_unstale":"2024-03-26T18:57:30.421291+0000","last_undegraded":"2024-03-26T18:57:30.421291+0000","last_fullsized":"2024-03-26T18:57:30.421291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:07:04.778039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":244,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:59.913566+0000","last_change":"2024-03-26T18:56:23.144522+0000","last_active":"2024-03-26T18:57:59.913566+0000","last_peered":"2024-03-26T18:57:59.913566+0000","last_clean":"2024-03-26T18:57:59.913566+0000","last_became_active":"2024-03-26T18:56:23.144176+0000","last_became_peered":"2024-03-26T18:56:23.144176+0000","last_unstale":"2024-03-26T18:57:59.913566+0000","last_undegraded":"2024-03-26T18:57:59.913566+0000","last_fullsized":"2024-03-26T18:57:59.913566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:42:20.319478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59979,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":192,"num_read_kb":154,"num_write":35,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.897502+0000","last_change":"2024-03-26T18:56:24.148307+0000","last_active":"2024-03-26T18:57:58.897502+0000","last_peered":"2024-03-26T18:57:58.897502+0000","last_clean":"2024-03-26T18:57:58.897502+0000","last_became_active":"2024-03-26T18:56:24.147973+0000","last_became_peered":"2024-03-26T18:56:24.147973+0000","last_unstale":"2024-03-26T18:57:58.897502+0000","last_undegraded":"2024-03-26T18:57:58.897502+0000","last_fullsized":"2024-03-26T18:57:58.897502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:08:45.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.068804+0000","last_change":"2024-03-26T18:56:23.163757+0000","last_active":"2024-03-26T18:57:30.068804+0000","last_peered":"2024-03-26T18:57:30.068804+0000","last_clean":"2024-03-26T18:57:30.068804+0000","last_became_active":"2024-03-26T18:56:23.163418+0000","last_became_peered":"2024-03-26T18:56:23.163418+0000","last_unstale":"2024-03-26T18:57:30.068804+0000","last_undegraded":"2024-03-26T18:57:30.068804+0000","last_fullsized":"2024-03-26T18:57:30.068804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:04:09.178853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.068140+0000","last_change":"2024-03-26T18:56:24.167274+0000","last_active":"2024-03-26T18:57:30.068140+0000","last_peered":"2024-03-26T18:57:30.068140+0000","last_clean":"2024-03-26T18:57:30.068140+0000","last_became_active":"2024-03-26T18:56:24.166353+0000","last_became_peered":"2024-03-26T18:56:24.166353+0000","last_unstale":"2024-03-26T18:57:30.068140+0000","last_undegraded":"2024-03-26T18:57:30.068140+0000","last_fullsized":"2024-03-26T18:57:30.068140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:14:44.794112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069498+0000","last_change":"2024-03-26T18:56:23.162115+0000","last_active":"2024-03-26T18:57:30.069498+0000","last_peered":"2024-03-26T18:57:30.069498+0000","last_clean":"2024-03-26T18:57:30.069498+0000","last_became_active":"2024-03-26T18:56:23.161746+0000","last_became_peered":"2024-03-26T18:56:23.161746+0000","last_unstale":"2024-03-26T18:57:30.069498+0000","last_undegraded":"2024-03-26T18:57:30.069498+0000","last_fullsized":"2024-03-26T18:57:30.069498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:08:26.583281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.931745+0000","last_change":"2024-03-26T18:56:24.166407+0000","last_active":"2024-03-26T18:57:58.931745+0000","last_peered":"2024-03-26T18:57:58.931745+0000","last_clean":"2024-03-26T18:57:58.931745+0000","last_became_active":"2024-03-26T18:56:24.166077+0000","last_became_peered":"2024-03-26T18:56:24.166077+0000","last_unstale":"2024-03-26T18:57:58.931745+0000","last_undegraded":"2024-03-26T18:57:58.931745+0000","last_fullsized":"2024-03-26T18:57:58.931745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:36:35.160247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:56.939957+0000","last_change":"2024-03-26T18:56:23.146908+0000","last_active":"2024-03-26T18:57:56.939957+0000","last_peered":"2024-03-26T18:57:56.939957+0000","last_clean":"2024-03-26T18:57:56.939957+0000","last_became_active":"2024-03-26T18:56:23.146576+0000","last_became_peered":"2024-03-26T18:56:23.146576+0000","last_unstale":"2024-03-26T18:57:56.939957+0000","last_undegraded":"2024-03-26T18:57:56.939957+0000","last_fullsized":"2024-03-26T18:57:56.939957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:00:55.708101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.905497+0000","last_change":"2024-03-26T18:57:30.423372+0000","last_active":"2024-03-26T18:57:58.905497+0000","last_peered":"2024-03-26T18:57:58.905497+0000","last_clean":"2024-03-26T18:57:58.905497+0000","last_became_active":"2024-03-26T18:57:30.423022+0000","last_became_peered":"2024-03-26T18:57:30.423022+0000","last_unstale":"2024-03-26T18:57:58.905497+0000","last_undegraded":"2024-03-26T18:57:58.905497+0000","last_fullsized":"2024-03-26T18:57:58.905497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:04:52.867481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.085892+0000","last_change":"2024-03-26T18:57:30.085458+0000","last_active":"2024-03-26T18:57:30.085892+0000","last_peered":"2024-03-26T18:57:30.085892+0000","last_clean":"2024-03-26T18:57:30.085892+0000","last_became_active":"2024-03-26T18:57:30.073182+0000","last_became_peered":"2024-03-26T18:57:30.073182+0000","last_unstale":"2024-03-26T18:57:30.085892+0000","last_undegraded":"2024-03-26T18:57:30.085892+0000","last_fullsized":"2024-03-26T18:57:30.085892+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:32:31.999866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069107+0000","last_change":"2024-03-26T18:56:24.146993+0000","last_active":"2024-03-26T18:57:30.069107+0000","last_peered":"2024-03-26T18:57:30.069107+0000","last_clean":"2024-03-26T18:57:30.069107+0000","last_became_active":"2024-03-26T18:56:24.146692+0000","last_became_peered":"2024-03-26T18:56:24.146692+0000","last_unstale":"2024-03-26T18:57:30.069107+0000","last_undegraded":"2024-03-26T18:57:30.069107+0000","last_fullsized":"2024-03-26T18:57:30.069107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:08:33.731716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065752+0000","last_change":"2024-03-26T18:56:23.148459+0000","last_active":"2024-03-26T18:57:30.065752+0000","last_peered":"2024-03-26T18:57:30.065752+0000","last_clean":"2024-03-26T18:57:30.065752+0000","last_became_active":"2024-03-26T18:56:23.148097+0000","last_became_peered":"2024-03-26T18:56:23.148097+0000","last_unstale":"2024-03-26T18:57:30.065752+0000","last_undegraded":"2024-03-26T18:57:30.065752+0000","last_fullsized":"2024-03-26T18:57:30.065752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:08:51.990705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066974+0000","last_change":"2024-03-26T18:56:24.146399+0000","last_active":"2024-03-26T18:57:30.066974+0000","last_peered":"2024-03-26T18:57:30.066974+0000","last_clean":"2024-03-26T18:57:30.066974+0000","last_became_active":"2024-03-26T18:56:24.146014+0000","last_became_peered":"2024-03-26T18:56:24.146014+0000","last_unstale":"2024-03-26T18:57:30.066974+0000","last_undegraded":"2024-03-26T18:57:30.066974+0000","last_fullsized":"2024-03-26T18:57:30.066974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:24:26.809602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069836+0000","last_change":"2024-03-26T18:56:23.148037+0000","last_active":"2024-03-26T18:57:30.069836+0000","last_peered":"2024-03-26T18:57:30.069836+0000","last_clean":"2024-03-26T18:57:30.069836+0000","last_became_active":"2024-03-26T18:56:23.147598+0000","last_became_peered":"2024-03-26T18:56:23.147598+0000","last_unstale":"2024-03-26T18:57:30.069836+0000","last_undegraded":"2024-03-26T18:57:30.069836+0000","last_fullsized":"2024-03-26T18:57:30.069836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:21:55.105299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.919622+0000","last_change":"2024-03-26T18:56:24.167488+0000","last_active":"2024-03-26T18:57:58.919622+0000","last_peered":"2024-03-26T18:57:58.919622+0000","last_clean":"2024-03-26T18:57:58.919622+0000","last_became_active":"2024-03-26T18:56:24.167154+0000","last_became_peered":"2024-03-26T18:56:24.167154+0000","last_unstale":"2024-03-26T18:57:58.919622+0000","last_undegraded":"2024-03-26T18:57:58.919622+0000","last_fullsized":"2024-03-26T18:57:58.919622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:00:35.606573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:57.991551+0000","last_change":"2024-03-26T18:56:23.160694+0000","last_active":"2024-03-26T18:57:57.991551+0000","last_peered":"2024-03-26T18:57:57.991551+0000","last_clean":"2024-03-26T18:57:57.991551+0000","last_became_active":"2024-03-26T18:56:23.160416+0000","last_became_peered":"2024-03-26T18:56:23.160416+0000","last_unstale":"2024-03-26T18:57:57.991551+0000","last_undegraded":"2024-03-26T18:57:57.991551+0000","last_fullsized":"2024-03-26T18:57:57.991551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:31:15.616473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.914934+0000","last_change":"2024-03-26T18:56:24.170503+0000","last_active":"2024-03-26T18:57:58.914934+0000","last_peered":"2024-03-26T18:57:58.914934+0000","last_clean":"2024-03-26T18:57:58.914934+0000","last_became_active":"2024-03-26T18:56:24.170113+0000","last_became_peered":"2024-03-26T18:56:24.170113+0000","last_unstale":"2024-03-26T18:57:58.914934+0000","last_undegraded":"2024-03-26T18:57:58.914934+0000","last_fullsized":"2024-03-26T18:57:58.914934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:44.076756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069474+0000","last_change":"2024-03-26T18:56:24.146856+0000","last_active":"2024-03-26T18:57:30.069474+0000","last_peered":"2024-03-26T18:57:30.069474+0000","last_clean":"2024-03-26T18:57:30.069474+0000","last_became_active":"2024-03-26T18:56:24.146545+0000","last_became_peered":"2024-03-26T18:56:24.146545+0000","last_unstale":"2024-03-26T18:57:30.069474+0000","last_undegraded":"2024-03-26T18:57:30.069474+0000","last_fullsized":"2024-03-26T18:57:30.069474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:58:26.876614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:56.939387+0000","last_change":"2024-03-26T18:56:23.173650+0000","last_active":"2024-03-26T18:57:56.939387+0000","last_peered":"2024-03-26T18:57:56.939387+0000","last_clean":"2024-03-26T18:57:56.939387+0000","last_became_active":"2024-03-26T18:56:23.173343+0000","last_became_peered":"2024-03-26T18:56:23.173343+0000","last_unstale":"2024-03-26T18:57:56.939387+0000","last_undegraded":"2024-03-26T18:57:56.939387+0000","last_fullsized":"2024-03-26T18:57:56.939387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:42:51.138892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073926+0000","last_change":"2024-03-26T18:56:24.175697+0000","last_active":"2024-03-26T18:57:30.073926+0000","last_peered":"2024-03-26T18:57:30.073926+0000","last_clean":"2024-03-26T18:57:30.073926+0000","last_became_active":"2024-03-26T18:56:24.175159+0000","last_became_peered":"2024-03-26T18:56:24.175159+0000","last_unstale":"2024-03-26T18:57:30.073926+0000","last_undegraded":"2024-03-26T18:57:30.073926+0000","last_fullsized":"2024-03-26T18:57:30.073926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:02:35.581432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.422165+0000","last_change":"2024-03-26T18:56:23.154766+0000","last_active":"2024-03-26T18:57:30.422165+0000","last_peered":"2024-03-26T18:57:30.422165+0000","last_clean":"2024-03-26T18:57:30.422165+0000","last_became_active":"2024-03-26T18:56:23.154475+0000","last_became_peered":"2024-03-26T18:56:23.154475+0000","last_unstale":"2024-03-26T18:57:30.422165+0000","last_undegraded":"2024-03-26T18:57:30.422165+0000","last_fullsized":"2024-03-26T18:57:30.422165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:29:38.484201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066918+0000","last_change":"2024-03-26T18:55:03.797254+0000","last_active":"2024-03-26T18:57:30.066918+0000","last_peered":"2024-03-26T18:57:30.066918+0000","last_clean":"2024-03-26T18:57:30.066918+0000","last_became_active":"2024-03-26T18:55:03.796795+0000","last_became_peered":"2024-03-26T18:55:03.796795+0000","last_unstale":"2024-03-26T18:57:30.066918+0000","last_undegraded":"2024-03-26T18:57:30.066918+0000","last_fullsized":"2024-03-26T18:57:30.066918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:51:11.590736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:51:11.590736+0000","last_clean_scrub_stamp":"2024-03-26T18:51:11.590736+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:48.215695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066872+0000","last_change":"2024-03-26T18:56:24.165552+0000","last_active":"2024-03-26T18:57:30.066872+0000","last_peered":"2024-03-26T18:57:30.066872+0000","last_clean":"2024-03-26T18:57:30.066872+0000","last_became_active":"2024-03-26T18:56:24.164230+0000","last_became_peered":"2024-03-26T18:56:24.164230+0000","last_unstale":"2024-03-26T18:57:30.066872+0000","last_undegraded":"2024-03-26T18:57:30.066872+0000","last_fullsized":"2024-03-26T18:57:30.066872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:36:19.750451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.426540+0000","last_change":"2024-03-26T18:57:30.426540+0000","last_active":"2024-03-26T18:57:30.426540+0000","last_peered":"2024-03-26T18:57:30.426540+0000","last_clean":"2024-03-26T18:57:30.426540+0000","last_became_active":"2024-03-26T18:57:30.423235+0000","last_became_peered":"2024-03-26T18:57:30.423235+0000","last_unstale":"2024-03-26T18:57:30.426540+0000","last_undegraded":"2024-03-26T18:57:30.426540+0000","last_fullsized":"2024-03-26T18:57:30.426540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:36:08.904800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065868+0000","last_change":"2024-03-26T18:56:23.154066+0000","last_active":"2024-03-26T18:57:30.065868+0000","last_peered":"2024-03-26T18:57:30.065868+0000","last_clean":"2024-03-26T18:57:30.065868+0000","last_became_active":"2024-03-26T18:56:23.153813+0000","last_became_peered":"2024-03-26T18:56:23.153813+0000","last_unstale":"2024-03-26T18:57:30.065868+0000","last_undegraded":"2024-03-26T18:57:30.065868+0000","last_fullsized":"2024-03-26T18:57:30.065868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:25:07.976414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067811+0000","last_change":"2024-03-26T18:56:24.162166+0000","last_active":"2024-03-26T18:57:30.067811+0000","last_peered":"2024-03-26T18:57:30.067811+0000","last_clean":"2024-03-26T18:57:30.067811+0000","last_became_active":"2024-03-26T18:56:24.161623+0000","last_became_peered":"2024-03-26T18:56:24.161623+0000","last_unstale":"2024-03-26T18:57:30.067811+0000","last_undegraded":"2024-03-26T18:57:30.067811+0000","last_fullsized":"2024-03-26T18:57:30.067811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:42.588275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'13","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.070450+0000","last_change":"2024-03-26T18:56:23.167462+0000","last_active":"2024-03-26T18:57:30.070450+0000","last_peered":"2024-03-26T18:57:30.070450+0000","last_clean":"2024-03-26T18:57:30.070450+0000","last_became_active":"2024-03-26T18:56:23.167124+0000","last_became_peered":"2024-03-26T18:56:23.167124+0000","last_unstale":"2024-03-26T18:57:30.070450+0000","last_undegraded":"2024-03-26T18:57:30.070450+0000","last_fullsized":"2024-03-26T18:57:30.070450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:38.550421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.074654+0000","last_change":"2024-03-26T18:56:24.173164+0000","last_active":"2024-03-26T18:57:30.074654+0000","last_peered":"2024-03-26T18:57:30.074654+0000","last_clean":"2024-03-26T18:57:30.074654+0000","last_became_active":"2024-03-26T18:56:24.170083+0000","last_became_peered":"2024-03-26T18:56:24.170083+0000","last_unstale":"2024-03-26T18:57:30.074654+0000","last_undegraded":"2024-03-26T18:57:30.074654+0000","last_fullsized":"2024-03-26T18:57:30.074654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:26.696625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.422308+0000","last_change":"2024-03-26T18:56:23.162835+0000","last_active":"2024-03-26T18:57:30.422308+0000","last_peered":"2024-03-26T18:57:30.422308+0000","last_clean":"2024-03-26T18:57:30.422308+0000","last_became_active":"2024-03-26T18:56:23.162497+0000","last_became_peered":"2024-03-26T18:56:23.162497+0000","last_unstale":"2024-03-26T18:57:30.422308+0000","last_undegraded":"2024-03-26T18:57:30.422308+0000","last_fullsized":"2024-03-26T18:57:30.422308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:29:47.903712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.071309+0000","last_change":"2024-03-26T18:56:24.178140+0000","last_active":"2024-03-26T18:57:30.071309+0000","last_peered":"2024-03-26T18:57:30.071309+0000","last_clean":"2024-03-26T18:57:30.071309+0000","last_became_active":"2024-03-26T18:56:24.177769+0000","last_became_peered":"2024-03-26T18:56:24.177769+0000","last_unstale":"2024-03-26T18:57:30.071309+0000","last_undegraded":"2024-03-26T18:57:30.071309+0000","last_fullsized":"2024-03-26T18:57:30.071309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:39.714335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421323+0000","last_change":"2024-03-26T18:56:23.143972+0000","last_active":"2024-03-26T18:57:30.421323+0000","last_peered":"2024-03-26T18:57:30.421323+0000","last_clean":"2024-03-26T18:57:30.421323+0000","last_became_active":"2024-03-26T18:56:23.143395+0000","last_became_peered":"2024-03-26T18:56:23.143395+0000","last_unstale":"2024-03-26T18:57:30.421323+0000","last_undegraded":"2024-03-26T18:57:30.421323+0000","last_fullsized":"2024-03-26T18:57:30.421323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:46:32.548546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.866670+0000","last_change":"2024-03-26T18:56:24.144839+0000","last_active":"2024-03-26T18:57:58.866670+0000","last_peered":"2024-03-26T18:57:58.866670+0000","last_clean":"2024-03-26T18:57:58.866670+0000","last_became_active":"2024-03-26T18:56:24.144519+0000","last_became_peered":"2024-03-26T18:56:24.144519+0000","last_unstale":"2024-03-26T18:57:58.866670+0000","last_undegraded":"2024-03-26T18:57:58.866670+0000","last_fullsized":"2024-03-26T18:57:58.866670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:54:59.787475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073699+0000","last_change":"2024-03-26T18:56:23.161380+0000","last_active":"2024-03-26T18:57:30.073699+0000","last_peered":"2024-03-26T18:57:30.073699+0000","last_clean":"2024-03-26T18:57:30.073699+0000","last_became_active":"2024-03-26T18:56:23.160376+0000","last_became_peered":"2024-03-26T18:56:23.160376+0000","last_unstale":"2024-03-26T18:57:30.073699+0000","last_undegraded":"2024-03-26T18:57:30.073699+0000","last_fullsized":"2024-03-26T18:57:30.073699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:49.610002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067094+0000","last_change":"2024-03-26T18:56:24.170284+0000","last_active":"2024-03-26T18:57:30.067094+0000","last_peered":"2024-03-26T18:57:30.067094+0000","last_clean":"2024-03-26T18:57:30.067094+0000","last_became_active":"2024-03-26T18:56:24.169954+0000","last_became_peered":"2024-03-26T18:56:24.169954+0000","last_unstale":"2024-03-26T18:57:30.067094+0000","last_undegraded":"2024-03-26T18:57:30.067094+0000","last_fullsized":"2024-03-26T18:57:30.067094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:05:21.383976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.088798+0000","last_change":"2024-03-26T18:57:30.088435+0000","last_active":"2024-03-26T18:57:30.088798+0000","last_peered":"2024-03-26T18:57:30.088798+0000","last_clean":"2024-03-26T18:57:30.088798+0000","last_became_active":"2024-03-26T18:57:30.075160+0000","last_became_peered":"2024-03-26T18:57:30.075160+0000","last_unstale":"2024-03-26T18:57:30.088798+0000","last_undegraded":"2024-03-26T18:57:30.088798+0000","last_fullsized":"2024-03-26T18:57:30.088798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:24:58.325454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.074292+0000","last_change":"2024-03-26T18:56:24.176940+0000","last_active":"2024-03-26T18:57:30.074292+0000","last_peered":"2024-03-26T18:57:30.074292+0000","last_clean":"2024-03-26T18:57:30.074292+0000","last_became_active":"2024-03-26T18:56:24.176396+0000","last_became_peered":"2024-03-26T18:56:24.176396+0000","last_unstale":"2024-03-26T18:57:30.074292+0000","last_undegraded":"2024-03-26T18:57:30.074292+0000","last_fullsized":"2024-03-26T18:57:30.074292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:53:06.994055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.065440+0000","last_change":"2024-03-26T18:56:23.149738+0000","last_active":"2024-03-26T18:57:30.065440+0000","last_peered":"2024-03-26T18:57:30.065440+0000","last_clean":"2024-03-26T18:57:30.065440+0000","last_became_active":"2024-03-26T18:56:23.149399+0000","last_became_peered":"2024-03-26T18:56:23.149399+0000","last_unstale":"2024-03-26T18:57:30.065440+0000","last_undegraded":"2024-03-26T18:57:30.065440+0000","last_fullsized":"2024-03-26T18:57:30.065440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:44:50.666181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.930643+0000","last_change":"2024-03-26T18:56:24.161505+0000","last_active":"2024-03-26T18:57:58.930643+0000","last_peered":"2024-03-26T18:57:58.930643+0000","last_clean":"2024-03-26T18:57:58.930643+0000","last_became_active":"2024-03-26T18:56:24.161035+0000","last_became_peered":"2024-03-26T18:56:24.161035+0000","last_unstale":"2024-03-26T18:57:58.930643+0000","last_undegraded":"2024-03-26T18:57:58.930643+0000","last_fullsized":"2024-03-26T18:57:58.930643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:22:30.062398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.940159+0000","last_change":"2024-03-26T18:56:23.150900+0000","last_active":"2024-03-26T18:57:58.940159+0000","last_peered":"2024-03-26T18:57:58.940159+0000","last_clean":"2024-03-26T18:57:58.940159+0000","last_became_active":"2024-03-26T18:56:23.150578+0000","last_became_peered":"2024-03-26T18:56:23.150578+0000","last_unstale":"2024-03-26T18:57:58.940159+0000","last_undegraded":"2024-03-26T18:57:58.940159+0000","last_fullsized":"2024-03-26T18:57:58.940159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:18:09.039791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.911393+0000","last_change":"2024-03-26T18:56:24.148744+0000","last_active":"2024-03-26T18:57:58.911393+0000","last_peered":"2024-03-26T18:57:58.911393+0000","last_clean":"2024-03-26T18:57:58.911393+0000","last_became_active":"2024-03-26T18:56:24.147857+0000","last_became_peered":"2024-03-26T18:56:24.147857+0000","last_unstale":"2024-03-26T18:57:58.911393+0000","last_undegraded":"2024-03-26T18:57:58.911393+0000","last_fullsized":"2024-03-26T18:57:58.911393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:59:15.890399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069711+0000","last_change":"2024-03-26T18:56:23.165228+0000","last_active":"2024-03-26T18:57:30.069711+0000","last_peered":"2024-03-26T18:57:30.069711+0000","last_clean":"2024-03-26T18:57:30.069711+0000","last_became_active":"2024-03-26T18:56:23.164852+0000","last_became_peered":"2024-03-26T18:56:23.164852+0000","last_unstale":"2024-03-26T18:57:30.069711+0000","last_undegraded":"2024-03-26T18:57:30.069711+0000","last_fullsized":"2024-03-26T18:57:30.069711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:16:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.902805+0000","last_change":"2024-03-26T18:56:24.154235+0000","last_active":"2024-03-26T18:57:58.902805+0000","last_peered":"2024-03-26T18:57:58.902805+0000","last_clean":"2024-03-26T18:57:58.902805+0000","last_became_active":"2024-03-26T18:56:24.153864+0000","last_became_peered":"2024-03-26T18:56:24.153864+0000","last_unstale":"2024-03-26T18:57:58.902805+0000","last_undegraded":"2024-03-26T18:57:58.902805+0000","last_fullsized":"2024-03-26T18:57:58.902805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:30:36.577695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067042+0000","last_change":"2024-03-26T18:56:23.175807+0000","last_active":"2024-03-26T18:57:30.067042+0000","last_peered":"2024-03-26T18:57:30.067042+0000","last_clean":"2024-03-26T18:57:30.067042+0000","last_became_active":"2024-03-26T18:56:23.175477+0000","last_became_peered":"2024-03-26T18:56:23.175477+0000","last_unstale":"2024-03-26T18:57:30.067042+0000","last_undegraded":"2024-03-26T18:57:30.067042+0000","last_fullsized":"2024-03-26T18:57:30.067042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:22.119294+0000","last_clean_scrub_stamp":"2024-03-26T18:56:22.119294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:45:28.494823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.904581+0000","last_change":"2024-03-26T18:56:24.145201+0000","last_active":"2024-03-26T18:57:58.904581+0000","last_peered":"2024-03-26T18:57:58.904581+0000","last_clean":"2024-03-26T18:57:58.904581+0000","last_became_active":"2024-03-26T18:56:24.144793+0000","last_became_peered":"2024-03-26T18:56:24.144793+0000","last_unstale":"2024-03-26T18:57:58.904581+0000","last_undegraded":"2024-03-26T18:57:58.904581+0000","last_fullsized":"2024-03-26T18:57:58.904581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:13:55.476075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.067210+0000","last_change":"2024-03-26T18:56:24.145445+0000","last_active":"2024-03-26T18:57:30.067210+0000","last_peered":"2024-03-26T18:57:30.067210+0000","last_clean":"2024-03-26T18:57:30.067210+0000","last_became_active":"2024-03-26T18:56:24.145115+0000","last_became_peered":"2024-03-26T18:56:24.145115+0000","last_unstale":"2024-03-26T18:57:30.067210+0000","last_undegraded":"2024-03-26T18:57:30.067210+0000","last_fullsized":"2024-03-26T18:57:30.067210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:30:45.728966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.902915+0000","last_change":"2024-03-26T18:56:24.148307+0000","last_active":"2024-03-26T18:57:58.902915+0000","last_peered":"2024-03-26T18:57:58.902915+0000","last_clean":"2024-03-26T18:57:58.902915+0000","last_became_active":"2024-03-26T18:56:24.147973+0000","last_became_peered":"2024-03-26T18:56:24.147973+0000","last_unstale":"2024-03-26T18:57:58.902915+0000","last_undegraded":"2024-03-26T18:57:58.902915+0000","last_fullsized":"2024-03-26T18:57:58.902915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:36:23.330139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.909822+0000","last_change":"2024-03-26T18:56:24.175594+0000","last_active":"2024-03-26T18:57:58.909822+0000","last_peered":"2024-03-26T18:57:58.909822+0000","last_clean":"2024-03-26T18:57:58.909822+0000","last_became_active":"2024-03-26T18:56:24.175261+0000","last_became_peered":"2024-03-26T18:56:24.175261+0000","last_unstale":"2024-03-26T18:57:58.909822+0000","last_undegraded":"2024-03-26T18:57:58.909822+0000","last_fullsized":"2024-03-26T18:57:58.909822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:12:02.744475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.072014+0000","last_change":"2024-03-26T18:56:24.177648+0000","last_active":"2024-03-26T18:57:30.072014+0000","last_peered":"2024-03-26T18:57:30.072014+0000","last_clean":"2024-03-26T18:57:30.072014+0000","last_became_active":"2024-03-26T18:56:24.176768+0000","last_became_peered":"2024-03-26T18:56:24.176768+0000","last_unstale":"2024-03-26T18:57:30.072014+0000","last_undegraded":"2024-03-26T18:57:30.072014+0000","last_fullsized":"2024-03-26T18:57:30.072014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:24:10.207407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.068087+0000","last_change":"2024-03-26T18:56:24.166746+0000","last_active":"2024-03-26T18:57:30.068087+0000","last_peered":"2024-03-26T18:57:30.068087+0000","last_clean":"2024-03-26T18:57:30.068087+0000","last_became_active":"2024-03-26T18:56:24.166220+0000","last_became_peered":"2024-03-26T18:56:24.166220+0000","last_unstale":"2024-03-26T18:57:30.068087+0000","last_undegraded":"2024-03-26T18:57:30.068087+0000","last_fullsized":"2024-03-26T18:57:30.068087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:16:54.411740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066856+0000","last_change":"2024-03-26T18:56:24.145511+0000","last_active":"2024-03-26T18:57:30.066856+0000","last_peered":"2024-03-26T18:57:30.066856+0000","last_clean":"2024-03-26T18:57:30.066856+0000","last_became_active":"2024-03-26T18:56:24.145164+0000","last_became_peered":"2024-03-26T18:56:24.145164+0000","last_unstale":"2024-03-26T18:57:30.066856+0000","last_undegraded":"2024-03-26T18:57:30.066856+0000","last_fullsized":"2024-03-26T18:57:30.066856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:25:05.239952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:58.898895+0000","last_change":"2024-03-26T18:56:24.145367+0000","last_active":"2024-03-26T18:57:58.898895+0000","last_peered":"2024-03-26T18:57:58.898895+0000","last_clean":"2024-03-26T18:57:58.898895+0000","last_became_active":"2024-03-26T18:56:24.144058+0000","last_became_peered":"2024-03-26T18:56:24.144058+0000","last_unstale":"2024-03-26T18:57:58.898895+0000","last_undegraded":"2024-03-26T18:57:58.898895+0000","last_fullsized":"2024-03-26T18:57:58.898895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:10.440142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.074415+0000","last_change":"2024-03-26T18:56:24.174209+0000","last_active":"2024-03-26T18:57:30.074415+0000","last_peered":"2024-03-26T18:57:30.074415+0000","last_clean":"2024-03-26T18:57:30.074415+0000","last_became_active":"2024-03-26T18:56:24.173873+0000","last_became_peered":"2024-03-26T18:56:24.173873+0000","last_unstale":"2024-03-26T18:57:30.074415+0000","last_undegraded":"2024-03-26T18:57:30.074415+0000","last_fullsized":"2024-03-26T18:57:30.074415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:21:34.317699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.069391+0000","last_change":"2024-03-26T18:56:24.166633+0000","last_active":"2024-03-26T18:57:30.069391+0000","last_peered":"2024-03-26T18:57:30.069391+0000","last_clean":"2024-03-26T18:57:30.069391+0000","last_became_active":"2024-03-26T18:56:24.166372+0000","last_became_peered":"2024-03-26T18:56:24.166372+0000","last_unstale":"2024-03-26T18:57:30.069391+0000","last_undegraded":"2024-03-26T18:57:30.069391+0000","last_fullsized":"2024-03-26T18:57:30.069391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:38:18.005223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.073449+0000","last_change":"2024-03-26T18:56:24.179979+0000","last_active":"2024-03-26T18:57:30.073449+0000","last_peered":"2024-03-26T18:57:30.073449+0000","last_clean":"2024-03-26T18:57:30.073449+0000","last_became_active":"2024-03-26T18:56:24.179533+0000","last_became_peered":"2024-03-26T18:56:24.179533+0000","last_unstale":"2024-03-26T18:57:30.073449+0000","last_undegraded":"2024-03-26T18:57:30.073449+0000","last_fullsized":"2024-03-26T18:57:30.073449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:19:26.104430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.421466+0000","last_change":"2024-03-26T18:56:24.175802+0000","last_active":"2024-03-26T18:57:30.421466+0000","last_peered":"2024-03-26T18:57:30.421466+0000","last_clean":"2024-03-26T18:57:30.421466+0000","last_became_active":"2024-03-26T18:56:24.175451+0000","last_became_peered":"2024-03-26T18:56:24.175451+0000","last_unstale":"2024-03-26T18:57:30.421466+0000","last_undegraded":"2024-03-26T18:57:30.421466+0000","last_fullsized":"2024-03-26T18:57:30.421466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:48:07.444891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.072336+0000","last_change":"2024-03-26T18:56:24.167708+0000","last_active":"2024-03-26T18:57:30.072336+0000","last_peered":"2024-03-26T18:57:30.072336+0000","last_clean":"2024-03-26T18:57:30.072336+0000","last_became_active":"2024-03-26T18:56:24.166985+0000","last_became_peered":"2024-03-26T18:56:24.166985+0000","last_unstale":"2024-03-26T18:57:30.072336+0000","last_undegraded":"2024-03-26T18:57:30.072336+0000","last_fullsized":"2024-03-26T18:57:30.072336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:13:35.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T18:57:30.066573+0000","last_change":"2024-03-26T18:56:24.151474+0000","last_active":"2024-03-26T18:57:30.066573+0000","last_peered":"2024-03-26T18:57:30.066573+0000","last_clean":"2024-03-26T18:57:30.066573+0000","last_became_active":"2024-03-26T18:56:24.151151+0000","last_became_peered":"2024-03-26T18:56:24.151151+0000","last_unstale":"2024-03-26T18:57:30.066573+0000","last_undegraded":"2024-03-26T18:57:30.066573+0000","last_fullsized":"2024-03-26T18:57:30.066573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T18:56:23.125020+0000","last_clean_scrub_stamp":"2024-03-26T18:56:23.125020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:34.926612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":191,"ondisk_log_size":191,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280636,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66401,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":932,"num_read_kb":539,"num_write":240,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":198954,"data_compressed":5610,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383712,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1107720,"data_compressed":37566,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":456642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":456908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1049147,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":517002,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":457698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":517471,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":456184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1048825,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35696,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61482,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61951,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61715,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T18:58:01.920 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-26T18:58:01.920 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-26T18:58:01.921 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-26T18:58:01.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-26T18:58:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:58:02.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:02 smithi105 ceph-mon[36270]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T18:58:02.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:02 smithi105 ceph-mon[36270]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:58:02.542 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:02 smithi097 ceph-mon[31955]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T18:58:02.542 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:02 smithi097 ceph-mon[31955]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:58:02.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:02 smithi144 ceph-mon[36053]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T18:58:02.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:02 smithi144 ceph-mon[36053]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:58:03.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:03 smithi105 ceph-mon[36270]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T18:58:03.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:03 smithi097 ceph-mon[31955]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T18:58:03.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:03 smithi144 ceph-mon[36053]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T18:58:04.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.680+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093165345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c7410eb30 msgr2=0x7f6c74110f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:04.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.680+0000 7f6c7a600700 1 --2- 172.21.15.97:0/2093165345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c7410eb30 0x7f6c74110f40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00b0a0 tx=0x7f6c6c00b3b0 comp rx=0 tx=0).stop 2024-03-26T18:58:04.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.680+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093165345 shutdown_connections 2024-03-26T18:58:04.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.680+0000 7f6c7a600700 1 --2- 172.21.15.97:0/2093165345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c7410eb30 0x7f6c74110f40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:04.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.680+0000 7f6c7a600700 1 --2- 172.21.15.97:0/2093165345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c74074780 0x7f6c74074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:04.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.680+0000 7f6c7a600700 1 --2- 172.21.15.97:0/2093165345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c740750a0 0x7f6c7410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:04.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.680+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093165345 >> 172.21.15.97:0/2093165345 conn(0x7f6c740fdda0 msgr2=0x7f6c741001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:04.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.680+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093165345 shutdown_connections 2024-03-26T18:58:04.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.680+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093165345 wait complete. 2024-03-26T18:58:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.682+0000 7f6c7a600700 1 Processor -- start 2024-03-26T18:58:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.682+0000 7f6c7a600700 1 -- start start 2024-03-26T18:58:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c7a600700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c74074780 0x7f6c741a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c7a600700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c740750a0 0x7f6c741a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c7a600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7410eb30 0x7f6c7419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c7a600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74114810 con 0x7f6c7410eb30 2024-03-26T18:58:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c7a600700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74114690 con 0x7f6c74074780 2024-03-26T18:58:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c7a600700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74114990 con 0x7f6c740750a0 2024-03-26T18:58:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c79200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c74074780 0x7f6c741a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c79200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c74074780 0x7f6c741a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54180/0 (socket says 172.21.15.97:54180) 2024-03-26T18:58:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c79200700 1 -- 172.21.15.97:0/2093512493 learned_addr learned my addr 172.21.15.97:0/2093512493 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c73e00700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c740750a0 0x7f6c741a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:04.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c79c00700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7410eb30 0x7f6c7419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:04.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c73e00700 1 -- 172.21.15.97:0/2093512493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c74074780 msgr2=0x7f6c741a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:04.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c73e00700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c74074780 0x7f6c741a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:04.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c73e00700 1 -- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7410eb30 msgr2=0x7f6c7419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:04.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c73e00700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7410eb30 0x7f6c7419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:04.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c73e00700 1 -- 172.21.15.97:0/2093512493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7419b270 con 0x7f6c740750a0 2024-03-26T18:58:04.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.683+0000 7f6c79c00700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7410eb30 0x7f6c7419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:04.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.684+0000 7f6c73e00700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c740750a0 0x7f6c741a0e40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000e9f0 tx=0x7f6c6000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:04.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.684+0000 7f6c71600700 1 -- 172.21.15.97:0/2093512493 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000cbd0 con 0x7f6c740750a0 2024-03-26T18:58:04.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.684+0000 7f6c71600700 1 -- 172.21.15.97:0/2093512493 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6000cd70 con 0x7f6c740750a0 2024-03-26T18:58:04.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.684+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093512493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7419b560 con 0x7f6c740750a0 2024-03-26T18:58:04.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.684+0000 7f6c71600700 1 -- 172.21.15.97:0/2093512493 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60018710 con 0x7f6c740750a0 2024-03-26T18:58:04.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.684+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093512493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c741a77d0 con 0x7f6c740750a0 2024-03-26T18:58:04.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.685+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093512493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c74069070 con 0x7f6c740750a0 2024-03-26T18:58:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.687+0000 7f6c71600700 1 -- 172.21.15.97:0/2093512493 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f6c60010430 con 0x7f6c740750a0 2024-03-26T18:58:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.688+0000 7f6c71600700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6c5c0a3030 0x7f6c5c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:04.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.688+0000 7f6c71600700 1 -- 172.21.15.97:0/2093512493 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6c600cf550 con 0x7f6c740750a0 2024-03-26T18:58:04.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.688+0000 7f6c79200700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6c5c0a3030 0x7f6c5c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:04.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.689+0000 7f6c79200700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6c5c0a3030 0x7f6c5c0a54f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6c74111370 tx=0x7f6c68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.692+0000 7f6c71600700 1 -- 172.21.15.97:0/2093512493 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6c6009c350 con 0x7f6c740750a0 2024-03-26T18:58:04.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.995+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093512493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6c7402dc00 con 0x7f6c740750a0 2024-03-26T18:58:04.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.996+0000 7f6c71600700 1 -- 172.21.15.97:0/2093512493 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6c6009bc50 con 0x7f6c740750a0 2024-03-26T18:58:04.989 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:04.989 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-26T18:58:04.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6c5c0a3030 msgr2=0x7f6c5c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:04.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6c5c0a3030 0x7f6c5c0a54f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6c74111370 tx=0x7f6c68009260 comp rx=0 tx=0).stop 2024-03-26T18:58:04.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093512493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c740750a0 msgr2=0x7f6c741a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:04.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c740750a0 0x7f6c741a0e40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000e9f0 tx=0x7f6c6000ed00 comp rx=0 tx=0).stop 2024-03-26T18:58:04.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093512493 shutdown_connections 2024-03-26T18:58:04.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7410eb30 0x7f6c7419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:04.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c74074780 0x7f6c741a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:04.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6c5c0a3030 0x7f6c5c0a54f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:04.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 --2- 172.21.15.97:0/2093512493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c740750a0 0x7f6c741a0e40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:04.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093512493 >> 172.21.15.97:0/2093512493 conn(0x7f6c740fdda0 msgr2=0x7f6c740fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:04.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093512493 shutdown_connections 2024-03-26T18:58:04.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:04.999+0000 7f6c7a600700 1 -- 172.21.15.97:0/2093512493 wait complete. 2024-03-26T18:58:05.988 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-26T18:58:05.988 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:58:06.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.208+0000 7fe879400640 1 -- 172.21.15.97:0/1090932622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8740742a0 msgr2=0x7fe874074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:06.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.208+0000 7fe879400640 1 --2- 172.21.15.97:0/1090932622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8740742a0 0x7fe874074700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009900 tx=0x7fe85c02bcd0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.209+0000 7fe879400640 1 -- 172.21.15.97:0/1090932622 shutdown_connections 2024-03-26T18:58:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.209+0000 7fe879400640 1 --2- 172.21.15.97:0/1090932622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8741095e0 0x7fe87410ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.209+0000 7fe879400640 1 --2- 172.21.15.97:0/1090932622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8740742a0 0x7fe874074700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.209+0000 7fe879400640 1 --2- 172.21.15.97:0/1090932622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe874075880 0x7fe874073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.209+0000 7fe879400640 1 -- 172.21.15.97:0/1090932622 >> 172.21.15.97:0/1090932622 conn(0x7fe8740fcca0 msgr2=0x7fe8740ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.209+0000 7fe879400640 1 -- 172.21.15.97:0/1090932622 shutdown_connections 2024-03-26T18:58:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.209+0000 7fe879400640 1 -- 172.21.15.97:0/1090932622 wait complete. 2024-03-26T18:58:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe879400640 1 Processor -- start 2024-03-26T18:58:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe879400640 1 -- start start 2024-03-26T18:58:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe879400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8740742a0 0x7fe874199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe879400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe874075880 0x7fe87419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe879400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8741095e0 0x7fe8741a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe879400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe87410f5e0 con 0x7fe874075880 2024-03-26T18:58:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe879400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe87410f460 con 0x7fe8741095e0 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe879400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe87410f760 con 0x7fe8740742a0 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe878a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8741095e0 0x7fe8741a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe86b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe874075880 0x7fe87419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe873e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8740742a0 0x7fe874199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe873e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8740742a0 0x7fe874199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56490/0 (socket says 172.21.15.97:56490) 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe873e00640 1 -- 172.21.15.97:0/59536528 learned_addr learned my addr 172.21.15.97:0/59536528 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe878a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8741095e0 0x7fe8741a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54196/0 (socket says 172.21.15.97:54196) 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.210+0000 7fe86b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe874075880 0x7fe87419a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48838/0 (socket says 172.21.15.97:48838) 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.211+0000 7fe86b400640 1 -- 172.21.15.97:0/59536528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8740742a0 msgr2=0x7fe874199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.211+0000 7fe86b400640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8740742a0 0x7fe874199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.211+0000 7fe86b400640 1 -- 172.21.15.97:0/59536528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8741095e0 msgr2=0x7fe8741a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.211+0000 7fe86b400640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8741095e0 0x7fe8741a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.211+0000 7fe86b400640 1 -- 172.21.15.97:0/59536528 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8741a1810 con 0x7fe874075880 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.211+0000 7fe878a00640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8741095e0 0x7fe8741a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.211+0000 7fe873e00640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8740742a0 0x7fe874199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.211+0000 7fe86b400640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe874075880 0x7fe87419a130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe85c036040 tx=0x7fe85c02c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.211+0000 7fe871600640 1 -- 172.21.15.97:0/59536528 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c043020 con 0x7fe874075880 2024-03-26T18:58:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.212+0000 7fe879400640 1 -- 172.21.15.97:0/59536528 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8741a1aa0 con 0x7fe874075880 2024-03-26T18:58:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.212+0000 7fe871600640 1 -- 172.21.15.97:0/59536528 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe85c0054e0 con 0x7fe874075880 2024-03-26T18:58:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.212+0000 7fe871600640 1 -- 172.21.15.97:0/59536528 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c003b50 con 0x7fe874075880 2024-03-26T18:58:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.212+0000 7fe879400640 1 -- 172.21.15.97:0/59536528 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8741a2010 con 0x7fe874075880 2024-03-26T18:58:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.213+0000 7fe879400640 1 -- 172.21.15.97:0/59536528 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8741083a0 con 0x7fe874075880 2024-03-26T18:58:06.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.215+0000 7fe871600640 1 -- 172.21.15.97:0/59536528 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fe85c046020 con 0x7fe874075880 2024-03-26T18:58:06.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.217+0000 7fe871600640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8580a2f20 0x7fe8580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.217+0000 7fe871600640 1 -- 172.21.15.97:0/59536528 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe85c0f1400 con 0x7fe874075880 2024-03-26T18:58:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.217+0000 7fe873e00640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8580a2f20 0x7fe8580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.218+0000 7fe873e00640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8580a2f20 0x7fe8580a53e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe864011550 tx=0x7fe864005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:06.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.222+0000 7fe871600640 1 -- 172.21.15.97:0/59536528 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe85c04d020 con 0x7fe874075880 2024-03-26T18:58:06.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:05 smithi097 ceph-mon[31955]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T18:58:06.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:05 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2093512493' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T18:58:06.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:05 smithi144 ceph-mon[36053]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T18:58:06.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:05 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2093512493' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T18:58:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.433+0000 7fe879400640 1 -- 172.21.15.97:0/59536528 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8740624c0 con 0x7fe874075880 2024-03-26T18:58:06.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.436+0000 7fe871600640 1 -- 172.21.15.97:0/59536528 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22581 (secure 0 0 0) 0x7fe85c0be200 con 0x7fe874075880 2024-03-26T18:58:06.429 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:06.429 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:57:30.061624+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:58:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.438+0000 7fe879400640 1 -- 172.21.15.97:0/59536528 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8580a2f20 msgr2=0x7fe8580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.438+0000 7fe879400640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8580a2f20 0x7fe8580a53e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe864011550 tx=0x7fe864005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.438+0000 7fe879400640 1 -- 172.21.15.97:0/59536528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe874075880 msgr2=0x7fe87419a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.438+0000 7fe879400640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe874075880 0x7fe87419a130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe85c036040 tx=0x7fe85c02c6f0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.438+0000 7fe879400640 1 -- 172.21.15.97:0/59536528 shutdown_connections 2024-03-26T18:58:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.438+0000 7fe879400640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe8580a2f20 0x7fe8580a53e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.438+0000 7fe879400640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8741095e0 0x7fe8741a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.438+0000 7fe879400640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe874075880 0x7fe87419a130 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.438+0000 7fe879400640 1 --2- 172.21.15.97:0/59536528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8740742a0 0x7fe874199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.438+0000 7fe879400640 1 -- 172.21.15.97:0/59536528 >> 172.21.15.97:0/59536528 conn(0x7fe8740fcca0 msgr2=0x7fe874105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.439+0000 7fe879400640 1 -- 172.21.15.97:0/59536528 shutdown_connections 2024-03-26T18:58:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.439+0000 7fe879400640 1 -- 172.21.15.97:0/59536528 wait complete. 2024-03-26T18:58:06.450 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T18:51:11.459500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T18:56:22.069048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T18:56:23.116030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T18:58:06.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:58:06.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:05 smithi105 ceph-mon[36270]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T18:58:06.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:05 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2093512493' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T18:58:06.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.647+0000 7f75cd200640 1 -- 172.21.15.97:0/3416719664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c8104cf0 msgr2=0x7f75c8105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:06.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.647+0000 7f75cd200640 1 --2- 172.21.15.97:0/3416719664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c8104cf0 0x7f75c8105170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009900 tx=0x7f75b402bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:06.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.647+0000 7f75cd200640 1 -- 172.21.15.97:0/3416719664 shutdown_connections 2024-03-26T18:58:06.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.647+0000 7f75cd200640 1 --2- 172.21.15.97:0/3416719664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75c81056b0 0x7f75c810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.647+0000 7f75cd200640 1 --2- 172.21.15.97:0/3416719664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c8104cf0 0x7f75c8105170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.647+0000 7f75cd200640 1 --2- 172.21.15.97:0/3416719664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75c8103b00 0x7f75c8103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.647+0000 7f75cd200640 1 -- 172.21.15.97:0/3416719664 >> 172.21.15.97:0/3416719664 conn(0x7f75c80ff070 msgr2=0x7f75c81014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:06.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.647+0000 7f75cd200640 1 -- 172.21.15.97:0/3416719664 shutdown_connections 2024-03-26T18:58:06.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.647+0000 7f75cd200640 1 -- 172.21.15.97:0/3416719664 wait complete. 2024-03-26T18:58:06.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.649+0000 7f75cd200640 1 Processor -- start 2024-03-26T18:58:06.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.649+0000 7f75cd200640 1 -- start start 2024-03-26T18:58:06.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.649+0000 7f75cd200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75c8103b00 0x7f75c8199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.649+0000 7f75cd200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c8104cf0 0x7f75c819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.649+0000 7f75cd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75c81056b0 0x7f75c81a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.649+0000 7f75cd200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f75c810f7d0 con 0x7f75c81056b0 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.649+0000 7f75cd200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75c810f650 con 0x7f75c8104cf0 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.649+0000 7f75cd200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75c810f950 con 0x7f75c8103b00 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c5e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75c8103b00 0x7f75c8199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c5e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75c8103b00 0x7f75c8199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56508/0 (socket says 172.21.15.97:56508) 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c5400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c8104cf0 0x7f75c819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c5e00640 1 -- 172.21.15.97:0/71283013 learned_addr learned my addr 172.21.15.97:0/71283013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c6800640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75c81056b0 0x7f75c81a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:06.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c5e00640 1 -- 172.21.15.97:0/71283013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c8104cf0 msgr2=0x7f75c819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:06.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c5e00640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c8104cf0 0x7f75c819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c5e00640 1 -- 172.21.15.97:0/71283013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75c81056b0 msgr2=0x7f75c81a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:06.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c5e00640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75c81056b0 0x7f75c81a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c5e00640 1 -- 172.21.15.97:0/71283013 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c81a1a10 con 0x7f75c8103b00 2024-03-26T18:58:06.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75c5e00640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75c8103b00 0x7f75c8199df0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f75b800e9c0 tx=0x7f75b800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:06.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75bea00640 1 -- 172.21.15.97:0/71283013 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b800cd80 con 0x7f75c8103b00 2024-03-26T18:58:06.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75bea00640 1 -- 172.21.15.97:0/71283013 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b80035f0 con 0x7f75c8103b00 2024-03-26T18:58:06.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75cd200640 1 -- 172.21.15.97:0/71283013 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c81a1d00 con 0x7f75c8103b00 2024-03-26T18:58:06.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.650+0000 7f75bea00640 1 -- 172.21.15.97:0/71283013 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8010690 con 0x7f75c8103b00 2024-03-26T18:58:06.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.651+0000 7f75cd200640 1 -- 172.21.15.97:0/71283013 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c81a21c0 con 0x7f75c8103b00 2024-03-26T18:58:06.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.652+0000 7f75cd200640 1 -- 172.21.15.97:0/71283013 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c810adc0 con 0x7f75c8103b00 2024-03-26T18:58:06.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.653+0000 7f75bea00640 1 -- 172.21.15.97:0/71283013 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f75b80037a0 con 0x7f75c8103b00 2024-03-26T18:58:06.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.654+0000 7f75bea00640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75ac0a2fa0 0x7f75ac0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:06.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.655+0000 7f75bea00640 1 -- 172.21.15.97:0/71283013 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f75b80cf780 con 0x7f75c8103b00 2024-03-26T18:58:06.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.655+0000 7f75c5400640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75ac0a2fa0 0x7f75ac0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:06.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.655+0000 7f75c5400640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75ac0a2fa0 0x7f75ac0a5460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f75b402c240 tx=0x7f75b402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:06.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.658+0000 7f75bea00640 1 -- 172.21.15.97:0/71283013 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f75b809c580 con 0x7f75c8103b00 2024-03-26T18:58:06.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.858+0000 7f75cd200640 1 -- 172.21.15.97:0/71283013 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75c802d240 con 0x7f75c8103b00 2024-03-26T18:58:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.859+0000 7f75bea00640 1 -- 172.21.15.97:0/71283013 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f75c802d240 con 0x7f75c8103b00 2024-03-26T18:58:06.852 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-26T18:58:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 -- 172.21.15.97:0/71283013 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75ac0a2fa0 msgr2=0x7f75ac0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75ac0a2fa0 0x7f75ac0a5460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f75b402c240 tx=0x7f75b402df90 comp rx=0 tx=0).stop 2024-03-26T18:58:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 -- 172.21.15.97:0/71283013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75c8103b00 msgr2=0x7f75c8199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75c8103b00 0x7f75c8199df0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f75b800e9c0 tx=0x7f75b800ee80 comp rx=0 tx=0).stop 2024-03-26T18:58:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 -- 172.21.15.97:0/71283013 shutdown_connections 2024-03-26T18:58:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f75ac0a2fa0 0x7f75ac0a5460 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75c81056b0 0x7f75c81a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75c8104cf0 0x7f75c819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 --2- 172.21.15.97:0/71283013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75c8103b00 0x7f75c8199df0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 -- 172.21.15.97:0/71283013 >> 172.21.15.97:0/71283013 conn(0x7f75c80ff070 msgr2=0x7f75c8107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 -- 172.21.15.97:0/71283013 shutdown_connections 2024-03-26T18:58:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:06.862+0000 7f75cd200640 1 -- 172.21.15.97:0/71283013 wait complete. 2024-03-26T18:58:06.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:58:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.143+0000 7f36a3000640 1 -- 172.21.15.97:0/1433320401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c100050 msgr2=0x7f369c100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.143+0000 7f36a3000640 1 --2- 172.21.15.97:0/1433320401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c100050 0x7f369c100470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3690009900 tx=0x7f369002bc80 comp rx=0 tx=0).stop 2024-03-26T18:58:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.143+0000 7f36a3000640 1 -- 172.21.15.97:0/1433320401 shutdown_connections 2024-03-26T18:58:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.143+0000 7f36a3000640 1 --2- 172.21.15.97:0/1433320401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c101350 0x7f369c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.143+0000 7f36a3000640 1 --2- 172.21.15.97:0/1433320401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f369c1009b0 0x7f369c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.143+0000 7f36a3000640 1 --2- 172.21.15.97:0/1433320401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c100050 0x7f369c100470 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.143+0000 7f36a3000640 1 -- 172.21.15.97:0/1433320401 >> 172.21.15.97:0/1433320401 conn(0x7f369c0fbc30 msgr2=0x7f369c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:07.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.143+0000 7f36a3000640 1 -- 172.21.15.97:0/1433320401 shutdown_connections 2024-03-26T18:58:07.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.143+0000 7f36a3000640 1 -- 172.21.15.97:0/1433320401 wait complete. 2024-03-26T18:58:07.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.145+0000 7f36a3000640 1 Processor -- start 2024-03-26T18:58:07.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.145+0000 7f36a3000640 1 -- start start 2024-03-26T18:58:07.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.145+0000 7f36a3000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f369c100050 0x7f369c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.145+0000 7f36a3000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1009b0 0x7f369c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.145+0000 7f36a3000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c101350 0x7f369c1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.145+0000 7f36a3000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f369c10f640 con 0x7f369c100050 2024-03-26T18:58:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.145+0000 7f36a3000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f369c10f4c0 con 0x7f369c101350 2024-03-26T18:58:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.145+0000 7f36a3000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f369c10f7c0 con 0x7f369c1009b0 2024-03-26T18:58:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1009b0 0x7f369c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a2600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c101350 0x7f369c1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a1c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f369c100050 0x7f369c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1009b0 0x7f369c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56534/0 (socket says 172.21.15.97:56534) 2024-03-26T18:58:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a2600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c101350 0x7f369c1a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54232/0 (socket says 172.21.15.97:54232) 2024-03-26T18:58:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a1200640 1 -- 172.21.15.97:0/251431622 learned_addr learned my addr 172.21.15.97:0/251431622 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a1200640 1 -- 172.21.15.97:0/251431622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c101350 msgr2=0x7f369c1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a1200640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c101350 0x7f369c1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a1200640 1 -- 172.21.15.97:0/251431622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f369c100050 msgr2=0x7f369c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a1200640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f369c100050 0x7f369c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a1200640 1 -- 172.21.15.97:0/251431622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c1a1980 con 0x7f369c1009b0 2024-03-26T18:58:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f36a1200640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1009b0 0x7f369c19a2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f368800e9f0 tx=0x7f368800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:07.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.146+0000 7f369a000640 1 -- 172.21.15.97:0/251431622 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368800ccc0 con 0x7f369c1009b0 2024-03-26T18:58:07.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.147+0000 7f369a000640 1 -- 172.21.15.97:0/251431622 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3688010910 con 0x7f369c1009b0 2024-03-26T18:58:07.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.147+0000 7f36a3000640 1 -- 172.21.15.97:0/251431622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c1a1c70 con 0x7f369c1009b0 2024-03-26T18:58:07.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.147+0000 7f369a000640 1 -- 172.21.15.97:0/251431622 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36880188a0 con 0x7f369c1009b0 2024-03-26T18:58:07.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.147+0000 7f36a3000640 1 -- 172.21.15.97:0/251431622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c1a21c0 con 0x7f369c1009b0 2024-03-26T18:58:07.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.148+0000 7f36a3000640 1 -- 172.21.15.97:0/251431622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369c0685d0 con 0x7f369c1009b0 2024-03-26T18:58:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.151+0000 7f369a000640 1 -- 172.21.15.97:0/251431622 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f3688010ac0 con 0x7f369c1009b0 2024-03-26T18:58:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.152+0000 7f369a000640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36840a2f20 0x7f36840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.152+0000 7f369a000640 1 -- 172.21.15.97:0/251431622 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f36880cf7c0 con 0x7f369c1009b0 2024-03-26T18:58:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.152+0000 7f36a1c00640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36840a2f20 0x7f36840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.153+0000 7f36a1c00640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36840a2f20 0x7f36840a53e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3690000c00 tx=0x7f369002c870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:07.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.156+0000 7f369a000640 1 -- 172.21.15.97:0/251431622 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f368809c600 con 0x7f369c1009b0 2024-03-26T18:58:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.359+0000 7f36a3000640 1 -- 172.21.15.97:0/251431622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f369c0624c0 con 0x7f369c1009b0 2024-03-26T18:58:07.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.359+0000 7f369a000640 1 -- 172.21.15.97:0/251431622 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f36880bc0f0 con 0x7f369c1009b0 2024-03-26T18:58:07.352 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-26T18:58:07.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:07 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/59536528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:07.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:07 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/71283013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 -- 172.21.15.97:0/251431622 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36840a2f20 msgr2=0x7f36840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36840a2f20 0x7f36840a53e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3690000c00 tx=0x7f369002c870 comp rx=0 tx=0).stop 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 -- 172.21.15.97:0/251431622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1009b0 msgr2=0x7f369c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1009b0 0x7f369c19a2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f368800e9f0 tx=0x7f368800edf0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 -- 172.21.15.97:0/251431622 shutdown_connections 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f36840a2f20 0x7f36840a53e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c101350 0x7f369c1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1009b0 0x7f369c19a2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 --2- 172.21.15.97:0/251431622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f369c100050 0x7f369c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 -- 172.21.15.97:0/251431622 >> 172.21.15.97:0/251431622 conn(0x7f369c0fbc30 msgr2=0x7f369c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 -- 172.21.15.97:0/251431622 shutdown_connections 2024-03-26T18:58:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.362+0000 7f36a3000640 1 -- 172.21.15.97:0/251431622 wait complete. 2024-03-26T18:58:07.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:58:07.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:07 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/59536528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:07.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:07 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/71283013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:07.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:07 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/59536528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:07.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:07 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/71283013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:07.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.598+0000 7fdbfaa00640 1 -- 172.21.15.97:0/4292990285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf41034a0 msgr2=0x7fdbf4109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:07.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.598+0000 7fdbfaa00640 1 --2- 172.21.15.97:0/4292990285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf41034a0 0x7fdbf4109d30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009ca0 tx=0x7fdbec02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.598+0000 7fdbfaa00640 1 -- 172.21.15.97:0/4292990285 shutdown_connections 2024-03-26T18:58:07.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.598+0000 7fdbfaa00640 1 --2- 172.21.15.97:0/4292990285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf41034a0 0x7fdbf4109d30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.598+0000 7fdbfaa00640 1 --2- 172.21.15.97:0/4292990285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf4102ae0 0x7fdbf4102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.598+0000 7fdbfaa00640 1 --2- 172.21.15.97:0/4292990285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf41018f0 0x7fdbf4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.598+0000 7fdbfaa00640 1 -- 172.21.15.97:0/4292990285 >> 172.21.15.97:0/4292990285 conn(0x7fdbf40fce60 msgr2=0x7fdbf40ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:07.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.598+0000 7fdbfaa00640 1 -- 172.21.15.97:0/4292990285 shutdown_connections 2024-03-26T18:58:07.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.598+0000 7fdbfaa00640 1 -- 172.21.15.97:0/4292990285 wait complete. 2024-03-26T18:58:07.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.599+0000 7fdbfaa00640 1 Processor -- start 2024-03-26T18:58:07.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.599+0000 7fdbfaa00640 1 -- start start 2024-03-26T18:58:07.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.599+0000 7fdbfaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf41018f0 0x7fdbf4199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:07.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.599+0000 7fdbfaa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf4102ae0 0x7fdbf419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:07.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.599+0000 7fdbfaa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf41034a0 0x7fdbf41a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:07.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.599+0000 7fdbfaa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410d550 con 0x7fdbf41018f0 2024-03-26T18:58:07.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.599+0000 7fdbfaa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410d3d0 con 0x7fdbf41034a0 2024-03-26T18:58:07.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbfaa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410d6d0 con 0x7fdbf4102ae0 2024-03-26T18:58:07.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbebe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf4102ae0 0x7fdbf419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:07.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf41018f0 0x7fdbf4199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:07.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbf8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf41034a0 0x7fdbf41a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:07.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbebe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf4102ae0 0x7fdbf419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56542/0 (socket says 172.21.15.97:56542) 2024-03-26T18:58:07.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbebe00640 1 -- 172.21.15.97:0/2551930553 learned_addr learned my addr 172.21.15.97:0/2551930553 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:07.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbebe00640 1 -- 172.21.15.97:0/2551930553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf41034a0 msgr2=0x7fdbf41a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:07.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbebe00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf41034a0 0x7fdbf41a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbebe00640 1 -- 172.21.15.97:0/2551930553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf41018f0 msgr2=0x7fdbf4199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbebe00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf41018f0 0x7fdbf4199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbebe00640 1 -- 172.21.15.97:0/2551930553 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf41a19c0 con 0x7fdbf4102ae0 2024-03-26T18:58:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.600+0000 7fdbebe00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf4102ae0 0x7fdbf419a2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00d950 tx=0x7fdbdc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.601+0000 7fdbf1600640 1 -- 172.21.15.97:0/2551930553 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc017030 con 0x7fdbf4102ae0 2024-03-26T18:58:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.601+0000 7fdbf1600640 1 -- 172.21.15.97:0/2551930553 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbdc004020 con 0x7fdbf4102ae0 2024-03-26T18:58:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.601+0000 7fdbfaa00640 1 -- 172.21.15.97:0/2551930553 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf41a1cb0 con 0x7fdbf4102ae0 2024-03-26T18:58:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.601+0000 7fdbf1600640 1 -- 172.21.15.97:0/2551930553 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0047a0 con 0x7fdbf4102ae0 2024-03-26T18:58:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.601+0000 7fdbfaa00640 1 -- 172.21.15.97:0/2551930553 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf41a21a0 con 0x7fdbf4102ae0 2024-03-26T18:58:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.602+0000 7fdbfaa00640 1 -- 172.21.15.97:0/2551930553 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf41a2460 con 0x7fdbf4102ae0 2024-03-26T18:58:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.604+0000 7fdbf1600640 1 -- 172.21.15.97:0/2551930553 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fdbdc0041d0 con 0x7fdbf4102ae0 2024-03-26T18:58:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.605+0000 7fdbf1600640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdbd80a2fa0 0x7fdbd80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.605+0000 7fdbf1600640 1 -- 172.21.15.97:0/2551930553 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fdbdc0cecd0 con 0x7fdbf4102ae0 2024-03-26T18:58:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.605+0000 7fdbf3e00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdbd80a2fa0 0x7fdbd80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.606+0000 7fdbf3e00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdbd80a2fa0 0x7fdbd80a5460 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdbf419b280 tx=0x7fdbe400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.609+0000 7fdbf1600640 1 -- 172.21.15.97:0/2551930553 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdbdc09bad0 con 0x7fdbf4102ae0 2024-03-26T18:58:07.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.840+0000 7fdbfaa00640 1 -- 172.21.15.97:0/2551930553 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdbf41a2d10 con 0x7fdbf4102ae0 2024-03-26T18:58:07.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.841+0000 7fdbf1600640 1 -- 172.21.15.97:0/2551930553 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fdbdc09b3d0 con 0x7fdbf4102ae0 2024-03-26T18:58:07.834 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-26T18:58:07.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 -- 172.21.15.97:0/2551930553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdbd80a2fa0 msgr2=0x7fdbd80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdbd80a2fa0 0x7fdbd80a5460 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdbf419b280 tx=0x7fdbe400b410 comp rx=0 tx=0).stop 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 -- 172.21.15.97:0/2551930553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf4102ae0 msgr2=0x7fdbf419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf4102ae0 0x7fdbf419a2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00d950 tx=0x7fdbdc00de10 comp rx=0 tx=0).stop 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 -- 172.21.15.97:0/2551930553 shutdown_connections 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdbd80a2fa0 0x7fdbd80a5460 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf41034a0 0x7fdbf41a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf4102ae0 0x7fdbf419a2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 --2- 172.21.15.97:0/2551930553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf41018f0 0x7fdbf4199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 -- 172.21.15.97:0/2551930553 >> 172.21.15.97:0/2551930553 conn(0x7fdbf40fce60 msgr2=0x7fdbf4073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 -- 172.21.15.97:0/2551930553 shutdown_connections 2024-03-26T18:58:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:07.844+0000 7fdbfaa00640 1 -- 172.21.15.97:0/2551930553 wait complete. 2024-03-26T18:58:07.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:58:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.069+0000 7fe2bb200640 1 -- 172.21.15.97:0/417965446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b4102190 msgr2=0x7fe2b4104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.069+0000 7fe2bb200640 1 --2- 172.21.15.97:0/417965446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b4102190 0x7fe2b4104580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009900 tx=0x7fe2a802bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.069+0000 7fe2bb200640 1 -- 172.21.15.97:0/417965446 shutdown_connections 2024-03-26T18:58:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.069+0000 7fe2bb200640 1 --2- 172.21.15.97:0/417965446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b41073f0 0x7fe2b4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.069+0000 7fe2bb200640 1 --2- 172.21.15.97:0/417965446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b4104ac0 0x7fe2b4106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.069+0000 7fe2bb200640 1 --2- 172.21.15.97:0/417965446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b4102190 0x7fe2b4104580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.069+0000 7fe2bb200640 1 -- 172.21.15.97:0/417965446 >> 172.21.15.97:0/417965446 conn(0x7fe2b40fbba0 msgr2=0x7fe2b40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.069+0000 7fe2bb200640 1 -- 172.21.15.97:0/417965446 shutdown_connections 2024-03-26T18:58:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.069+0000 7fe2bb200640 1 -- 172.21.15.97:0/417965446 wait complete. 2024-03-26T18:58:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.070+0000 7fe2bb200640 1 Processor -- start 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.070+0000 7fe2bb200640 1 -- start start 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2bb200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b4102190 0x7fe2b4197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2bb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b4104ac0 0x7fe2b4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2bb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b41073f0 0x7fe2b419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2bb200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410d3e0 con 0x7fe2b4104ac0 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2bb200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410d260 con 0x7fe2b4102190 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2bb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410d560 con 0x7fe2b41073f0 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2b9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b4102190 0x7fe2b4197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2b9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b4102190 0x7fe2b4197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54282/0 (socket says 172.21.15.97:54282) 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2b9e00640 1 -- 172.21.15.97:0/4152440361 learned_addr learned my addr 172.21.15.97:0/4152440361 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2ba800640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b41073f0 0x7fe2b419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2b9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b4104ac0 0x7fe2b4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2b9e00640 1 -- 172.21.15.97:0/4152440361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b41073f0 msgr2=0x7fe2b419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2b9e00640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b41073f0 0x7fe2b419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2b9e00640 1 -- 172.21.15.97:0/4152440361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b4104ac0 msgr2=0x7fe2b4197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2b9e00640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b4104ac0 0x7fe2b4197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.071+0000 7fe2b9e00640 1 -- 172.21.15.97:0/4152440361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b419f670 con 0x7fe2b4102190 2024-03-26T18:58:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.072+0000 7fe2b9e00640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b4102190 0x7fe2b4197a50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8000c00 tx=0x7fe2a802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.072+0000 7fe2aea00640 1 -- 172.21.15.97:0/4152440361 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8043020 con 0x7fe2b4102190 2024-03-26T18:58:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.072+0000 7fe2aea00640 1 -- 172.21.15.97:0/4152440361 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2a80054e0 con 0x7fe2b4102190 2024-03-26T18:58:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.072+0000 7fe2bb200640 1 -- 172.21.15.97:0/4152440361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b40fff80 con 0x7fe2b4102190 2024-03-26T18:58:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.072+0000 7fe2aea00640 1 -- 172.21.15.97:0/4152440361 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8003c40 con 0x7fe2b4102190 2024-03-26T18:58:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.072+0000 7fe2bb200640 1 -- 172.21.15.97:0/4152440361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41004d0 con 0x7fe2b4102190 2024-03-26T18:58:08.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.073+0000 7fe2bb200640 1 -- 172.21.15.97:0/4152440361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b40fd690 con 0x7fe2b4102190 2024-03-26T18:58:08.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.076+0000 7fe2aea00640 1 -- 172.21.15.97:0/4152440361 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fe2a8046020 con 0x7fe2b4102190 2024-03-26T18:58:08.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.077+0000 7fe2aea00640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe29c0a2f20 0x7fe29c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.077+0000 7fe2aea00640 1 -- 172.21.15.97:0/4152440361 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe2a80f13e0 con 0x7fe2b4102190 2024-03-26T18:58:08.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.077+0000 7fe2b9400640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe29c0a2f20 0x7fe29c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.078+0000 7fe2b9400640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe29c0a2f20 0x7fe29c0a53e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4198f60 tx=0x7fe2a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:08.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.081+0000 7fe2aea00640 1 -- 172.21.15.97:0/4152440361 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe2a804d020 con 0x7fe2b4102190 2024-03-26T18:58:08.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.331+0000 7fe2bb200640 1 -- 172.21.15.97:0/4152440361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2b402d240 con 0x7fe2b4102190 2024-03-26T18:58:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.332+0000 7fe2aea00640 1 -- 172.21.15.97:0/4152440361 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11935 (secure 0 0 0) 0x7fe2a80be240 con 0x7fe2b4102190 2024-03-26T18:58:08.325 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:08.325 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T18:56:23.642405+0000","modified":"2024-03-26T18:56:31.851339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24535,"mds_3":24494,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2391073232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2391073232},{"type":"v1","addr":"172.21.15.105:6839","nonce":2391073232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24535,24494,24481]},"id":1}]} 2024-03-26T18:58:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-03-26T18:58:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.335+0000 7fe2bb200640 1 -- 172.21.15.97:0/4152440361 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe29c0a2f20 msgr2=0x7fe29c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.335+0000 7fe2bb200640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe29c0a2f20 0x7fe29c0a53e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4198f60 tx=0x7fe2a4005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.335+0000 7fe2bb200640 1 -- 172.21.15.97:0/4152440361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b4102190 msgr2=0x7fe2b4197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.335+0000 7fe2bb200640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b4102190 0x7fe2b4197a50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8000c00 tx=0x7fe2a802c650 comp rx=0 tx=0).stop 2024-03-26T18:58:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.335+0000 7fe2bb200640 1 -- 172.21.15.97:0/4152440361 shutdown_connections 2024-03-26T18:58:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.335+0000 7fe2bb200640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe29c0a2f20 0x7fe29c0a53e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.335+0000 7fe2bb200640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b41073f0 0x7fe2b419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.335+0000 7fe2bb200640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b4104ac0 0x7fe2b4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.335+0000 7fe2bb200640 1 --2- 172.21.15.97:0/4152440361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b4102190 0x7fe2b4197a50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.335+0000 7fe2bb200640 1 -- 172.21.15.97:0/4152440361 >> 172.21.15.97:0/4152440361 conn(0x7fe2b40fbba0 msgr2=0x7fe2b4102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.336+0000 7fe2bb200640 1 -- 172.21.15.97:0/4152440361 shutdown_connections 2024-03-26T18:58:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.336+0000 7fe2bb200640 1 -- 172.21.15.97:0/4152440361 wait complete. 2024-03-26T18:58:08.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:08 smithi097 ceph-mon[31955]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T18:58:08.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:08 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/251431622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:08.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:08 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2551930553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:08.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:58:08.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:08 smithi144 ceph-mon[36053]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T18:58:08.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:08 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/251431622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:08.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:08 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2551930553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:08.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:08 smithi105 ceph-mon[36270]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T18:58:08.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:08 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/251431622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:08.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:08 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2551930553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.544+0000 7f14c9800640 1 -- 172.21.15.97:0/1914939700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c4104cf0 msgr2=0x7f14c4105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.544+0000 7f14c9800640 1 --2- 172.21.15.97:0/1914939700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c4104cf0 0x7f14c4105170 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009900 tx=0x7f14b002bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.544+0000 7f14c9800640 1 -- 172.21.15.97:0/1914939700 shutdown_connections 2024-03-26T18:58:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.544+0000 7f14c9800640 1 --2- 172.21.15.97:0/1914939700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14c41056b0 0x7f14c410bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.544+0000 7f14c9800640 1 --2- 172.21.15.97:0/1914939700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c4104cf0 0x7f14c4105170 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.544+0000 7f14c9800640 1 --2- 172.21.15.97:0/1914939700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c4103b00 0x7f14c4103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.544+0000 7f14c9800640 1 -- 172.21.15.97:0/1914939700 >> 172.21.15.97:0/1914939700 conn(0x7f14c40ff070 msgr2=0x7f14c41014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.544+0000 7f14c9800640 1 -- 172.21.15.97:0/1914939700 shutdown_connections 2024-03-26T18:58:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.544+0000 7f14c9800640 1 -- 172.21.15.97:0/1914939700 wait complete. 2024-03-26T18:58:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.545+0000 7f14c9800640 1 Processor -- start 2024-03-26T18:58:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.545+0000 7f14c9800640 1 -- start start 2024-03-26T18:58:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c4103b00 0x7f14c4199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c9800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14c4104cf0 0x7f14c419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c9800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c41056b0 0x7f14c41a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c9800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f14c410f7d0 con 0x7f14c4103b00 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c9800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f14c410f650 con 0x7f14c4104cf0 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c9800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14c410f950 con 0x7f14c41056b0 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c2000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14c4104cf0 0x7f14c419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c4103b00 0x7f14c4199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c41056b0 0x7f14c41a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c2000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14c4104cf0 0x7f14c419a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54304/0 (socket says 172.21.15.97:54304) 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.546+0000 7f14c2000640 1 -- 172.21.15.97:0/3679333166 learned_addr learned my addr 172.21.15.97:0/3679333166 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14c2000640 1 -- 172.21.15.97:0/3679333166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c41056b0 msgr2=0x7f14c41a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14c2000640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c41056b0 0x7f14c41a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14c2000640 1 -- 172.21.15.97:0/3679333166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c4103b00 msgr2=0x7f14c4199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14c2000640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c4103b00 0x7f14c4199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14c2000640 1 -- 172.21.15.97:0/3679333166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c41a1a00 con 0x7f14c4104cf0 2024-03-26T18:58:08.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14c3400640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c41056b0 0x7f14c41a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14c2000640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14c4104cf0 0x7f14c419a320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14b0000c00 tx=0x7f14b002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14bb400640 1 -- 172.21.15.97:0/3679333166 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0043020 con 0x7f14c4104cf0 2024-03-26T18:58:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14bb400640 1 -- 172.21.15.97:0/3679333166 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14b00054e0 con 0x7f14c4104cf0 2024-03-26T18:58:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14bb400640 1 -- 172.21.15.97:0/3679333166 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0003c40 con 0x7f14c4104cf0 2024-03-26T18:58:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14c9800640 1 -- 172.21.15.97:0/3679333166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c41a1c90 con 0x7f14c4104cf0 2024-03-26T18:58:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.547+0000 7f14c9800640 1 -- 172.21.15.97:0/3679333166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c41a21d0 con 0x7f14c4104cf0 2024-03-26T18:58:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.548+0000 7f14c9800640 1 -- 172.21.15.97:0/3679333166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c410adc0 con 0x7f14c4104cf0 2024-03-26T18:58:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.552+0000 7f14bb400640 1 -- 172.21.15.97:0/3679333166 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f14b0046020 con 0x7f14c4104cf0 2024-03-26T18:58:08.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.553+0000 7f14bb400640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f14a80a2fa0 0x7f14a80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.553+0000 7f14bb400640 1 -- 172.21.15.97:0/3679333166 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f14b00f1400 con 0x7f14c4104cf0 2024-03-26T18:58:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.553+0000 7f14c2a00640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f14a80a2fa0 0x7f14a80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.554+0000 7f14c2a00640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f14a80a2fa0 0x7f14a80a5460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f14c4104b50 tx=0x7f14b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:08.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.557+0000 7f14bb400640 1 -- 172.21.15.97:0/3679333166 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f14b004d020 con 0x7f14c4104cf0 2024-03-26T18:58:08.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.757+0000 7f14c9800640 1 -- 172.21.15.97:0/3679333166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14c4062560 con 0x7f14c4104cf0 2024-03-26T18:58:08.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.759+0000 7f14bb400640 1 -- 172.21.15.97:0/3679333166 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22581 (secure 0 0 0) 0x7f14b00be200 con 0x7f14c4104cf0 2024-03-26T18:58:08.752 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:08.752 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:57:30.061624+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:58:08.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.761+0000 7f14c9800640 1 -- 172.21.15.97:0/3679333166 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f14a80a2fa0 msgr2=0x7f14a80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.761+0000 7f14c9800640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f14a80a2fa0 0x7f14a80a5460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f14c4104b50 tx=0x7f14b4005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:08.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.761+0000 7f14c9800640 1 -- 172.21.15.97:0/3679333166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14c4104cf0 msgr2=0x7f14c419a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.761+0000 7f14c9800640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14c4104cf0 0x7f14c419a320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14b0000c00 tx=0x7f14b002c650 comp rx=0 tx=0).stop 2024-03-26T18:58:08.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.761+0000 7f14c9800640 1 -- 172.21.15.97:0/3679333166 shutdown_connections 2024-03-26T18:58:08.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.761+0000 7f14c9800640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f14a80a2fa0 0x7f14a80a5460 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.761+0000 7f14c9800640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c41056b0 0x7f14c41a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.761+0000 7f14c9800640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14c4104cf0 0x7f14c419a320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.761+0000 7f14c9800640 1 --2- 172.21.15.97:0/3679333166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c4103b00 0x7f14c4199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.761+0000 7f14c9800640 1 -- 172.21.15.97:0/3679333166 >> 172.21.15.97:0/3679333166 conn(0x7f14c40ff070 msgr2=0x7f14c4107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:08.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.762+0000 7f14c9800640 1 -- 172.21.15.97:0/3679333166 shutdown_connections 2024-03-26T18:58:08.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.762+0000 7f14c9800640 1 -- 172.21.15.97:0/3679333166 wait complete. 2024-03-26T18:58:08.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T18:51:11.459500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T18:56:22.069048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T18:56:23.116030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T18:58:08.776 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:58:08.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.998+0000 7fe19ea00640 1 -- 172.21.15.97:0/2081656039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1981018d0 msgr2=0x7fe198101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.998+0000 7fe19ea00640 1 --2- 172.21.15.97:0/2081656039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1981018d0 0x7fe198101cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe188009900 tx=0x7fe18802bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:08.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.998+0000 7fe19ea00640 1 -- 172.21.15.97:0/2081656039 shutdown_connections 2024-03-26T18:58:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.998+0000 7fe19ea00640 1 --2- 172.21.15.97:0/2081656039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe198103480 0x7fe198109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.998+0000 7fe19ea00640 1 --2- 172.21.15.97:0/2081656039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe198102ac0 0x7fe198102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.998+0000 7fe19ea00640 1 --2- 172.21.15.97:0/2081656039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1981018d0 0x7fe198101cf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.998+0000 7fe19ea00640 1 -- 172.21.15.97:0/2081656039 >> 172.21.15.97:0/2081656039 conn(0x7fe1980fce60 msgr2=0x7fe1980ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.999+0000 7fe19ea00640 1 -- 172.21.15.97:0/2081656039 shutdown_connections 2024-03-26T18:58:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:08.999+0000 7fe19ea00640 1 -- 172.21.15.97:0/2081656039 wait complete. 2024-03-26T18:58:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.000+0000 7fe19ea00640 1 Processor -- start 2024-03-26T18:58:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.000+0000 7fe19ea00640 1 -- start start 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe19ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1981018d0 0x7fe198197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe19ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe198102ac0 0x7fe1981981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe19ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe198103480 0x7fe19819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe19ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe19810d590 con 0x7fe198102ac0 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe19ea00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe19810d410 con 0x7fe1981018d0 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe19ea00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe19810d710 con 0x7fe198103480 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe197e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1981018d0 0x7fe198197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe197e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1981018d0 0x7fe198197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54332/0 (socket says 172.21.15.97:54332) 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe197e00640 1 -- 172.21.15.97:0/3251719147 learned_addr learned my addr 172.21.15.97:0/3251719147 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe197400640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe198102ac0 0x7fe1981981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe19ca00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe198103480 0x7fe19819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe197e00640 1 -- 172.21.15.97:0/3251719147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe198103480 msgr2=0x7fe19819f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe197e00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe198103480 0x7fe19819f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe197e00640 1 -- 172.21.15.97:0/3251719147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe198102ac0 msgr2=0x7fe1981981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:08.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe197e00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe198102ac0 0x7fe1981981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:08.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.001+0000 7fe197e00640 1 -- 172.21.15.97:0/3251719147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19819f890 con 0x7fe1981018d0 2024-03-26T18:58:08.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.002+0000 7fe197e00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1981018d0 0x7fe198197c80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe188000c00 tx=0x7fe18802c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:08.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.002+0000 7fe194c00640 1 -- 172.21.15.97:0/3251719147 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188043020 con 0x7fe1981018d0 2024-03-26T18:58:08.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.002+0000 7fe194c00640 1 -- 172.21.15.97:0/3251719147 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1880054e0 con 0x7fe1981018d0 2024-03-26T18:58:08.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.002+0000 7fe19ea00640 1 -- 172.21.15.97:0/3251719147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19819fb20 con 0x7fe1981018d0 2024-03-26T18:58:08.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.002+0000 7fe194c00640 1 -- 172.21.15.97:0/3251719147 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188003ce0 con 0x7fe1981018d0 2024-03-26T18:58:08.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.002+0000 7fe19ea00640 1 -- 172.21.15.97:0/3251719147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1981a0040 con 0x7fe1981018d0 2024-03-26T18:58:08.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.004+0000 7fe19ea00640 1 -- 172.21.15.97:0/3251719147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe198068b00 con 0x7fe1981018d0 2024-03-26T18:58:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.005+0000 7fe194c00640 1 -- 172.21.15.97:0/3251719147 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fe188046020 con 0x7fe1981018d0 2024-03-26T18:58:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.007+0000 7fe194c00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe17c0a2f20 0x7fe17c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.008+0000 7fe194c00640 1 -- 172.21.15.97:0/3251719147 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe1880f1560 con 0x7fe1981018d0 2024-03-26T18:58:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.008+0000 7fe197400640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe17c0a2f20 0x7fe17c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:09.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.008+0000 7fe197400640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe17c0a2f20 0x7fe17c0a53e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe184009bd0 tx=0x7fe184005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:09.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.014+0000 7fe194c00640 1 -- 172.21.15.97:0/3251719147 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe18804d020 con 0x7fe1981018d0 2024-03-26T18:58:09.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:09 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/4152440361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:09.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:09 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3679333166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.269+0000 7fe19ea00640 1 -- 172.21.15.97:0/3251719147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe1980624c0 con 0x7fe1981018d0 2024-03-26T18:58:09.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.269+0000 7fe194c00640 1 -- 172.21.15.97:0/3251719147 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fe1880be360 con 0x7fe1981018d0 2024-03-26T18:58:09.262 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 -- 172.21.15.97:0/3251719147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe17c0a2f20 msgr2=0x7fe17c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe17c0a2f20 0x7fe17c0a53e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe184009bd0 tx=0x7fe184005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 -- 172.21.15.97:0/3251719147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1981018d0 msgr2=0x7fe198197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1981018d0 0x7fe198197c80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe188000c00 tx=0x7fe18802c6c0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 -- 172.21.15.97:0/3251719147 shutdown_connections 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe17c0a2f20 0x7fe17c0a53e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe198103480 0x7fe19819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe198102ac0 0x7fe1981981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 --2- 172.21.15.97:0/3251719147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1981018d0 0x7fe198197c80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 -- 172.21.15.97:0/3251719147 >> 172.21.15.97:0/3251719147 conn(0x7fe1980fce60 msgr2=0x7fe198105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 -- 172.21.15.97:0/3251719147 shutdown_connections 2024-03-26T18:58:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.272+0000 7fe19ea00640 1 -- 172.21.15.97:0/3251719147 wait complete. 2024-03-26T18:58:09.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:58:09.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:09 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/4152440361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:09.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:09 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3679333166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:09.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:09 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/4152440361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:09.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:09 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3679333166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:09.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.520+0000 7f10db200640 1 -- 172.21.15.97:0/1879222855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10dc10dd90 msgr2=0x7f10dc110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:09.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.520+0000 7f10db200640 1 --2- 172.21.15.97:0/1879222855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10dc10dd90 0x7f10dc110180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009ca0 tx=0x7f10d402c100 comp rx=0 tx=0).stop 2024-03-26T18:58:09.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.520+0000 7f10db200640 1 -- 172.21.15.97:0/1879222855 shutdown_connections 2024-03-26T18:58:09.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.520+0000 7f10db200640 1 --2- 172.21.15.97:0/1879222855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10dc10dd90 0x7f10dc110180 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.520+0000 7f10db200640 1 --2- 172.21.15.97:0/1879222855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10dc074320 0x7f10dc0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.520+0000 7f10db200640 1 --2- 172.21.15.97:0/1879222855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10dc075900 0x7f10dc073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.520+0000 7f10db200640 1 -- 172.21.15.97:0/1879222855 >> 172.21.15.97:0/1879222855 conn(0x7f10dc0fcd30 msgr2=0x7f10dc0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.520+0000 7f10db200640 1 -- 172.21.15.97:0/1879222855 shutdown_connections 2024-03-26T18:58:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.520+0000 7f10db200640 1 -- 172.21.15.97:0/1879222855 wait complete. 2024-03-26T18:58:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.522+0000 7f10db200640 1 Processor -- start 2024-03-26T18:58:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.522+0000 7f10db200640 1 -- start start 2024-03-26T18:58:09.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.522+0000 7f10db200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10dc074320 0x7f10dc19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:09.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.522+0000 7f10db200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10dc075900 0x7f10dc19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.522+0000 7f10db200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10dc10dd90 0x7f10dc1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.522+0000 7f10db200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc113a30 con 0x7f10dc075900 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.522+0000 7f10db200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc1138b0 con 0x7f10dc074320 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.522+0000 7f10db200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc113bb0 con 0x7f10dc10dd90 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10dc074320 0x7f10dc19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10dc074320 0x7f10dc19e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54350/0 (socket says 172.21.15.97:54350) 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d9e00640 1 -- 172.21.15.97:0/3353788010 learned_addr learned my addr 172.21.15.97:0/3353788010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d3e00640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10dc075900 0x7f10dc19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10da800640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10dc10dd90 0x7f10dc1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d9e00640 1 -- 172.21.15.97:0/3353788010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10dc10dd90 msgr2=0x7f10dc1a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d9e00640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10dc10dd90 0x7f10dc1a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d9e00640 1 -- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10dc075900 msgr2=0x7f10dc19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:09.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d9e00640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10dc075900 0x7f10dc19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d9e00640 1 -- 172.21.15.97:0/3353788010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc1a5c90 con 0x7f10dc074320 2024-03-26T18:58:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d3e00640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10dc075900 0x7f10dc19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.523+0000 7f10d9e00640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10dc074320 0x7f10dc19e070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00e9f0 tx=0x7f10cc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.524+0000 7f10d2a00640 1 -- 172.21.15.97:0/3353788010 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc00cd80 con 0x7f10dc074320 2024-03-26T18:58:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.524+0000 7f10d2a00640 1 -- 172.21.15.97:0/3353788010 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10cc005500 con 0x7f10dc074320 2024-03-26T18:58:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.524+0000 7f10db200640 1 -- 172.21.15.97:0/3353788010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc1a5f80 con 0x7f10dc074320 2024-03-26T18:58:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.524+0000 7f10d2a00640 1 -- 172.21.15.97:0/3353788010 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc010690 con 0x7f10dc074320 2024-03-26T18:58:09.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.524+0000 7f10db200640 1 -- 172.21.15.97:0/3353788010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1a63e0 con 0x7f10dc074320 2024-03-26T18:58:09.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.525+0000 7f10db200640 1 -- 172.21.15.97:0/3353788010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc068600 con 0x7f10dc074320 2024-03-26T18:58:09.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.526+0000 7f10d2a00640 1 -- 172.21.15.97:0/3353788010 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f10cc0046c0 con 0x7f10dc074320 2024-03-26T18:58:09.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.528+0000 7f10d2a00640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10c00a2fa0 0x7f10c00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:09.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.528+0000 7f10d2a00640 1 -- 172.21.15.97:0/3353788010 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f10cc0cf7d0 con 0x7f10dc074320 2024-03-26T18:58:09.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.528+0000 7f10d3e00640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10c00a2fa0 0x7f10c00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:09.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.529+0000 7f10d3e00640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10c00a2fa0 0x7f10c00a5460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f10dc19f580 tx=0x7f10c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:09.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.532+0000 7f10d2a00640 1 -- 172.21.15.97:0/3353788010 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f10dc068600 con 0x7f10dc074320 2024-03-26T18:58:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.737+0000 7f10db200640 1 -- 172.21.15.97:0/3353788010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f10dc0624c0 con 0x7f10dc074320 2024-03-26T18:58:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.738+0000 7f10d2a00640 1 -- 172.21.15.97:0/3353788010 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f10cc09c630 con 0x7f10dc074320 2024-03-26T18:58:09.731 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-26T18:58:09.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.740+0000 7f10db200640 1 -- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10c00a2fa0 msgr2=0x7f10c00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:09.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.740+0000 7f10db200640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10c00a2fa0 0x7f10c00a5460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f10dc19f580 tx=0x7f10c4005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:09.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.740+0000 7f10db200640 1 -- 172.21.15.97:0/3353788010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10dc074320 msgr2=0x7f10dc19e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.740+0000 7f10db200640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10dc074320 0x7f10dc19e070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00e9f0 tx=0x7f10cc00edf0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.740+0000 7f10db200640 1 -- 172.21.15.97:0/3353788010 shutdown_connections 2024-03-26T18:58:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.740+0000 7f10db200640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10c00a2fa0 0x7f10c00a5460 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.741+0000 7f10db200640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10dc10dd90 0x7f10dc1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.741+0000 7f10db200640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10dc075900 0x7f10dc19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.741+0000 7f10db200640 1 --2- 172.21.15.97:0/3353788010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10dc074320 0x7f10dc19e070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.741+0000 7f10db200640 1 -- 172.21.15.97:0/3353788010 >> 172.21.15.97:0/3353788010 conn(0x7f10dc0fcd30 msgr2=0x7f10dc101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.741+0000 7f10db200640 1 -- 172.21.15.97:0/3353788010 shutdown_connections 2024-03-26T18:58:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.741+0000 7f10db200640 1 -- 172.21.15.97:0/3353788010 wait complete. 2024-03-26T18:58:09.758 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:58:09.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.959+0000 7ff5f2e00640 1 -- 172.21.15.97:0/1402111247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ec101890 msgr2=0x7ff5ec101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.959+0000 7ff5f2e00640 1 --2- 172.21.15.97:0/1402111247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ec101890 0x7ff5ec101d10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009900 tx=0x7ff5d802bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.959+0000 7ff5f2e00640 1 -- 172.21.15.97:0/1402111247 shutdown_connections 2024-03-26T18:58:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.959+0000 7ff5f2e00640 1 --2- 172.21.15.97:0/1402111247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec102250 0x7ff5ec108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.959+0000 7ff5f2e00640 1 --2- 172.21.15.97:0/1402111247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ec101890 0x7ff5ec101d10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.959+0000 7ff5f2e00640 1 --2- 172.21.15.97:0/1402111247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec1006a0 0x7ff5ec100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.959+0000 7ff5f2e00640 1 -- 172.21.15.97:0/1402111247 >> 172.21.15.97:0/1402111247 conn(0x7ff5ec0fbc30 msgr2=0x7ff5ec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.959+0000 7ff5f2e00640 1 -- 172.21.15.97:0/1402111247 shutdown_connections 2024-03-26T18:58:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.959+0000 7ff5f2e00640 1 -- 172.21.15.97:0/1402111247 wait complete. 2024-03-26T18:58:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.960+0000 7ff5f2e00640 1 Processor -- start 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.960+0000 7ff5f2e00640 1 -- start start 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.960+0000 7ff5f2e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec1006a0 0x7ff5ec10b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.960+0000 7ff5f2e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec101890 0x7ff5ec109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.960+0000 7ff5f2e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ec102250 0x7ff5ec109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.960+0000 7ff5f2e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec10d4a0 con 0x7ff5ec101890 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.960+0000 7ff5f2e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec10d320 con 0x7ff5ec1006a0 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.960+0000 7ff5f2e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec10d620 con 0x7ff5ec102250 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec1006a0 0x7ff5ec10b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec101890 0x7ff5ec109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f2400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ec102250 0x7ff5ec109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec1006a0 0x7ff5ec10b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54374/0 (socket says 172.21.15.97:54374) 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f1a00640 1 -- 172.21.15.97:0/3032500775 learned_addr learned my addr 172.21.15.97:0/3032500775 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f1a00640 1 -- 172.21.15.97:0/3032500775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ec102250 msgr2=0x7ff5ec109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:09.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f1a00640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ec102250 0x7ff5ec109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f1a00640 1 -- 172.21.15.97:0/3032500775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec101890 msgr2=0x7ff5ec109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:09.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f1a00640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec101890 0x7ff5ec109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:09.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f1a00640 1 -- 172.21.15.97:0/3032500775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec10a430 con 0x7ff5ec1006a0 2024-03-26T18:58:09.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5f1a00640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec1006a0 0x7ff5ec10b020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000e9c0 tx=0x7ff5e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:09.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.961+0000 7ff5ea000640 1 -- 172.21.15.97:0/3032500775 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000cd80 con 0x7ff5ec1006a0 2024-03-26T18:58:09.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.962+0000 7ff5ea000640 1 -- 172.21.15.97:0/3032500775 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e00035f0 con 0x7ff5ec1006a0 2024-03-26T18:58:09.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.962+0000 7ff5ea000640 1 -- 172.21.15.97:0/3032500775 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0010690 con 0x7ff5ec1006a0 2024-03-26T18:58:09.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.962+0000 7ff5f2e00640 1 -- 172.21.15.97:0/3032500775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec1a1b70 con 0x7ff5ec1006a0 2024-03-26T18:58:09.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.962+0000 7ff5f2e00640 1 -- 172.21.15.97:0/3032500775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec1a2040 con 0x7ff5ec1006a0 2024-03-26T18:58:09.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.963+0000 7ff5f2e00640 1 -- 172.21.15.97:0/3032500775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec0685d0 con 0x7ff5ec1006a0 2024-03-26T18:58:09.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.965+0000 7ff5ea000640 1 -- 172.21.15.97:0/3032500775 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7ff5e00037a0 con 0x7ff5ec1006a0 2024-03-26T18:58:09.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.966+0000 7ff5ea000640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5d40a2fa0 0x7ff5d40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:09.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.966+0000 7ff5ea000640 1 -- 172.21.15.97:0/3032500775 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff5e00cf7d0 con 0x7ff5ec1006a0 2024-03-26T18:58:09.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.966+0000 7ff5f1000640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5d40a2fa0 0x7ff5d40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:09.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.967+0000 7ff5f1000640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5d40a2fa0 0x7ff5d40a5460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff5d802c240 tx=0x7ff5d802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:09.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:09.970+0000 7ff5ea000640 1 -- 172.21.15.97:0/3032500775 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff5e009c5d0 con 0x7ff5ec1006a0 2024-03-26T18:58:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.172+0000 7ff5f2e00640 1 -- 172.21.15.97:0/3032500775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff5ec0624f0 con 0x7ff5ec1006a0 2024-03-26T18:58:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.173+0000 7ff5ea000640 1 -- 172.21.15.97:0/3032500775 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7ff5e009bed0 con 0x7ff5ec1006a0 2024-03-26T18:58:10.165 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-26T18:58:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.175+0000 7ff5f2e00640 1 -- 172.21.15.97:0/3032500775 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5d40a2fa0 msgr2=0x7ff5d40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.175+0000 7ff5f2e00640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5d40a2fa0 0x7ff5d40a5460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff5d802c240 tx=0x7ff5d802df90 comp rx=0 tx=0).stop 2024-03-26T18:58:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.175+0000 7ff5f2e00640 1 -- 172.21.15.97:0/3032500775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec1006a0 msgr2=0x7ff5ec10b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.175+0000 7ff5f2e00640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec1006a0 0x7ff5ec10b020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000e9c0 tx=0x7ff5e000ee80 comp rx=0 tx=0).stop 2024-03-26T18:58:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.176+0000 7ff5f2e00640 1 -- 172.21.15.97:0/3032500775 shutdown_connections 2024-03-26T18:58:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.176+0000 7ff5f2e00640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff5d40a2fa0 0x7ff5d40a5460 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.176+0000 7ff5f2e00640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ec102250 0x7ff5ec109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.176+0000 7ff5f2e00640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec101890 0x7ff5ec109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.176+0000 7ff5f2e00640 1 --2- 172.21.15.97:0/3032500775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec1006a0 0x7ff5ec10b020 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.176+0000 7ff5f2e00640 1 -- 172.21.15.97:0/3032500775 >> 172.21.15.97:0/3032500775 conn(0x7ff5ec0fbc30 msgr2=0x7ff5ec104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.176+0000 7ff5f2e00640 1 -- 172.21.15.97:0/3032500775 shutdown_connections 2024-03-26T18:58:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.176+0000 7ff5f2e00640 1 -- 172.21.15.97:0/3032500775 wait complete. 2024-03-26T18:58:10.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:58:10.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:10 smithi097 ceph-mon[31955]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T18:58:10.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:10 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3251719147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:10.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:10 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3353788010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:10.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:10 smithi144 ceph-mon[36053]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T18:58:10.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:10 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3251719147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:10.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:10 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3353788010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.399+0000 7ff074c00640 1 -- 172.21.15.97:0/193418142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0701054c0 msgr2=0x7ff07010bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.399+0000 7ff074c00640 1 --2- 172.21.15.97:0/193418142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0701054c0 0x7ff07010bd50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff068009be0 tx=0x7ff06802bef0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.399+0000 7ff074c00640 1 -- 172.21.15.97:0/193418142 shutdown_connections 2024-03-26T18:58:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.399+0000 7ff074c00640 1 --2- 172.21.15.97:0/193418142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0701054c0 0x7ff07010bd50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.399+0000 7ff074c00640 1 --2- 172.21.15.97:0/193418142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff070104b00 0x7ff070104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.399+0000 7ff074c00640 1 --2- 172.21.15.97:0/193418142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff070103910 0x7ff070103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.399+0000 7ff074c00640 1 -- 172.21.15.97:0/193418142 >> 172.21.15.97:0/193418142 conn(0x7ff0700feea0 msgr2=0x7ff0701012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.399+0000 7ff074c00640 1 -- 172.21.15.97:0/193418142 shutdown_connections 2024-03-26T18:58:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.399+0000 7ff074c00640 1 -- 172.21.15.97:0/193418142 wait complete. 2024-03-26T18:58:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.401+0000 7ff074c00640 1 Processor -- start 2024-03-26T18:58:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.401+0000 7ff074c00640 1 -- start start 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff074c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff070103910 0x7ff070199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff074c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff070104b00 0x7ff07019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff074c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0701054c0 0x7ff0701a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff074c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff07010f570 con 0x7ff070104b00 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff074c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff07010f3f0 con 0x7ff0701054c0 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff074c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff07010f6f0 con 0x7ff070103910 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff070103910 0x7ff070199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff070103910 0x7ff070199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56664/0 (socket says 172.21.15.97:56664) 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06f400640 1 -- 172.21.15.97:0/4135624393 learned_addr learned my addr 172.21.15.97:0/4135624393 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06ea00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff070104b00 0x7ff07019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06fe00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0701054c0 0x7ff0701a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06f400640 1 -- 172.21.15.97:0/4135624393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0701054c0 msgr2=0x7ff0701a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06f400640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0701054c0 0x7ff0701a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06f400640 1 -- 172.21.15.97:0/4135624393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff070104b00 msgr2=0x7ff07019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06f400640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff070104b00 0x7ff07019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06f400640 1 -- 172.21.15.97:0/4135624393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0701a17d0 con 0x7ff070103910 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.402+0000 7ff06fe00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0701054c0 0x7ff0701a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.403+0000 7ff06f400640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff070103910 0x7ff070199bb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff06000d950 tx=0x7ff06000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.403+0000 7ff067e00640 1 -- 172.21.15.97:0/4135624393 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060017020 con 0x7ff070103910 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.403+0000 7ff067e00640 1 -- 172.21.15.97:0/4135624393 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff060009c30 con 0x7ff070103910 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.403+0000 7ff067e00640 1 -- 172.21.15.97:0/4135624393 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06000f720 con 0x7ff070103910 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.403+0000 7ff074c00640 1 -- 172.21.15.97:0/4135624393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0701a1ac0 con 0x7ff070103910 2024-03-26T18:58:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.403+0000 7ff074c00640 1 -- 172.21.15.97:0/4135624393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0701a1fc0 con 0x7ff070103910 2024-03-26T18:58:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.404+0000 7ff074c00640 1 -- 172.21.15.97:0/4135624393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff070068a20 con 0x7ff070103910 2024-03-26T18:58:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.407+0000 7ff067e00640 1 -- 172.21.15.97:0/4135624393 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7ff06001a020 con 0x7ff070103910 2024-03-26T18:58:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.409+0000 7ff067e00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff0540a2f20 0x7ff0540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.409+0000 7ff067e00640 1 -- 172.21.15.97:0/4135624393 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff0600cf320 con 0x7ff070103910 2024-03-26T18:58:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.409+0000 7ff06ea00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff0540a2f20 0x7ff0540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.410+0000 7ff06ea00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff0540a2f20 0x7ff0540a53e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0096c0 tx=0x7ff05c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:10.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.413+0000 7ff067e00640 1 -- 172.21.15.97:0/4135624393 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff060021020 con 0x7ff070103910 2024-03-26T18:58:10.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:10 smithi105 ceph-mon[36270]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T18:58:10.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:10 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3251719147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:10.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:10 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3353788010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.694+0000 7ff074c00640 1 -- 172.21.15.97:0/4135624393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff07010abd0 con 0x7ff070103910 2024-03-26T18:58:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.695+0000 7ff067e00640 1 -- 172.21.15.97:0/4135624393 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11935 (secure 0 0 0) 0x7ff060009de0 con 0x7ff070103910 2024-03-26T18:58:10.687 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:10.688 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T18:56:23.642405+0000","modified":"2024-03-26T18:56:31.851339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24535,"mds_3":24494,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2391073232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2391073232},{"type":"v1","addr":"172.21.15.105:6839","nonce":2391073232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24535,24494,24481]},"id":1}]} 2024-03-26T18:58:10.688 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-03-26T18:58:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.697+0000 7ff074c00640 1 -- 172.21.15.97:0/4135624393 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff0540a2f20 msgr2=0x7ff0540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.697+0000 7ff074c00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff0540a2f20 0x7ff0540a53e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0096c0 tx=0x7ff05c009260 comp rx=0 tx=0).stop 2024-03-26T18:58:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.697+0000 7ff074c00640 1 -- 172.21.15.97:0/4135624393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff070103910 msgr2=0x7ff070199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.697+0000 7ff074c00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff070103910 0x7ff070199bb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff06000d950 tx=0x7ff06000de10 comp rx=0 tx=0).stop 2024-03-26T18:58:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.697+0000 7ff074c00640 1 -- 172.21.15.97:0/4135624393 shutdown_connections 2024-03-26T18:58:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.697+0000 7ff074c00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff0540a2f20 0x7ff0540a53e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.697+0000 7ff074c00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0701054c0 0x7ff0701a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.697+0000 7ff074c00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff070104b00 0x7ff07019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.697+0000 7ff074c00640 1 --2- 172.21.15.97:0/4135624393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff070103910 0x7ff070199bb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.697+0000 7ff074c00640 1 -- 172.21.15.97:0/4135624393 >> 172.21.15.97:0/4135624393 conn(0x7ff0700feea0 msgr2=0x7ff070107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.698+0000 7ff074c00640 1 -- 172.21.15.97:0/4135624393 shutdown_connections 2024-03-26T18:58:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.698+0000 7ff074c00640 1 -- 172.21.15.97:0/4135624393 wait complete. 2024-03-26T18:58:10.708 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:58:10.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.909+0000 7f6e40a00640 1 -- 172.21.15.97:0/1406851077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c101720 msgr2=0x7f6e3c101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:10.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.909+0000 7f6e40a00640 1 --2- 172.21.15.97:0/1406851077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c101720 0x7f6e3c101b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009900 tx=0x7f6e2c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:10.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.909+0000 7f6e40a00640 1 -- 172.21.15.97:0/1406851077 shutdown_connections 2024-03-26T18:58:10.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.909+0000 7f6e40a00640 1 --2- 172.21.15.97:0/1406851077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c1032d0 0x7f6e3c109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.909+0000 7f6e40a00640 1 --2- 172.21.15.97:0/1406851077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e3c102910 0x7f6e3c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.909+0000 7f6e40a00640 1 --2- 172.21.15.97:0/1406851077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c101720 0x7f6e3c101b40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.909+0000 7f6e40a00640 1 -- 172.21.15.97:0/1406851077 >> 172.21.15.97:0/1406851077 conn(0x7f6e3c0fcc90 msgr2=0x7f6e3c0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:10.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.909+0000 7f6e40a00640 1 -- 172.21.15.97:0/1406851077 shutdown_connections 2024-03-26T18:58:10.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.909+0000 7f6e40a00640 1 -- 172.21.15.97:0/1406851077 wait complete. 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.910+0000 7f6e40a00640 1 Processor -- start 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.910+0000 7f6e40a00640 1 -- start start 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e40a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c101720 0x7f6e3c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e40a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c102910 0x7f6e3c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e40a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e3c1032d0 0x7f6e3c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e40a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c10d3c0 con 0x7f6e3c1032d0 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e40a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c10d240 con 0x7f6e3c101720 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e40a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c10d540 con 0x7f6e3c102910 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e3b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c101720 0x7f6e3c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e3b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c101720 0x7f6e3c197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52184/0 (socket says 172.21.15.97:52184) 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e3b400640 1 -- 172.21.15.97:0/1157467679 learned_addr learned my addr 172.21.15.97:0/1157467679 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:10.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e3be00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e3c1032d0 0x7f6e3c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.911+0000 7f6e3aa00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c102910 0x7f6e3c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e3b400640 1 -- 172.21.15.97:0/1157467679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c102910 msgr2=0x7f6e3c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e3b400640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c102910 0x7f6e3c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e3b400640 1 -- 172.21.15.97:0/1157467679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e3c1032d0 msgr2=0x7f6e3c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e3b400640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e3c1032d0 0x7f6e3c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e3b400640 1 -- 172.21.15.97:0/1157467679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c19f630 con 0x7f6e3c101720 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e3aa00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c102910 0x7f6e3c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e3b400640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c101720 0x7f6e3c197a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c000c00 tx=0x7f6e2c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e33e00640 1 -- 172.21.15.97:0/1157467679 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c043020 con 0x7f6e3c101720 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e40a00640 1 -- 172.21.15.97:0/1157467679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c19f8c0 con 0x7f6e3c101720 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e33e00640 1 -- 172.21.15.97:0/1157467679 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2c0054e0 con 0x7f6e3c101720 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e33e00640 1 -- 172.21.15.97:0/1157467679 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c003c40 con 0x7f6e3c101720 2024-03-26T18:58:10.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.912+0000 7f6e40a00640 1 -- 172.21.15.97:0/1157467679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c19fe10 con 0x7f6e3c101720 2024-03-26T18:58:10.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.914+0000 7f6e40a00640 1 -- 172.21.15.97:0/1157467679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3c108a40 con 0x7f6e3c101720 2024-03-26T18:58:10.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.915+0000 7f6e33e00640 1 -- 172.21.15.97:0/1157467679 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f6e2c046020 con 0x7f6e3c101720 2024-03-26T18:58:10.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.917+0000 7f6e33e00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6e200a2f20 0x7f6e200a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:10.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.918+0000 7f6e33e00640 1 -- 172.21.15.97:0/1157467679 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6e2c0f14c0 con 0x7f6e3c101720 2024-03-26T18:58:10.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.918+0000 7f6e3aa00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6e200a2f20 0x7f6e200a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:10.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.918+0000 7f6e3aa00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6e200a2f20 0x7f6e200a53e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c198f20 tx=0x7f6e28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:10.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:10.924+0000 7f6e33e00640 1 -- 172.21.15.97:0/1157467679 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e2c04d020 con 0x7f6e3c101720 2024-03-26T18:58:11.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.145+0000 7f6e40a00640 1 -- 172.21.15.97:0/1157467679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e3c0624c0 con 0x7f6e3c101720 2024-03-26T18:58:11.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.146+0000 7f6e33e00640 1 -- 172.21.15.97:0/1157467679 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22581 (secure 0 0 0) 0x7f6e2c0be2c0 con 0x7f6e3c101720 2024-03-26T18:58:11.139 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:11.140 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:57:30.061624+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:58:11.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 -- 172.21.15.97:0/1157467679 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6e200a2f20 msgr2=0x7f6e200a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6e200a2f20 0x7f6e200a53e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c198f20 tx=0x7f6e28005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:11.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 -- 172.21.15.97:0/1157467679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c101720 msgr2=0x7f6e3c197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c101720 0x7f6e3c197a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c000c00 tx=0x7f6e2c02c650 comp rx=0 tx=0).stop 2024-03-26T18:58:11.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 -- 172.21.15.97:0/1157467679 shutdown_connections 2024-03-26T18:58:11.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6e200a2f20 0x7f6e200a53e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e3c1032d0 0x7f6e3c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c102910 0x7f6e3c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 --2- 172.21.15.97:0/1157467679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c101720 0x7f6e3c197a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 -- 172.21.15.97:0/1157467679 >> 172.21.15.97:0/1157467679 conn(0x7f6e3c0fcc90 msgr2=0x7f6e3c105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.149+0000 7f6e40a00640 1 -- 172.21.15.97:0/1157467679 shutdown_connections 2024-03-26T18:58:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.150+0000 7f6e40a00640 1 -- 172.21.15.97:0/1157467679 wait complete. 2024-03-26T18:58:11.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:58:11.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:11 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3032500775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:11.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:11 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/4135624393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:11.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:11 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3032500775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:11.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:11 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/4135624393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:11.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.389+0000 7f522de00640 1 -- 172.21.15.97:0/1173357320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5228100b30 msgr2=0x7f5228100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.389+0000 7f522de00640 1 --2- 172.21.15.97:0/1173357320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5228100b30 0x7f5228100f90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5210009900 tx=0x7f521002bcd0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.389+0000 7f522de00640 1 -- 172.21.15.97:0/1173357320 shutdown_connections 2024-03-26T18:58:11.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.389+0000 7f522de00640 1 --2- 172.21.15.97:0/1173357320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52281014d0 0x7f5228101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.389+0000 7f522de00640 1 --2- 172.21.15.97:0/1173357320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5228100b30 0x7f5228100f90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.389+0000 7f522de00640 1 --2- 172.21.15.97:0/1173357320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52281001d0 0x7f52281005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.389+0000 7f522de00640 1 -- 172.21.15.97:0/1173357320 >> 172.21.15.97:0/1173357320 conn(0x7f52280fbd50 msgr2=0x7f52280fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:11.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.389+0000 7f522de00640 1 -- 172.21.15.97:0/1173357320 shutdown_connections 2024-03-26T18:58:11.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.389+0000 7f522de00640 1 -- 172.21.15.97:0/1173357320 wait complete. 2024-03-26T18:58:11.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.390+0000 7f522de00640 1 Processor -- start 2024-03-26T18:58:11.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.390+0000 7f522de00640 1 -- start start 2024-03-26T18:58:11.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f522de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52281001d0 0x7f5228199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f522de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5228100b30 0x7f522819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f522de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52281014d0 0x7f52281a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f522de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f522810f790 con 0x7f5228100b30 2024-03-26T18:58:11.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f522de00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f522810f610 con 0x7f52281001d0 2024-03-26T18:58:11.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f522de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f522810f910 con 0x7f52281014d0 2024-03-26T18:58:11.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f5226a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52281001d0 0x7f5228199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f5226a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52281001d0 0x7f5228199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52210/0 (socket says 172.21.15.97:52210) 2024-03-26T18:58:11.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f5226a00640 1 -- 172.21.15.97:0/1172685497 learned_addr learned my addr 172.21.15.97:0/1172685497 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:11.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f5227400640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52281014d0 0x7f52281a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f521e000640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5228100b30 0x7f522819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f5226a00640 1 -- 172.21.15.97:0/1172685497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52281014d0 msgr2=0x7f52281a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f5226a00640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52281014d0 0x7f52281a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f5226a00640 1 -- 172.21.15.97:0/1172685497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5228100b30 msgr2=0x7f522819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f5226a00640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5228100b30 0x7f522819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.391+0000 7f5226a00640 1 -- 172.21.15.97:0/1172685497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52281a1a30 con 0x7f52281001d0 2024-03-26T18:58:11.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.392+0000 7f5226a00640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52281001d0 0x7f5228199e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f521800ebc0 tx=0x7f521800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:11.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.392+0000 7f522ca00640 1 -- 172.21.15.97:0/1172685497 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218010430 con 0x7f52281001d0 2024-03-26T18:58:11.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.392+0000 7f522ca00640 1 -- 172.21.15.97:0/1172685497 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5218010ab0 con 0x7f52281001d0 2024-03-26T18:58:11.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.392+0000 7f522ca00640 1 -- 172.21.15.97:0/1172685497 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218018b30 con 0x7f52281001d0 2024-03-26T18:58:11.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.392+0000 7f522de00640 1 -- 172.21.15.97:0/1172685497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52281a1d20 con 0x7f52281001d0 2024-03-26T18:58:11.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.392+0000 7f522de00640 1 -- 172.21.15.97:0/1172685497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52281a21d0 con 0x7f52281001d0 2024-03-26T18:58:11.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.394+0000 7f522de00640 1 -- 172.21.15.97:0/1172685497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522810b0d0 con 0x7f52281001d0 2024-03-26T18:58:11.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.395+0000 7f522ca00640 1 -- 172.21.15.97:0/1172685497 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f5218018cd0 con 0x7f52281001d0 2024-03-26T18:58:11.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.396+0000 7f522ca00640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f520c0a2fa0 0x7f520c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.396+0000 7f522ca00640 1 -- 172.21.15.97:0/1172685497 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f52180cfa30 con 0x7f52281001d0 2024-03-26T18:58:11.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.396+0000 7f521e000640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f520c0a2fa0 0x7f520c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.397+0000 7f521e000640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f520c0a2fa0 0x7f520c0a5460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f521002c2e0 tx=0x7f521002c920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:11.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.400+0000 7f522ca00640 1 -- 172.21.15.97:0/1172685497 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f521809c830 con 0x7f52281001d0 2024-03-26T18:58:11.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:11 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3032500775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:11.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:11 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/4135624393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:11.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.639+0000 7f522de00640 1 -- 172.21.15.97:0/1172685497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f522802d240 con 0x7f52281001d0 2024-03-26T18:58:11.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.640+0000 7f522ca00640 1 -- 172.21.15.97:0/1172685497 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11935 (secure 0 0 0) 0x7f521809c130 con 0x7f52281001d0 2024-03-26T18:58:11.633 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:11.633 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T18:56:23.642405+0000","modified":"2024-03-26T18:56:31.851339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24535,"mds_3":24494,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2391073232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2391073232},{"type":"v1","addr":"172.21.15.105:6839","nonce":2391073232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24535,24494,24481]},"id":1}]} 2024-03-26T18:58:11.633 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-03-26T18:58:11.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.642+0000 7f522de00640 1 -- 172.21.15.97:0/1172685497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f520c0a2fa0 msgr2=0x7f520c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.642+0000 7f522de00640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f520c0a2fa0 0x7f520c0a5460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f521002c2e0 tx=0x7f521002c920 comp rx=0 tx=0).stop 2024-03-26T18:58:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.642+0000 7f522de00640 1 -- 172.21.15.97:0/1172685497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52281001d0 msgr2=0x7f5228199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.642+0000 7f522de00640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52281001d0 0x7f5228199e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f521800ebc0 tx=0x7f521800c2d0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.643+0000 7f522de00640 1 -- 172.21.15.97:0/1172685497 shutdown_connections 2024-03-26T18:58:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.643+0000 7f522de00640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f520c0a2fa0 0x7f520c0a5460 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.643+0000 7f522de00640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52281014d0 0x7f52281a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.643+0000 7f522de00640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5228100b30 0x7f522819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.643+0000 7f522de00640 1 --2- 172.21.15.97:0/1172685497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52281001d0 0x7f5228199e10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.643+0000 7f522de00640 1 -- 172.21.15.97:0/1172685497 >> 172.21.15.97:0/1172685497 conn(0x7f52280fbd50 msgr2=0x7f52280fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.643+0000 7f522de00640 1 -- 172.21.15.97:0/1172685497 shutdown_connections 2024-03-26T18:58:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.643+0000 7f522de00640 1 -- 172.21.15.97:0/1172685497 wait complete. 2024-03-26T18:58:11.656 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T18:56:23.642405+0000', 'modified': '2024-03-26T18:56:31.851339+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24581, 'mds_1': 24569, 'mds_2': 24535, 'mds_3': 24494, 'mds_4': 24481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6833/3099308706', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3099308706}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3099308706}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6833/254387341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6832', 'nonce': 254387341}, {'type': 'v1', 'addr': '172.21.15.105:6833', 'nonce': 254387341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'd', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.97:6837/4124660285', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 4124660285}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 4124660285}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.105:6835/1538692215', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 1538692215}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 1538692215}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/411039946', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 411039946}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 411039946}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6837/1138636767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 1138636767}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 1138636767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6837/1343808654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 1343808654}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1343808654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.144:6839/994192292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 994192292}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 994192292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'a', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.97:6841/4165060179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6840', 'nonce': 4165060179}, {'type': 'v1', 'addr': '172.21.15.97:6841', 'nonce': 4165060179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6839/2391073232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 2391073232}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 2391073232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24569, 24581, 24535, 24494, 24481]} 2024-03-26T18:58:11.656 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-26T18:58:11.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-26T18:58:11.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.868+0000 7f8d7d800640 1 -- 172.21.15.97:0/3033976902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d78104bb0 msgr2=0x7f8d78106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.868+0000 7f8d7d800640 1 --2- 172.21.15.97:0/3033976902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d78104bb0 0x7f8d78106fa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009900 tx=0x7f8d6402bc40 comp rx=0 tx=0).stop 2024-03-26T18:58:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.868+0000 7f8d7d800640 1 -- 172.21.15.97:0/3033976902 shutdown_connections 2024-03-26T18:58:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.868+0000 7f8d7d800640 1 --2- 172.21.15.97:0/3033976902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d781074e0 0x7f8d781098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.868+0000 7f8d7d800640 1 --2- 172.21.15.97:0/3033976902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d78104bb0 0x7f8d78106fa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.868+0000 7f8d7d800640 1 --2- 172.21.15.97:0/3033976902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d78102280 0x7f8d78104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.868+0000 7f8d7d800640 1 -- 172.21.15.97:0/3033976902 >> 172.21.15.97:0/3033976902 conn(0x7f8d780fbc30 msgr2=0x7f8d780fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.868+0000 7f8d7d800640 1 -- 172.21.15.97:0/3033976902 shutdown_connections 2024-03-26T18:58:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.868+0000 7f8d7d800640 1 -- 172.21.15.97:0/3033976902 wait complete. 2024-03-26T18:58:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.869+0000 7f8d7d800640 1 Processor -- start 2024-03-26T18:58:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.869+0000 7f8d7d800640 1 -- start start 2024-03-26T18:58:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d78102280 0x7f8d78197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7d800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d78104bb0 0x7f8d78198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7d800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d781074e0 0x7f8d7819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7810d420 con 0x7f8d78102280 2024-03-26T18:58:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7d800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7810d2a0 con 0x7f8d781074e0 2024-03-26T18:58:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7d800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7810d5a0 con 0x7f8d78104bb0 2024-03-26T18:58:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d78102280 0x7f8d78197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d781074e0 0x7f8d7819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d77400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d78104bb0 0x7f8d78198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d77400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d78104bb0 0x7f8d78198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37268/0 (socket says 172.21.15.97:37268) 2024-03-26T18:58:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d77400640 1 -- 172.21.15.97:0/3801104303 learned_addr learned my addr 172.21.15.97:0/3801104303 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7ce00640 1 -- 172.21.15.97:0/3801104303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d78104bb0 msgr2=0x7f8d78198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7ce00640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d78104bb0 0x7f8d78198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7ce00640 1 -- 172.21.15.97:0/3801104303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d78102280 msgr2=0x7f8d78197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7ce00640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d78102280 0x7f8d78197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.870+0000 7f8d7ce00640 1 -- 172.21.15.97:0/3801104303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7819f6e0 con 0x7f8d781074e0 2024-03-26T18:58:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.871+0000 7f8d7ce00640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d781074e0 0x7f8d7819f070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009fd0 tx=0x7f8d7000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.871+0000 7f8d74c00640 1 -- 172.21.15.97:0/3801104303 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7000ce20 con 0x7f8d781074e0 2024-03-26T18:58:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.871+0000 7f8d74c00640 1 -- 172.21.15.97:0/3801104303 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d70005020 con 0x7f8d781074e0 2024-03-26T18:58:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.871+0000 7f8d74c00640 1 -- 172.21.15.97:0/3801104303 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70010720 con 0x7f8d781074e0 2024-03-26T18:58:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.871+0000 7f8d7d800640 1 -- 172.21.15.97:0/3801104303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7819f9d0 con 0x7f8d781074e0 2024-03-26T18:58:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.871+0000 7f8d7d800640 1 -- 172.21.15.97:0/3801104303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7819ff40 con 0x7f8d781074e0 2024-03-26T18:58:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.874+0000 7f8d81ba8740 1 Processor -- start 2024-03-26T18:58:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.875+0000 7f8d74c00640 1 -- 172.21.15.97:0/3801104303 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f8d70005540 con 0x7f8d781074e0 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d81ba8740 1 -- start start 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644075fb50 0x55644075ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556440760540 0x556440765040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 0x556440767b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5564406c41b0 con 0x556440760540 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55644066cb00 con 0x5564407656b0 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5564406c0160 con 0x55644075fb50 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d77400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644075fb50 0x55644075ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556440760540 0x556440765040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d77400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644075fb50 0x55644075ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37276/0 (socket says 172.21.15.97:37276) 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.876+0000 7f8d77400640 1 -- 172.21.15.97:0/774444668 learned_addr learned my addr 172.21.15.97:0/774444668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d77400640 1 --2- 172.21.15.97:0/774444668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 0x556440767b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d77400640 1 -- 172.21.15.97:0/774444668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 msgr2=0x556440767b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:11.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d77400640 1 --2- 172.21.15.97:0/774444668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 0x556440767b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d77400640 1 -- 172.21.15.97:0/774444668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556440760540 msgr2=0x556440765040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:11.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d77400640 1 --2- 172.21.15.97:0/774444668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556440760540 0x556440765040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d77400640 1 -- 172.21.15.97:0/774444668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564403d82d0 con 0x55644075fb50 2024-03-26T18:58:11.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d77400640 1 --2- 172.21.15.97:0/774444668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644075fb50 0x55644075ff70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d64005c40 tx=0x7f8d64005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b177a4b985efde2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:11.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d74c00640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8d5c0a2fa0 0x7f8d5c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d6d600640 1 -- 172.21.15.97:0/774444668 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64004230 con 0x55644075fb50 2024-03-26T18:58:11.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d6d600640 1 -- 172.21.15.97:0/774444668 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d640043d0 con 0x55644075fb50 2024-03-26T18:58:11.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d6d600640 1 -- 172.21.15.97:0/774444668 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64005410 con 0x55644075fb50 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.877+0000 7f8d74c00640 1 -- 172.21.15.97:0/3801104303 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8d700cf910 con 0x7f8d781074e0 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.878+0000 7f8d77e00640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8d5c0a2fa0 0x7f8d5c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.878+0000 7f8d81ba8740 1 -- 172.21.15.97:0/774444668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644075fb50 msgr2=0x55644075ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.878+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/774444668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644075fb50 0x55644075ff70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d64005c40 tx=0x7f8d64005b50 comp rx=0 tx=0).stop 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.878+0000 7f8d77e00640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8d5c0a2fa0 0x7f8d5c0a5460 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x556440760940 tx=0x7f8d68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.878+0000 7f8d81ba8740 1 -- 172.21.15.97:0/774444668 shutdown_connections 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.878+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/774444668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 0x556440767b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.878+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/774444668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556440760540 0x556440765040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.878+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/774444668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644075fb50 0x55644075ff70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.878+0000 7f8d81ba8740 1 -- 172.21.15.97:0/774444668 >> 172.21.15.97:0/774444668 conn(0x55644075b0c0 msgr2=0x55644075b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.879+0000 7f8d81ba8740 1 -- 172.21.15.97:0/774444668 shutdown_connections 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.879+0000 7f8d81ba8740 1 -- 172.21.15.97:0/774444668 wait complete. 2024-03-26T18:58:11.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.879+0000 7f8d81ba8740 1 -- start start 2024-03-26T18:58:11.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.879+0000 7f8d81ba8740 1 Processor -- start 2024-03-26T18:58:11.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x55644076a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 0x55644076a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564407656b0 0x5564407715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556440768600 con 0x5564407656b0 2024-03-26T18:58:11.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55644067fed0 con 0x55644075fb50 2024-03-26T18:58:11.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556440768780 con 0x556440760540 2024-03-26T18:58:11.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d81ba8740 10 client.0 _ll_register_callbacks cb 0x5564406bb710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:11.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x55644076a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x55644076a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52228/0 (socket says 172.21.15.97:52228) 2024-03-26T18:58:11.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d77e00640 1 -- 172.21.15.97:0/2953122659 learned_addr learned my addr 172.21.15.97:0/2953122659 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d77400640 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 0x55644076a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d7ce00640 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564407656b0 0x5564407715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d77e00640 1 -- 172.21.15.97:0/2953122659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 msgr2=0x55644076a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d77e00640 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 0x55644076a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d77e00640 1 -- 172.21.15.97:0/2953122659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564407656b0 msgr2=0x5564407715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d77e00640 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564407656b0 0x5564407715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.880+0000 7f8d77e00640 1 -- 172.21.15.97:0/2953122659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564404a26b0 con 0x55644075fb50 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.881+0000 7f8d77e00640 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x55644076a070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009730 tx=0x7f8d68004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=965ea563e60060b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.881+0000 7f8d62000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.881+0000 7f8d62000640 1 -- 172.21.15.97:0/2953122659 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6801f640 con 0x55644075fb50 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.881+0000 7f8d62000640 1 -- 172.21.15.97:0/2953122659 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8d6801f7e0 con 0x55644075fb50 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.881+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2953122659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d640043d0 con 0x55644075fb50 2024-03-26T18:58:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.881+0000 7f8d62000640 1 -- 172.21.15.97:0/2953122659 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d6801f9e0 con 0x55644075fb50 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.881+0000 7f8d81ba8740 10 client.24989 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.881+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2953122659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556440774640 con 0x55644075fb50 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.881+0000 7f8d62000640 1 -- 172.21.15.97:0/2953122659 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d640043d0 con 0x55644075fb50 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.882+0000 7f8d62000640 1 -- 172.21.15.97:0/2953122659 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8d68027070 con 0x55644075fb50 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.882+0000 7f8d81ba8740 10 client.24989 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.882+0000 7f8d81ba8740 10 client.24989 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.882+0000 7f8d81ba8740 20 client.24989 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.882+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x556440776cf0 0x556440778160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.882+0000 7f8d81ba8740 4 client.24989 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.882+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2953122659 --> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564406d16f0 con 0x556440776cf0 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.882+0000 7f8d77400640 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x556440776cf0 0x556440778160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.883+0000 7f8d77400640 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x556440776cf0 0x556440778160 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3f3b6b012eef7718 server_cookie=73c358786cbefb52 in_seq=0 out_seq=0 2024-03-26T18:58:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.883+0000 7f8d62000640 10 client.24989 ms_handle_connect on v2:172.21.15.97:6836/4124660285 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d62000640 1 -- 172.21.15.97:0/2953122659 <== mds.4 v2:172.21.15.97:6836/4124660285 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5564406d16f0 con 0x556440776cf0 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d62000640 10 client.24989 handle_command_reply: tid=0 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 client.24989 shutdown 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2953122659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 msgr2=0x55644076a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x55644076a070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009730 tx=0x7f8d68004730 comp rx=0 tx=0).stop 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2953122659 shutdown_connections 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x556440776cf0 0x556440778160 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564407656b0 0x5564407715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 0x55644076a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d62000640 0 client.24989 ms_handle_reset on v2:172.21.15.97:6836/4124660285 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2953122659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x55644076a070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2953122659 >> 172.21.15.97:0/2953122659 conn(0x55644075b0c0 msgr2=0x55644075a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2953122659 shutdown_connections 2024-03-26T18:58:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2953122659 wait complete. 2024-03-26T18:58:11.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.884+0000 7f8d81ba8740 20 client.24989 trim_cache size 0 max 16384 2024-03-26T18:58:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.915+0000 7f8d81ba8740 1 Processor -- start 2024-03-26T18:58:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.915+0000 7f8d81ba8740 1 -- start start 2024-03-26T18:58:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55644075fb50 0x55644076ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 0x556440761aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 0x556440761fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5564407764a0 con 0x55644075fb50 2024-03-26T18:58:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556440776b70 con 0x5564407656b0 2024-03-26T18:58:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556440777240 con 0x556440760540 2024-03-26T18:58:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d77400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 0x556440761aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d7ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 0x556440761fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55644075fb50 0x55644076ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d77400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 0x556440761aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37292/0 (socket says 172.21.15.97:37292) 2024-03-26T18:58:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d7ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 0x556440761fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52232/0 (socket says 172.21.15.97:52232) 2024-03-26T18:58:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d77400640 1 -- 172.21.15.97:0/1983179028 learned_addr learned my addr 172.21.15.97:0/1983179028 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d77400640 1 -- 172.21.15.97:0/1983179028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 msgr2=0x556440761fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d77400640 1 --2- 172.21.15.97:0/1983179028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 0x556440761fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d77400640 1 -- 172.21.15.97:0/1983179028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55644075fb50 msgr2=0x55644076ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.916+0000 7f8d77400640 1 --2- 172.21.15.97:0/1983179028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55644075fb50 0x55644076ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.917+0000 7f8d77400640 1 -- 172.21.15.97:0/1983179028 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556440777dc0 con 0x556440760540 2024-03-26T18:58:11.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.917+0000 7f8d77400640 1 --2- 172.21.15.97:0/1983179028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 0x556440761aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8d64005e90 tx=0x7f8d64005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95731d351b3af529 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.917+0000 7f8d6cc00640 1 -- 172.21.15.97:0/1983179028 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64053070 con 0x556440760540 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.917+0000 7f8d6cc00640 1 -- 172.21.15.97:0/1983179028 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d64004040 con 0x556440760540 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.917+0000 7f8d6cc00640 1 -- 172.21.15.97:0/1983179028 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64053470 con 0x556440760540 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.918+0000 7f8d81ba8740 1 -- 172.21.15.97:0/1983179028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 msgr2=0x556440761aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.918+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/1983179028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 0x556440761aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8d64005e90 tx=0x7f8d64005e20 comp rx=0 tx=0).stop 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.918+0000 7f8d81ba8740 1 -- 172.21.15.97:0/1983179028 shutdown_connections 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.918+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/1983179028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564407656b0 0x556440761fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.918+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/1983179028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556440760540 0x556440761aa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.918+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/1983179028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55644075fb50 0x55644076ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.918+0000 7f8d81ba8740 1 -- 172.21.15.97:0/1983179028 >> 172.21.15.97:0/1983179028 conn(0x55644075b0c0 msgr2=0x55644075b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.918+0000 7f8d81ba8740 1 -- 172.21.15.97:0/1983179028 shutdown_connections 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.918+0000 7f8d81ba8740 1 -- 172.21.15.97:0/1983179028 wait complete. 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d81ba8740 1 -- start start 2024-03-26T18:58:11.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d81ba8740 1 Processor -- start 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x5564407daa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556440760540 0x5564407dcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d81ba8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564407656b0 0x5564407df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556440777fe0 con 0x556440760540 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556440777910 con 0x55644075fb50 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d81ba8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5564407786b0 con 0x5564407656b0 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d81ba8740 10 client.0 _ll_register_callbacks cb 0x7f8d68009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x5564407daa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d77400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556440760540 0x5564407dcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.919+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x5564407daa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52236/0 (socket says 172.21.15.97:52236) 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d77e00640 1 -- 172.21.15.97:0/2817370407 learned_addr learned my addr 172.21.15.97:0/2817370407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d7ce00640 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564407656b0 0x5564407df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d77e00640 1 -- 172.21.15.97:0/2817370407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564407656b0 msgr2=0x5564407df4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d77e00640 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564407656b0 0x5564407df4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d77e00640 1 -- 172.21.15.97:0/2817370407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556440760540 msgr2=0x5564407dcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:11.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d77e00640 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556440760540 0x5564407dcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d77e00640 1 -- 172.21.15.97:0/2817370407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564407dfcd0 con 0x55644075fb50 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d7ce00640 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564407656b0 0x5564407df4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d77e00640 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x5564407daa50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8d68013010 tx=0x7f8d6801fe30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a77704a04fe1941 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d62000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.920+0000 7f8d62000640 1 -- 172.21.15.97:0/2817370407 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68005220 con 0x55644075fb50 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.921+0000 7f8d62000640 1 -- 172.21.15.97:0/2817370407 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8d680053c0 con 0x55644075fb50 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.921+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2817370407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d64004040 con 0x55644075fb50 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.921+0000 7f8d62000640 1 -- 172.21.15.97:0/2817370407 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d68003e50 con 0x55644075fb50 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.921+0000 7f8d81ba8740 10 client.24995 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.921+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2817370407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5564407e1040 con 0x55644075fb50 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.921+0000 7f8d62000640 1 -- 172.21.15.97:0/2817370407 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d64004040 con 0x55644075fb50 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.921+0000 7f8d62000640 1 -- 172.21.15.97:0/2817370407 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8d68027040 con 0x55644075fb50 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.921+0000 7f8d81ba8740 10 client.24995 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.921+0000 7f8d81ba8740 10 client.24995 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.921+0000 7f8d81ba8740 20 client.24995 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.922+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5564407e5c90 0x5564407e20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.922+0000 7f8d81ba8740 4 client.24995 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.922+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2817370407 --> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] -- command(tid 0: {"prefix": "status"}) -- 0x5564406cba40 con 0x5564407e5c90 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.922+0000 7f8d77400640 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5564407e5c90 0x5564407e20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.923+0000 7f8d77400640 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5564407e5c90 0x5564407e20d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46dd205d237c412b server_cookie=d76e443432ca0225 in_seq=0 out_seq=0 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.923+0000 7f8d62000640 10 client.24995 ms_handle_connect on v2:172.21.15.97:6836/4124660285 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.923+0000 7f8d62000640 1 -- 172.21.15.97:0/2817370407 <== mds.4 v2:172.21.15.97:6836/4124660285 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5564406cba40 con 0x5564407e5c90 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.923+0000 7f8d62000640 10 client.24995 handle_command_reply: tid=0 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.923+0000 7f8d81ba8740 1 client.24995 shutdown 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.923+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2817370407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 msgr2=0x5564407daa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.923+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x5564407daa50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8d68013010 tx=0x7f8d6801fe30 comp rx=0 tx=0).stop 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.924+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2817370407 shutdown_connections 2024-03-26T18:58:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.924+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5564407e5c90 0x5564407e20d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.924+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564407656b0 0x5564407df4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.924+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556440760540 0x5564407dcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.924+0000 7f8d62000640 0 client.24995 ms_handle_reset on v2:172.21.15.97:6836/4124660285 2024-03-26T18:58:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.924+0000 7f8d81ba8740 1 --2- 172.21.15.97:0/2817370407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644075fb50 0x5564407daa50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.924+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2817370407 >> 172.21.15.97:0/2817370407 conn(0x55644075b0c0 msgr2=0x556440761aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.924+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2817370407 shutdown_connections 2024-03-26T18:58:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.924+0000 7f8d81ba8740 1 -- 172.21.15.97:0/2817370407 wait complete. 2024-03-26T18:58:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.924+0000 7f8d81ba8740 20 client.24995 trim_cache size 0 max 16384 2024-03-26T18:58:11.924 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:58:11.925 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:58:11.925 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 4, 2024-03-26T18:58:11.925 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24481, 2024-03-26T18:58:11.925 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-26T18:58:11.925 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-26T18:58:11.925 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-26T18:58:11.925 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 101.099187603, 2024-03-26T18:58:11.925 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-03-26T18:58:11.925 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2024-03-26T18:58:11.927 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T18:58:11.927 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 139.94539690100001 2024-03-26T18:58:11.927 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 -- 172.21.15.97:0/3801104303 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8d5c0a2fa0 msgr2=0x7f8d5c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8d5c0a2fa0 0x7f8d5c0a5460 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x556440760940 tx=0x7f8d68005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 -- 172.21.15.97:0/3801104303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d781074e0 msgr2=0x7f8d7819f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d781074e0 0x7f8d7819f070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009fd0 tx=0x7f8d7000ef10 comp rx=0 tx=0).stop 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 -- 172.21.15.97:0/3801104303 shutdown_connections 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8d5c0a2fa0 0x7f8d5c0a5460 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d781074e0 0x7f8d7819f070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d78104bb0 0x7f8d78198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 --2- 172.21.15.97:0/3801104303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d78102280 0x7f8d78197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 -- 172.21.15.97:0/3801104303 >> 172.21.15.97:0/3801104303 conn(0x7f8d780fbc30 msgr2=0x7f8d78102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 -- 172.21.15.97:0/3801104303 shutdown_connections 2024-03-26T18:58:11.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:11.925+0000 7f8d7d800640 1 -- 172.21.15.97:0/3801104303 wait complete. 2024-03-26T18:58:11.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-26T18:58:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.142+0000 7ff9bb000640 1 -- 172.21.15.97:0/1761173972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9b4107640 msgr2=0x7ff9b4109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.142+0000 7ff9bb000640 1 --2- 172.21.15.97:0/1761173972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9b4107640 0x7ff9b4109a30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009ca0 tx=0x7ff9ac02c100 comp rx=0 tx=0).stop 2024-03-26T18:58:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.142+0000 7ff9bb000640 1 -- 172.21.15.97:0/1761173972 shutdown_connections 2024-03-26T18:58:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.142+0000 7ff9bb000640 1 --2- 172.21.15.97:0/1761173972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9b4107640 0x7ff9b4109a30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.142+0000 7ff9bb000640 1 --2- 172.21.15.97:0/1761173972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9b4104d10 0x7ff9b4107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.142+0000 7ff9bb000640 1 --2- 172.21.15.97:0/1761173972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9b41023e0 0x7ff9b41047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.142+0000 7ff9bb000640 1 -- 172.21.15.97:0/1761173972 >> 172.21.15.97:0/1761173972 conn(0x7ff9b40fbd30 msgr2=0x7ff9b40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.142+0000 7ff9bb000640 1 -- 172.21.15.97:0/1761173972 shutdown_connections 2024-03-26T18:58:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.142+0000 7ff9bb000640 1 -- 172.21.15.97:0/1761173972 wait complete. 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.143+0000 7ff9bb000640 1 Processor -- start 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.143+0000 7ff9bb000640 1 -- start start 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9bb000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9b41023e0 0x7ff9b4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9bb000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9b4104d10 0x7ff9b4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9bb000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9b4107640 0x7ff9b419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9bb000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b410d540 con 0x7ff9b41023e0 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9bb000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b410d3c0 con 0x7ff9b4104d10 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9bb000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b410d6c0 con 0x7ff9b4107640 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9b41023e0 0x7ff9b4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9b3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9b4104d10 0x7ff9b4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9b3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9b4104d10 0x7ff9b4198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52256/0 (socket says 172.21.15.97:52256) 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9b3400640 1 -- 172.21.15.97:0/4291774439 learned_addr learned my addr 172.21.15.97:0/4291774439 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9b9000640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9b4107640 0x7ff9b419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9b3400640 1 -- 172.21.15.97:0/4291774439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9b4107640 msgr2=0x7ff9b419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9b3400640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9b4107640 0x7ff9b419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9b3400640 1 -- 172.21.15.97:0/4291774439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9b41023e0 msgr2=0x7ff9b4197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9b3400640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9b41023e0 0x7ff9b4197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.144+0000 7ff9b3400640 1 -- 172.21.15.97:0/4291774439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b419f7e0 con 0x7ff9b4104d10 2024-03-26T18:58:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.145+0000 7ff9b3400640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9b4104d10 0x7ff9b4198100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000e9c0 tx=0x7ff9a000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.145+0000 7ff9b0c00640 1 -- 172.21.15.97:0/4291774439 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000cd80 con 0x7ff9b4104d10 2024-03-26T18:58:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.145+0000 7ff9b0c00640 1 -- 172.21.15.97:0/4291774439 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a0005500 con 0x7ff9b4104d10 2024-03-26T18:58:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.145+0000 7ff9bb000640 1 -- 172.21.15.97:0/4291774439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b419fad0 con 0x7ff9b4104d10 2024-03-26T18:58:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.145+0000 7ff9b0c00640 1 -- 172.21.15.97:0/4291774439 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0010690 con 0x7ff9b4104d10 2024-03-26T18:58:12.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.145+0000 7ff9bb000640 1 -- 172.21.15.97:0/4291774439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b41a0070 con 0x7ff9b4104d10 2024-03-26T18:58:12.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.148+0000 7ff9bdc05740 1 Processor -- start 2024-03-26T18:58:12.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.148+0000 7ff9bdc05740 1 -- start start 2024-03-26T18:58:12.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9b0c00640 1 -- 172.21.15.97:0/4291774439 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7ff9a00046c0 con 0x7ff9b4104d10 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 0x5609612a9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612abd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5609612041b0 con 0x5609612a98e0 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5609611acb00 con 0x5609612a4770 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560961200160 con 0x5609612a3d80 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9b9000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9b9000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612a41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37312/0 (socket says 172.21.15.97:37312) 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9b9000640 1 -- 172.21.15.97:0/1103317378 learned_addr learned my addr 172.21.15.97:0/1103317378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/1103317378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 0x5609612a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9b3400640 1 --2- 172.21.15.97:0/1103317378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9b9000640 1 -- 172.21.15.97:0/1103317378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 msgr2=0x5609612a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9b9000640 1 --2- 172.21.15.97:0/1103317378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 0x5609612a9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9b9000640 1 -- 172.21.15.97:0/1103317378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 msgr2=0x5609612abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.149+0000 7ff9b9000640 1 --2- 172.21.15.97:0/1103317378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612abd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.150+0000 7ff9b9000640 1 -- 172.21.15.97:0/1103317378 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560960f182d0 con 0x5609612a3d80 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.150+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/1103317378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 0x5609612a9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.150+0000 7ff9b9000640 1 --2- 172.21.15.97:0/1103317378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612a41a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac02c710 tx=0x7ff9ac0323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e99ea5e1b6c7bdc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.150+0000 7ff9a9600640 1 -- 172.21.15.97:0/1103317378 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac043070 con 0x5609612a3d80 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.150+0000 7ff9a9600640 1 -- 172.21.15.97:0/1103317378 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9ac032c90 con 0x5609612a3d80 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.150+0000 7ff9a9600640 1 -- 172.21.15.97:0/1103317378 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac039040 con 0x5609612a3d80 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.150+0000 7ff9b0c00640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff9980a2fa0 0x7ff9980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9b0c00640 1 -- 172.21.15.97:0/4291774439 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff9a00cf6c0 con 0x7ff9b4104d10 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff9980a2fa0 0x7ff9980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1103317378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 msgr2=0x5609612a41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/1103317378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612a41a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac02c710 tx=0x7ff9ac0323d0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1103317378 shutdown_connections 2024-03-26T18:58:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/1103317378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/1103317378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 0x5609612a9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/1103317378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612a41a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1103317378 >> 172.21.15.97:0/1103317378 conn(0x56096129aee0 msgr2=0x56096129b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1103317378 shutdown_connections 2024-03-26T18:58:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.151+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1103317378 wait complete. 2024-03-26T18:58:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.152+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff9980a2fa0 0x7ff9980a5460 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x56096129be50 tx=0x7ff9a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.152+0000 7ff9bdc05740 1 -- start start 2024-03-26T18:58:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.152+0000 7ff9bdc05740 1 Processor -- start 2024-03-26T18:58:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a3d80 0x5609612ae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a4770 0x5609612ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5609612ac830 con 0x5609612a98e0 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5609611bfed0 con 0x5609612a3d80 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609612ac9b0 con 0x5609612a4770 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9bdc05740 10 client.0 _ll_register_callbacks cb 0x5609611fb710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a3d80 0x5609612ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a3d80 0x5609612ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52266/0 (socket says 172.21.15.97:52266) 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9b3e00640 1 -- 172.21.15.97:0/296837380 learned_addr learned my addr 172.21.15.97:0/296837380 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9b3400640 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a4770 0x5609612ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9b9000640 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9b3400640 1 -- 172.21.15.97:0/296837380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a3d80 msgr2=0x5609612ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9b3400640 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a3d80 0x5609612ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9b3400640 1 -- 172.21.15.97:0/296837380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 msgr2=0x5609612b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9b3400640 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.153+0000 7ff9b3400640 1 -- 172.21.15.97:0/296837380 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560960fe26b0 con 0x5609612a4770 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.154+0000 7ff9b3400640 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a4770 0x5609612ae7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c6a0 tx=0x7ff9a00d3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41ac717e39ed9262 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.154+0000 7ff99e000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.154+0000 7ff99e000640 1 -- 172.21.15.97:0/296837380 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a009bb90 con 0x5609612a4770 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.154+0000 7ff99e000640 1 -- 172.21.15.97:0/296837380 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff9a009c220 con 0x5609612a4770 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.154+0000 7ff9bdc05740 1 -- 172.21.15.97:0/296837380 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9ac032c90 con 0x5609612a4770 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.154+0000 7ff99e000640 1 -- 172.21.15.97:0/296837380 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9a009b720 con 0x5609612a4770 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.154+0000 7ff9bdc05740 10 client.24991 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.154+0000 7ff9bdc05740 1 -- 172.21.15.97:0/296837380 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609612b88e0 con 0x5609612a4770 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.154+0000 7ff99e000640 1 -- 172.21.15.97:0/296837380 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9ac032c90 con 0x5609612a4770 2024-03-26T18:58:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.155+0000 7ff99e000640 1 -- 172.21.15.97:0/296837380 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff9a00a4340 con 0x5609612a4770 2024-03-26T18:58:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.155+0000 7ff9bdc05740 10 client.24991 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.155+0000 7ff9bdc05740 10 client.24991 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T18:58:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.155+0000 7ff9bdc05740 20 client.24991 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.155+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x5609612baf80 0x5609612bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.156+0000 7ff9bdc05740 4 client.24991 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:58:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.156+0000 7ff9bdc05740 1 -- 172.21.15.97:0/296837380 --> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609612116f0 con 0x5609612baf80 2024-03-26T18:58:12.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.156+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x5609612baf80 0x5609612bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.158+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x5609612baf80 0x5609612bc3f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5e40db646f053fd4 server_cookie=f1c4fb8cafad663e in_seq=0 out_seq=0 2024-03-26T18:58:12.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.158+0000 7ff99e000640 10 client.24991 ms_handle_connect on v2:172.21.15.105:6834/1538692215 2024-03-26T18:58:12.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.158+0000 7ff99e000640 1 -- 172.21.15.97:0/296837380 <== mds.3 v2:172.21.15.105:6834/1538692215 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5609612116f0 con 0x5609612baf80 2024-03-26T18:58:12.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.158+0000 7ff99e000640 10 client.24991 handle_command_reply: tid=0 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 client.24991 shutdown 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 -- 172.21.15.97:0/296837380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a4770 msgr2=0x5609612ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a4770 0x5609612ae7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c6a0 tx=0x7ff9a00d3040 comp rx=0 tx=0).stop 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 -- 172.21.15.97:0/296837380 shutdown_connections 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x5609612baf80 0x5609612bc3f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a4770 0x5609612ae7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff99e000640 0 client.24991 ms_handle_reset on v2:172.21.15.105:6834/1538692215 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/296837380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a3d80 0x5609612ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 -- 172.21.15.97:0/296837380 >> 172.21.15.97:0/296837380 conn(0x56096129aee0 msgr2=0x56096129a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 -- 172.21.15.97:0/296837380 shutdown_connections 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 1 -- 172.21.15.97:0/296837380 wait complete. 2024-03-26T18:58:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.159+0000 7ff9bdc05740 20 client.24991 trim_cache size 0 max 16384 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9bdc05740 1 Processor -- start 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9bdc05740 1 -- start start 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x560961299790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a4770 0x5609612b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a98e0 0x560961299cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5609612ba730 con 0x5609612a4770 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5609612bae00 con 0x5609612a98e0 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609612bb4d0 con 0x5609612a3d80 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x560961299790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9b9000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a98e0 0x560961299cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x560961299790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37334/0 (socket says 172.21.15.97:37334) 2024-03-26T18:58:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9b3e00640 1 -- 172.21.15.97:0/3528785662 learned_addr learned my addr 172.21.15.97:0/3528785662 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.179+0000 7ff9b3400640 1 --2- 172.21.15.97:0/3528785662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a4770 0x5609612b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.180+0000 7ff9b3e00640 1 -- 172.21.15.97:0/3528785662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a98e0 msgr2=0x560961299cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.180+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/3528785662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a98e0 0x560961299cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.180+0000 7ff9b3e00640 1 -- 172.21.15.97:0/3528785662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a4770 msgr2=0x5609612b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.180+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/3528785662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a4770 0x5609612b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.180+0000 7ff9b3e00640 1 -- 172.21.15.97:0/3528785662 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609612a4b50 con 0x5609612a3d80 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.180+0000 7ff9b3400640 1 --2- 172.21.15.97:0/3528785662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a4770 0x5609612b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.180+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/3528785662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x560961299790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009fd0 tx=0x7ff9a401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c8aed32378b8228 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.180+0000 7ff9a8c00640 1 -- 172.21.15.97:0/3528785662 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4004490 con 0x5609612a3d80 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.180+0000 7ff9a8c00640 1 -- 172.21.15.97:0/3528785662 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a402bb90 con 0x5609612a3d80 2024-03-26T18:58:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.180+0000 7ff9a8c00640 1 -- 172.21.15.97:0/3528785662 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40055b0 con 0x5609612a3d80 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.181+0000 7ff9bdc05740 1 -- 172.21.15.97:0/3528785662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 msgr2=0x560961299790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.181+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/3528785662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x560961299790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009fd0 tx=0x7ff9a401b000 comp rx=0 tx=0).stop 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.181+0000 7ff9bdc05740 1 -- 172.21.15.97:0/3528785662 shutdown_connections 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.181+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/3528785662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a98e0 0x560961299cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.181+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/3528785662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a4770 0x5609612b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.181+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/3528785662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x560961299790 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.181+0000 7ff9bdc05740 1 -- 172.21.15.97:0/3528785662 >> 172.21.15.97:0/3528785662 conn(0x56096129aee0 msgr2=0x5609612a5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.181+0000 7ff9bdc05740 1 -- 172.21.15.97:0/3528785662 shutdown_connections 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.181+0000 7ff9bdc05740 1 -- 172.21.15.97:0/3528785662 wait complete. 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.182+0000 7ff9bdc05740 1 -- start start 2024-03-26T18:58:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.182+0000 7ff9bdc05740 1 Processor -- start 2024-03-26T18:58:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 0x5609612b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9bdc05740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5609612bc270 con 0x5609612a98e0 2024-03-26T18:58:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5609612bbba0 con 0x5609612a4770 2024-03-26T18:58:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9bdc05740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609612bc940 con 0x5609612a3d80 2024-03-26T18:58:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9bdc05740 10 client.0 _ll_register_callbacks cb 0x7ff9a0009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37340/0 (socket says 172.21.15.97:37340) 2024-03-26T18:58:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9b3e00640 1 -- 172.21.15.97:0/1228154865 learned_addr learned my addr 172.21.15.97:0/1228154865 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9b9000640 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9b3400640 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 0x5609612b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9b3e00640 1 -- 172.21.15.97:0/1228154865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 msgr2=0x5609612b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 0x5609612b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9b3e00640 1 -- 172.21.15.97:0/1228154865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 msgr2=0x5609612b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.183+0000 7ff9b3e00640 1 -- 172.21.15.97:0/1228154865 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609612b5210 con 0x5609612a3d80 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.184+0000 7ff9b3e00640 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612b16c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4003e50 tx=0x7ff9a4004490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d728f77a2894724 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.184+0000 7ff99e000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.184+0000 7ff99e000640 1 -- 172.21.15.97:0/1228154865 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4038020 con 0x5609612a3d80 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.184+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1228154865 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a402bb90 con 0x5609612a3d80 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.184+0000 7ff99e000640 1 -- 172.21.15.97:0/1228154865 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff9a4005530 con 0x5609612a3d80 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.184+0000 7ff99e000640 1 -- 172.21.15.97:0/1228154865 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9a402b470 con 0x5609612a3d80 2024-03-26T18:58:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.184+0000 7ff9bdc05740 10 client.24997 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.184+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1228154865 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609613214d0 con 0x5609612a3d80 2024-03-26T18:58:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.184+0000 7ff99e000640 1 -- 172.21.15.97:0/1228154865 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a402bb90 con 0x5609612a3d80 2024-03-26T18:58:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.185+0000 7ff99e000640 1 -- 172.21.15.97:0/1228154865 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff9a401f430 con 0x5609612a3d80 2024-03-26T18:58:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.185+0000 7ff9bdc05740 10 client.24997 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.185+0000 7ff9bdc05740 10 client.24997 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T18:58:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.185+0000 7ff9bdc05740 20 client.24997 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.185+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x560961325d50 0x560961324a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.185+0000 7ff9bdc05740 4 client.24997 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:58:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.185+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1228154865 --> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] -- command(tid 0: {"prefix": "status"}) -- 0x56096120ba40 con 0x560961325d50 2024-03-26T18:58:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.185+0000 7ff9b3400640 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x560961325d50 0x560961324a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.185+0000 7ff9b3400640 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x560961325d50 0x560961324a60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d2b7ce44f7bb92f server_cookie=150c018a3a42a8c7 in_seq=0 out_seq=0 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.185+0000 7ff99e000640 10 client.24997 ms_handle_connect on v2:172.21.15.105:6834/1538692215 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff99e000640 1 -- 172.21.15.97:0/1228154865 <== mds.3 v2:172.21.15.105:6834/1538692215 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56096120ba40 con 0x560961325d50 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff99e000640 10 client.24997 handle_command_reply: tid=0 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 client.24997 shutdown 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1228154865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 msgr2=0x5609612b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612b16c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4003e50 tx=0x7ff9a4004490 comp rx=0 tx=0).stop 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1228154865 shutdown_connections 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x560961325d50 0x560961324a60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5609612a98e0 0x5609612b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609612a4770 0x5609612b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff99e000640 0 client.24997 ms_handle_reset on v2:172.21.15.105:6834/1538692215 2024-03-26T18:58:12.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 --2- 172.21.15.97:0/1228154865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609612a3d80 0x5609612b16c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1228154865 >> 172.21.15.97:0/1228154865 conn(0x56096129aee0 msgr2=0x560961299790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1228154865 shutdown_connections 2024-03-26T18:58:12.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 1 -- 172.21.15.97:0/1228154865 wait complete. 2024-03-26T18:58:12.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.186+0000 7ff9bdc05740 20 client.24997 trim_cache size 0 max 16384 2024-03-26T18:58:12.185 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:58:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:58:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 3, 2024-03-26T18:58:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24494, 2024-03-26T18:58:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-26T18:58:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-26T18:58:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-26T18:58:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 102.369318095, 2024-03-26T18:58:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-03-26T18:58:12.186 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2024-03-26T18:58:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T18:58:12.187 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 137.185637636 2024-03-26T18:58:12.187 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:58:12.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 -- 172.21.15.97:0/4291774439 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff9980a2fa0 msgr2=0x7ff9980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff9980a2fa0 0x7ff9980a5460 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x56096129be50 tx=0x7ff9a4005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:12.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 -- 172.21.15.97:0/4291774439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9b4104d10 msgr2=0x7ff9b4198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9b4104d10 0x7ff9b4198100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000e9c0 tx=0x7ff9a000ee80 comp rx=0 tx=0).stop 2024-03-26T18:58:12.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 -- 172.21.15.97:0/4291774439 shutdown_connections 2024-03-26T18:58:12.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff9980a2fa0 0x7ff9980a5460 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9b4107640 0x7ff9b419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9b4104d10 0x7ff9b4198100 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 --2- 172.21.15.97:0/4291774439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9b41023e0 0x7ff9b4197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 -- 172.21.15.97:0/4291774439 >> 172.21.15.97:0/4291774439 conn(0x7ff9b40fbd30 msgr2=0x7ff9b4102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 -- 172.21.15.97:0/4291774439 shutdown_connections 2024-03-26T18:58:12.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.187+0000 7ff9bb000640 1 -- 172.21.15.97:0/4291774439 wait complete. 2024-03-26T18:58:12.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:12 smithi097 ceph-mon[31955]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T18:58:12.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:12 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1157467679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:12.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:12 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1172685497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:12.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-26T18:58:12.277 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:12 smithi144 ceph-mon[36053]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T18:58:12.277 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:12 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1157467679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:12.277 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:12 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1172685497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.394+0000 7fcbb6800640 1 -- 172.21.15.97:0/870023352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbb01018d0 msgr2=0x7fcbb0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.394+0000 7fcbb6800640 1 --2- 172.21.15.97:0/870023352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbb01018d0 0x7fcbb0101cf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009900 tx=0x7fcba002bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.394+0000 7fcbb6800640 1 -- 172.21.15.97:0/870023352 shutdown_connections 2024-03-26T18:58:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.394+0000 7fcbb6800640 1 --2- 172.21.15.97:0/870023352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb0103480 0x7fcbb0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.394+0000 7fcbb6800640 1 --2- 172.21.15.97:0/870023352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb0102ac0 0x7fcbb0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.394+0000 7fcbb6800640 1 --2- 172.21.15.97:0/870023352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbb01018d0 0x7fcbb0101cf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.394+0000 7fcbb6800640 1 -- 172.21.15.97:0/870023352 >> 172.21.15.97:0/870023352 conn(0x7fcbb00fce40 msgr2=0x7fcbb00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.394+0000 7fcbb6800640 1 -- 172.21.15.97:0/870023352 shutdown_connections 2024-03-26T18:58:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.394+0000 7fcbb6800640 1 -- 172.21.15.97:0/870023352 wait complete. 2024-03-26T18:58:12.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.395+0000 7fcbb6800640 1 Processor -- start 2024-03-26T18:58:12.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbb6800640 1 -- start start 2024-03-26T18:58:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbb6800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbb01018d0 0x7fcbb0199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbb6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb0102ac0 0x7fcbb019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbb6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb0103480 0x7fcbb01a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbb6800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010d5a0 con 0x7fcbb0102ac0 2024-03-26T18:58:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbb6800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010d420 con 0x7fcbb01018d0 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbb6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010d720 con 0x7fcbb0103480 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbb01018d0 0x7fcbb0199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbb01018d0 0x7fcbb0199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52298/0 (socket says 172.21.15.97:52298) 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbaf400640 1 -- 172.21.15.97:0/580623635 learned_addr learned my addr 172.21.15.97:0/580623635 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbafe00640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb0103480 0x7fcbb01a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.396+0000 7fcbaea00640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb0102ac0 0x7fcbb019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcbaf400640 1 -- 172.21.15.97:0/580623635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb0103480 msgr2=0x7fcbb01a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcbaf400640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb0103480 0x7fcbb01a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcbaf400640 1 -- 172.21.15.97:0/580623635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb0102ac0 msgr2=0x7fcbb019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcbaf400640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb0102ac0 0x7fcbb019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcbaf400640 1 -- 172.21.15.97:0/580623635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb01a1a50 con 0x7fcbb01018d0 2024-03-26T18:58:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcbafe00640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb0103480 0x7fcbb01a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcbaf400640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbb01018d0 0x7fcbb0199e30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcba0000c00 tx=0x7fcba002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcba7e00640 1 -- 172.21.15.97:0/580623635 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0043020 con 0x7fcbb01018d0 2024-03-26T18:58:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcba7e00640 1 -- 172.21.15.97:0/580623635 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba00054e0 con 0x7fcbb01018d0 2024-03-26T18:58:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcba7e00640 1 -- 172.21.15.97:0/580623635 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0003c40 con 0x7fcbb01018d0 2024-03-26T18:58:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcbb6800640 1 -- 172.21.15.97:0/580623635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb01a1ce0 con 0x7fcbb01018d0 2024-03-26T18:58:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.397+0000 7fcbb6800640 1 -- 172.21.15.97:0/580623635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a21d0 con 0x7fcbb01018d0 2024-03-26T18:58:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.400+0000 7fcba7e00640 1 -- 172.21.15.97:0/580623635 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fcba0046020 con 0x7fcbb01018d0 2024-03-26T18:58:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.401+0000 7fcba7e00640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb940a2fa0 0x7fcb940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.402+0000 7fcba7e00640 1 -- 172.21.15.97:0/580623635 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcba00f1390 con 0x7fcbb01018d0 2024-03-26T18:58:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.402+0000 7fcbaea00640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb940a2fa0 0x7fcb940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.402+0000 7fcbb956f740 1 Processor -- start 2024-03-26T18:58:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.402+0000 7fcbb956f740 1 -- start start 2024-03-26T18:58:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.402+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.402+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ae360 0x558c811b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.402+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 0x558c811b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.402+0000 7fcbb956f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558c811121b0 con 0x558c811ad970 2024-03-26T18:58:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.402+0000 7fcbb956f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c810bab00 con 0x558c811b34d0 2024-03-26T18:58:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.402+0000 7fcbb956f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c8110e160 con 0x558c811ae360 2024-03-26T18:58:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaea00640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb940a2fa0 0x7fcb940a5460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcbb019b340 tx=0x7fcb9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ae360 0x558c811b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 0x558c811b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ae360 0x558c811b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37378/0 (socket says 172.21.15.97:37378) 2024-03-26T18:58:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59274/0 (socket says 172.21.15.97:59274) 2024-03-26T18:58:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaea00640 1 -- 172.21.15.97:0/1178961006 learned_addr learned my addr 172.21.15.97:0/1178961006 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaea00640 1 -- 172.21.15.97:0/1178961006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 msgr2=0x558c811b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaea00640 1 --2- 172.21.15.97:0/1178961006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 0x558c811b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaea00640 1 -- 172.21.15.97:0/1178961006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 msgr2=0x558c811add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaea00640 1 --2- 172.21.15.97:0/1178961006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaea00640 1 -- 172.21.15.97:0/1178961006 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c80e262d0 con 0x558c811ae360 2024-03-26T18:58:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.403+0000 7fcbaea00640 1 --2- 172.21.15.97:0/1178961006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ae360 0x558c811b2e60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005bc0 tx=0x7fcb9c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=563cb9c4009e5db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.404+0000 7fcba4c00640 1 -- 172.21.15.97:0/1178961006 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c005020 con 0x558c811ae360 2024-03-26T18:58:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.404+0000 7fcba4c00640 1 -- 172.21.15.97:0/1178961006 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c0047a0 con 0x558c811ae360 2024-03-26T18:58:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.404+0000 7fcba4c00640 1 -- 172.21.15.97:0/1178961006 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c026820 con 0x558c811ae360 2024-03-26T18:58:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.404+0000 7fcbb956f740 1 -- 172.21.15.97:0/1178961006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ae360 msgr2=0x558c811b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.404+0000 7fcbb956f740 1 --2- 172.21.15.97:0/1178961006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ae360 0x558c811b2e60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005bc0 tx=0x7fcb9c014000 comp rx=0 tx=0).stop 2024-03-26T18:58:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.405+0000 7fcbb956f740 1 -- 172.21.15.97:0/1178961006 shutdown_connections 2024-03-26T18:58:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.405+0000 7fcbb956f740 1 --2- 172.21.15.97:0/1178961006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 0x558c811b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.405+0000 7fcbb956f740 1 --2- 172.21.15.97:0/1178961006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ae360 0x558c811b2e60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.405+0000 7fcbb956f740 1 --2- 172.21.15.97:0/1178961006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.405+0000 7fcbb956f740 1 -- 172.21.15.97:0/1178961006 >> 172.21.15.97:0/1178961006 conn(0x558c811a8ee0 msgr2=0x558c811a92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.405+0000 7fcbb956f740 1 -- 172.21.15.97:0/1178961006 shutdown_connections 2024-03-26T18:58:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.405+0000 7fcbb956f740 1 -- 172.21.15.97:0/1178961006 wait complete. 2024-03-26T18:58:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.405+0000 7fcbb956f740 1 -- start start 2024-03-26T18:58:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.405+0000 7fcbb956f740 1 Processor -- start 2024-03-26T18:58:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 0x558c811bf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbb956f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558c811b6480 con 0x558c811ad970 2024-03-26T18:58:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbb956f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c810cded0 con 0x558c811ae360 2024-03-26T18:58:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbb956f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c811b6600 con 0x558c811b34d0 2024-03-26T18:58:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbb956f740 10 client.0 _ll_register_callbacks cb 0x558c81109710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbaea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbaea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52316/0 (socket says 172.21.15.97:52316) 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbaea00640 1 -- 172.21.15.97:0/4232993743 learned_addr learned my addr 172.21.15.97:0/4232993743 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.406+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 0x558c811bf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcbafe00640 1 -- 172.21.15.97:0/4232993743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 msgr2=0x558c811b83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcbafe00640 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcbafe00640 1 -- 172.21.15.97:0/4232993743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 msgr2=0x558c811b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcbafe00640 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcbafe00640 1 -- 172.21.15.97:0/4232993743 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c80ef06b0 con 0x558c811b34d0 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcbaea00640 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcbaf400640 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcbafe00640 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 0x558c811bf3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009fd0 tx=0x7fcba8012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a85012a1a0d2dc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcb99600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcb99600640 1 -- 172.21.15.97:0/4232993743 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8003970 con 0x558c811b34d0 2024-03-26T18:58:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcb99600640 1 -- 172.21.15.97:0/4232993743 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcba8003b10 con 0x558c811b34d0 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.407+0000 7fcbb956f740 1 -- 172.21.15.97:0/4232993743 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb9c0047a0 con 0x558c811b34d0 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcb99600640 1 -- 172.21.15.97:0/4232993743 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcba8023070 con 0x558c811b34d0 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcbb956f740 10 client.25009 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcbb956f740 1 -- 172.21.15.97:0/4232993743 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c811c24f0 con 0x558c811b34d0 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcb99600640 1 -- 172.21.15.97:0/4232993743 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c0047a0 con 0x558c811b34d0 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcb99600640 1 -- 172.21.15.97:0/4232993743 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcba80105c0 con 0x558c811b34d0 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcbb956f740 10 client.25009 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcbb956f740 10 client.25009 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcbb956f740 20 client.25009 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcbb956f740 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x558c811c4bc0 0x558c811c6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcbb956f740 4 client.25009 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.408+0000 7fcbb956f740 1 -- 172.21.15.97:0/4232993743 --> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c8111f6f0 con 0x558c811c4bc0 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.409+0000 7fcbaf400640 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x558c811c4bc0 0x558c811c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.409+0000 7fcbaf400640 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x558c811c4bc0 0x558c811c6030 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a343c650e015396d server_cookie=478556c5e7f3b732 in_seq=0 out_seq=0 2024-03-26T18:58:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.409+0000 7fcb99600640 10 client.25009 ms_handle_connect on v2:172.21.15.144:6838/994192292 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcb99600640 1 -- 172.21.15.97:0/4232993743 <== mds.2 v2:172.21.15.144:6838/994192292 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558c8111f6f0 con 0x558c811c4bc0 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcb99600640 10 client.25009 handle_command_reply: tid=0 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 client.25009 shutdown 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 -- 172.21.15.97:0/4232993743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 msgr2=0x558c811bf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 0x558c811bf3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009fd0 tx=0x7fcba8012820 comp rx=0 tx=0).stop 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 -- 172.21.15.97:0/4232993743 shutdown_connections 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x558c811c4bc0 0x558c811c6030 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 0x558c811bf3f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcb99600640 0 client.25009 ms_handle_reset on v2:172.21.15.144:6838/994192292 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 --2- 172.21.15.97:0/4232993743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 -- 172.21.15.97:0/4232993743 >> 172.21.15.97:0/4232993743 conn(0x558c811a8ee0 msgr2=0x558c811a8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 -- 172.21.15.97:0/4232993743 shutdown_connections 2024-03-26T18:58:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 1 -- 172.21.15.97:0/4232993743 wait complete. 2024-03-26T18:58:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.410+0000 7fcbb956f740 20 client.25009 trim_cache size 0 max 16384 2024-03-26T18:58:12.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.430+0000 7fcbb956f740 1 Processor -- start 2024-03-26T18:58:12.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.430+0000 7fcbb956f740 1 -- start start 2024-03-26T18:58:12.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ad970 0x558c811aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ae360 0x558c811bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 0x558c811a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbb956f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558c811c4370 con 0x558c811ae360 2024-03-26T18:58:12.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbb956f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c811c4a40 con 0x558c811b34d0 2024-03-26T18:58:12.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbb956f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c811c5110 con 0x558c811ad970 2024-03-26T18:58:12.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ad970 0x558c811aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 0x558c811a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbaea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ae360 0x558c811bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ad970 0x558c811aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37388/0 (socket says 172.21.15.97:37388) 2024-03-26T18:58:12.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbaf400640 1 -- 172.21.15.97:0/1588788770 learned_addr learned my addr 172.21.15.97:0/1588788770 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbafe00640 1 --2- 172.21.15.97:0/1588788770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 0x558c811a92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52328/0 (socket says 172.21.15.97:52328) 2024-03-26T18:58:12.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbaf400640 1 -- 172.21.15.97:0/1588788770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 msgr2=0x558c811a92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbaf400640 1 --2- 172.21.15.97:0/1588788770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 0x558c811a92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbaf400640 1 -- 172.21.15.97:0/1588788770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ae360 msgr2=0x558c811bf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbaf400640 1 --2- 172.21.15.97:0/1588788770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ae360 0x558c811bf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbaf400640 1 -- 172.21.15.97:0/1588788770 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c811c2310 con 0x558c811ad970 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.431+0000 7fcbafe00640 1 --2- 172.21.15.97:0/1588788770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 0x558c811a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.432+0000 7fcbaf400640 1 --2- 172.21.15.97:0/1588788770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ad970 0x558c811aed60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcba00ba410 tx=0x7fcba00eed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b8301fe5dea442 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.432+0000 7fcb9be00640 1 -- 172.21.15.97:0/1588788770 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0004c60 con 0x558c811ad970 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.432+0000 7fcb9be00640 1 -- 172.21.15.97:0/1588788770 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba00be190 con 0x558c811ad970 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.432+0000 7fcb9be00640 1 -- 172.21.15.97:0/1588788770 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00bbd90 con 0x558c811ad970 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.433+0000 7fcbb956f740 1 -- 172.21.15.97:0/1588788770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ad970 msgr2=0x558c811aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.433+0000 7fcbb956f740 1 --2- 172.21.15.97:0/1588788770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ad970 0x558c811aed60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcba00ba410 tx=0x7fcba00eed30 comp rx=0 tx=0).stop 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.433+0000 7fcbb956f740 1 -- 172.21.15.97:0/1588788770 shutdown_connections 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.433+0000 7fcbb956f740 1 --2- 172.21.15.97:0/1588788770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811b34d0 0x558c811a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.433+0000 7fcbb956f740 1 --2- 172.21.15.97:0/1588788770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ae360 0x558c811bf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.433+0000 7fcbb956f740 1 --2- 172.21.15.97:0/1588788770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811ad970 0x558c811aed60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.433+0000 7fcbb956f740 1 -- 172.21.15.97:0/1588788770 >> 172.21.15.97:0/1588788770 conn(0x558c811a8ee0 msgr2=0x558c811a3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.433+0000 7fcbb956f740 1 -- 172.21.15.97:0/1588788770 shutdown_connections 2024-03-26T18:58:12.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.433+0000 7fcbb956f740 1 -- 172.21.15.97:0/1588788770 wait complete. 2024-03-26T18:58:12.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.433+0000 7fcbb956f740 1 -- start start 2024-03-26T18:58:12.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbb956f740 1 Processor -- start 2024-03-26T18:58:12.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbb956f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 0x558c81228920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbb956f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558c811c5eb0 con 0x558c811ad970 2024-03-26T18:58:12.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbb956f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c811c57e0 con 0x558c811ae360 2024-03-26T18:58:12.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbb956f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c811c6580 con 0x558c811b34d0 2024-03-26T18:58:12.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbb956f740 10 client.0 _ll_register_callbacks cb 0x7fcba800f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbaea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 0x558c81228920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbaea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52338/0 (socket says 172.21.15.97:52338) 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.434+0000 7fcbaea00640 1 -- 172.21.15.97:0/204027400 learned_addr learned my addr 172.21.15.97:0/204027400 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcbaea00640 1 -- 172.21.15.97:0/204027400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 msgr2=0x558c81228920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcbaea00640 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 0x558c81228920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcbaea00640 1 -- 172.21.15.97:0/204027400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 msgr2=0x558c811b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcbaea00640 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcbaea00640 1 -- 172.21.15.97:0/204027400 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c81228ee0 con 0x558c811ae360 2024-03-26T18:58:12.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcbaea00640 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b1d30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009a90 tx=0x7fcb9c0055a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12671a01941c9d76 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcb99600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:58:12.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcb99600640 1 -- 172.21.15.97:0/204027400 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0037f0 con 0x558c811ae360 2024-03-26T18:58:12.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcb99600640 1 -- 172.21.15.97:0/204027400 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcb9c003990 con 0x558c811ae360 2024-03-26T18:58:12.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcbb956f740 1 -- 172.21.15.97:0/204027400 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba00be190 con 0x558c811ae360 2024-03-26T18:58:12.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcb99600640 1 -- 172.21.15.97:0/204027400 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb9c038070 con 0x558c811ae360 2024-03-26T18:58:12.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcbb956f740 10 client.25031 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:12.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.435+0000 7fcbb956f740 1 -- 172.21.15.97:0/204027400 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c8122a190 con 0x558c811ae360 2024-03-26T18:58:12.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.436+0000 7fcb99600640 1 -- 172.21.15.97:0/204027400 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba00be190 con 0x558c811ae360 2024-03-26T18:58:12.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.436+0000 7fcb99600640 1 -- 172.21.15.97:0/204027400 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcb9c026b90 con 0x558c811ae360 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.436+0000 7fcbb956f740 10 client.25031 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.436+0000 7fcbb956f740 10 client.25031 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.436+0000 7fcbb956f740 20 client.25031 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.436+0000 7fcbb956f740 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x558c8122e990 0x558c8122d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.436+0000 7fcbb956f740 4 client.25031 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.436+0000 7fcbb956f740 1 -- 172.21.15.97:0/204027400 --> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] -- command(tid 0: {"prefix": "status"}) -- 0x558c81119a40 con 0x558c8122e990 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.436+0000 7fcbaf400640 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x558c8122e990 0x558c8122d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbaf400640 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x558c8122e990 0x558c8122d6a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d0de3c901961fbe server_cookie=b55791cf3af01baf in_seq=0 out_seq=0 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcb99600640 10 client.25031 ms_handle_connect on v2:172.21.15.144:6838/994192292 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcb99600640 1 -- 172.21.15.97:0/204027400 <== mds.2 v2:172.21.15.144:6838/994192292 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558c81119a40 con 0x558c8122e990 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcb99600640 10 client.25031 handle_command_reply: tid=0 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 client.25031 shutdown 2024-03-26T18:58:12.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 -- 172.21.15.97:0/204027400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 msgr2=0x558c811b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b1d30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009a90 tx=0x7fcb9c0055a0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 -- 172.21.15.97:0/204027400 shutdown_connections 2024-03-26T18:58:12.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x558c8122e990 0x558c8122d6a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c811b34d0 0x558c81228920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c811ae360 0x558c811b1d30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcb99600640 0 client.25031 ms_handle_reset on v2:172.21.15.144:6838/994192292 2024-03-26T18:58:12.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 --2- 172.21.15.97:0/204027400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c811ad970 0x558c811b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 -- 172.21.15.97:0/204027400 >> 172.21.15.97:0/204027400 conn(0x558c811a8ee0 msgr2=0x558c811aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 -- 172.21.15.97:0/204027400 shutdown_connections 2024-03-26T18:58:12.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 1 -- 172.21.15.97:0/204027400 wait complete. 2024-03-26T18:58:12.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.437+0000 7fcbb956f740 20 client.25031 trim_cache size 0 max 16384 2024-03-26T18:58:12.437 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:58:12.437 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:58:12.437 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2024-03-26T18:58:12.437 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24535, 2024-03-26T18:58:12.437 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-26T18:58:12.438 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-26T18:58:12.438 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-26T18:58:12.438 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 103.587504885, 2024-03-26T18:58:12.438 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-03-26T18:58:12.438 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2024-03-26T18:58:12.438 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T18:58:12.438 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 125.93030772900001 2024-03-26T18:58:12.438 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:58:12.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 -- 172.21.15.97:0/580623635 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb940a2fa0 msgr2=0x7fcb940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb940a2fa0 0x7fcb940a5460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcbb019b340 tx=0x7fcb9c005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:12.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 -- 172.21.15.97:0/580623635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbb01018d0 msgr2=0x7fcbb0199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbb01018d0 0x7fcbb0199e30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcba0000c00 tx=0x7fcba002c650 comp rx=0 tx=0).stop 2024-03-26T18:58:12.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 -- 172.21.15.97:0/580623635 shutdown_connections 2024-03-26T18:58:12.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb940a2fa0 0x7fcb940a5460 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb0103480 0x7fcbb01a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb0102ac0 0x7fcbb019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 --2- 172.21.15.97:0/580623635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbb01018d0 0x7fcbb0199e30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 -- 172.21.15.97:0/580623635 >> 172.21.15.97:0/580623635 conn(0x7fcbb00fce40 msgr2=0x7fcbb0073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 -- 172.21.15.97:0/580623635 shutdown_connections 2024-03-26T18:58:12.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.438+0000 7fcbb6800640 1 -- 172.21.15.97:0/580623635 wait complete. 2024-03-26T18:58:12.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-26T18:58:12.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:12 smithi105 ceph-mon[36270]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T18:58:12.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:12 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1157467679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:12.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:12 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1172685497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:12.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.658+0000 7fd2b3200640 1 -- 172.21.15.97:0/3478754037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac102420 msgr2=0x7fd2ac104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.658+0000 7fd2b3200640 1 --2- 172.21.15.97:0/3478754037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac102420 0x7fd2ac104810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009900 tx=0x7fd29c02bc50 comp rx=0 tx=0).stop 2024-03-26T18:58:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.658+0000 7fd2b3200640 1 -- 172.21.15.97:0/3478754037 shutdown_connections 2024-03-26T18:58:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.659+0000 7fd2b3200640 1 --2- 172.21.15.97:0/3478754037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2ac107680 0x7fd2ac109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.659+0000 7fd2b3200640 1 --2- 172.21.15.97:0/3478754037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2ac104d50 0x7fd2ac107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.659+0000 7fd2b3200640 1 --2- 172.21.15.97:0/3478754037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac102420 0x7fd2ac104810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.659+0000 7fd2b3200640 1 -- 172.21.15.97:0/3478754037 >> 172.21.15.97:0/3478754037 conn(0x7fd2ac0fbd70 msgr2=0x7fd2ac0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.659+0000 7fd2b3200640 1 -- 172.21.15.97:0/3478754037 shutdown_connections 2024-03-26T18:58:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.659+0000 7fd2b3200640 1 -- 172.21.15.97:0/3478754037 wait complete. 2024-03-26T18:58:12.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2b3200640 1 Processor -- start 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2b3200640 1 -- start start 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2b3200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac102420 0x7fd2ac199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2b3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2ac104d50 0x7fd2ac19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2b3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2ac107680 0x7fd2ac1a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2b3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac10d5e0 con 0x7fd2ac104d50 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2b3200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac10d460 con 0x7fd2ac102420 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2b3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac10d760 con 0x7fd2ac107680 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac102420 0x7fd2ac199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac102420 0x7fd2ac199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52348/0 (socket says 172.21.15.97:52348) 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2abe00640 1 -- 172.21.15.97:0/925620456 learned_addr learned my addr 172.21.15.97:0/925620456 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.660+0000 7fd2ab400640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2ac104d50 0x7fd2ac19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2b1200640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2ac107680 0x7fd2ac1a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2abe00640 1 -- 172.21.15.97:0/925620456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2ac107680 msgr2=0x7fd2ac1a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2abe00640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2ac107680 0x7fd2ac1a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2abe00640 1 -- 172.21.15.97:0/925620456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2ac104d50 msgr2=0x7fd2ac19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2abe00640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2ac104d50 0x7fd2ac19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2abe00640 1 -- 172.21.15.97:0/925620456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac1a1a70 con 0x7fd2ac102420 2024-03-26T18:58:12.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2abe00640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac102420 0x7fd2ac199e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd29c02c260 tx=0x7fd29c02c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2a8c00640 1 -- 172.21.15.97:0/925620456 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c041070 con 0x7fd2ac102420 2024-03-26T18:58:12.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2a8c00640 1 -- 172.21.15.97:0/925620456 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c004450 con 0x7fd2ac102420 2024-03-26T18:58:12.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2a8c00640 1 -- 172.21.15.97:0/925620456 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c034440 con 0x7fd2ac102420 2024-03-26T18:58:12.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.661+0000 7fd2b3200640 1 -- 172.21.15.97:0/925620456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ac1a1d00 con 0x7fd2ac102420 2024-03-26T18:58:12.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.662+0000 7fd2b3200640 1 -- 172.21.15.97:0/925620456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ac1a21e0 con 0x7fd2ac102420 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b5e18740 1 Processor -- start 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b5e18740 1 -- start start 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d06d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d0be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d0c4d0 0x55bc76d0e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc76c6b1b0 con 0x55bc76d0c4d0 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bc76c13b00 con 0x55bc76d07360 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc76c67160 con 0x55bc76d06970 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2a8c00640 1 -- 172.21.15.97:0/925620456 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fd29c044070 con 0x7fd2ac102420 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d06d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d06d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37432/0 (socket says 172.21.15.97:37432) 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2b1200640 1 -- 172.21.15.97:0/836339625 learned_addr learned my addr 172.21.15.97:0/836339625 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.665+0000 7fd2ab400640 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d0c4d0 0x55bc76d0e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2abe00640 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d0be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2b1200640 1 -- 172.21.15.97:0/836339625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 msgr2=0x55bc76d0be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2b1200640 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d0be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2b1200640 1 -- 172.21.15.97:0/836339625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d0c4d0 msgr2=0x55bc76d0e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2b1200640 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d0c4d0 0x55bc76d0e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2b1200640 1 -- 172.21.15.97:0/836339625 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7697f2d0 con 0x55bc76d06970 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2ab400640 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d0c4d0 0x55bc76d0e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2abe00640 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d0be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2b1200640 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d06d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009fd0 tx=0x7fd2a40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c52324ddf822be7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2a1600640 1 -- 172.21.15.97:0/836339625 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4004d20 con 0x55bc76d06970 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2a1600640 1 -- 172.21.15.97:0/836339625 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2a4004050 con 0x55bc76d06970 2024-03-26T18:58:12.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.666+0000 7fd2a1600640 1 -- 172.21.15.97:0/836339625 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4005260 con 0x55bc76d06970 2024-03-26T18:58:12.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2b5e18740 1 -- 172.21.15.97:0/836339625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 msgr2=0x55bc76d06d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d06d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009fd0 tx=0x7fd2a40118d0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2b5e18740 1 -- 172.21.15.97:0/836339625 shutdown_connections 2024-03-26T18:58:12.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d0c4d0 0x55bc76d0e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d0be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/836339625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d06d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2b5e18740 1 -- 172.21.15.97:0/836339625 >> 172.21.15.97:0/836339625 conn(0x55bc76d01ee0 msgr2=0x55bc76d022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2b5e18740 1 -- 172.21.15.97:0/836339625 shutdown_connections 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2b5e18740 1 -- 172.21.15.97:0/836339625 wait complete. 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2a8c00640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2900a2eb0 0x7fd2900a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.667+0000 7fd2a8c00640 1 -- 172.21.15.97:0/925620456 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd29c0f0db0 con 0x7fd2ac102420 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.668+0000 7fd2ab400640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2900a2eb0 0x7fd2900a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.668+0000 7fd2b5e18740 1 -- start start 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.668+0000 7fd2b5e18740 1 Processor -- start 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.668+0000 7fd2ab400640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2900a2eb0 0x7fd2900a5370 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55bc76d02e50 tx=0x7fd298005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d10e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d07360 0x55bc76d11380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d183c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc76d0f480 con 0x55bc76d07360 2024-03-26T18:58:12.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bc76c26ed0 con 0x55bc76d0c4d0 2024-03-26T18:58:12.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc76d0f600 con 0x55bc76d06970 2024-03-26T18:58:12.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b5e18740 10 client.0 _ll_register_callbacks cb 0x55bc76c62710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:12.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d183c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d183c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52370/0 (socket says 172.21.15.97:52370) 2024-03-26T18:58:12.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b1200640 1 -- 172.21.15.97:0/502776946 learned_addr learned my addr 172.21.15.97:0/502776946 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2ab400640 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d07360 0x55bc76d11380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b1200640 1 -- 172.21.15.97:0/502776946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 msgr2=0x55bc76d10e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2abe00640 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d10e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b1200640 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d10e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b1200640 1 -- 172.21.15.97:0/502776946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d07360 msgr2=0x55bc76d11380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b1200640 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d07360 0x55bc76d11380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.669+0000 7fd2b1200640 1 -- 172.21.15.97:0/502776946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc76a496b0 con 0x55bc76d0c4d0 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.670+0000 7fd2abe00640 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d10e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.670+0000 7fd2b1200640 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d183c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009fa0 tx=0x7fd2a4004cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3de4daac003ba1f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.670+0000 7fd296000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.670+0000 7fd296000640 1 -- 172.21.15.97:0/502776946 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4028020 con 0x55bc76d0c4d0 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.670+0000 7fd296000640 1 -- 172.21.15.97:0/502776946 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd2a4004410 con 0x55bc76d0c4d0 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.670+0000 7fd2b5e18740 1 -- 172.21.15.97:0/502776946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2a4004050 con 0x55bc76d0c4d0 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.670+0000 7fd296000640 1 -- 172.21.15.97:0/502776946 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd2a4004650 con 0x55bc76d0c4d0 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.670+0000 7fd2b5e18740 10 client.25040 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.670+0000 7fd2b5e18740 1 -- 172.21.15.97:0/502776946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc76d1b360 con 0x55bc76d0c4d0 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.671+0000 7fd296000640 1 -- 172.21.15.97:0/502776946 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2a4004050 con 0x55bc76d0c4d0 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.671+0000 7fd296000640 1 -- 172.21.15.97:0/502776946 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd2a400f7e0 con 0x55bc76d0c4d0 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.671+0000 7fd2b5e18740 10 client.25040 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.671+0000 7fd2b5e18740 10 client.25040 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.671+0000 7fd2b5e18740 20 client.25040 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.671+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x55bc76d1d800 0x55bc76d1ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.671+0000 7fd2b5e18740 4 client.25040 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.671+0000 7fd2b5e18740 1 -- 172.21.15.97:0/502776946 --> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc76c786f0 con 0x55bc76d1d800 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.671+0000 7fd2abe00640 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x55bc76d1d800 0x55bc76d1ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.672+0000 7fd2abe00640 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x55bc76d1d800 0x55bc76d1ec70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a4f08440c15997b server_cookie=c6e7431dd634db8a in_seq=0 out_seq=0 2024-03-26T18:58:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.672+0000 7fd296000640 10 client.25040 ms_handle_connect on v2:172.21.15.97:6840/4165060179 2024-03-26T18:58:12.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd296000640 1 -- 172.21.15.97:0/502776946 <== mds.1 v2:172.21.15.97:6840/4165060179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bc76c786f0 con 0x55bc76d1d800 2024-03-26T18:58:12.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd296000640 10 client.25040 handle_command_reply: tid=0 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 client.25040 shutdown 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 -- 172.21.15.97:0/502776946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 msgr2=0x55bc76d183c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d183c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009fa0 tx=0x7fd2a4004cd0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 -- 172.21.15.97:0/502776946 shutdown_connections 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x55bc76d1d800 0x55bc76d1ec70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d183c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d07360 0x55bc76d11380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/502776946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d10e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd296000640 0 client.25040 ms_handle_reset on v2:172.21.15.97:6840/4165060179 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 -- 172.21.15.97:0/502776946 >> 172.21.15.97:0/502776946 conn(0x55bc76d01ee0 msgr2=0x55bc76d01020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 -- 172.21.15.97:0/502776946 shutdown_connections 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.673+0000 7fd2b5e18740 1 -- 172.21.15.97:0/502776946 wait complete. 2024-03-26T18:58:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.674+0000 7fd2b5e18740 20 client.25040 trim_cache size 0 max 16384 2024-03-26T18:58:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.693+0000 7fd2b5e18740 1 Processor -- start 2024-03-26T18:58:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.693+0000 7fd2b5e18740 1 -- start start 2024-03-26T18:58:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.693+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d06970 0x55bc76d022c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.693+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d18380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.693+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d0c4d0 0x55bc76d10a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.693+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc76d1cfb0 con 0x55bc76d06970 2024-03-26T18:58:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.693+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bc76d1d680 con 0x55bc76d07360 2024-03-26T18:58:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.693+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc76d1dd50 con 0x55bc76d0c4d0 2024-03-26T18:58:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2abe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d06970 0x55bc76d022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d18380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2b1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d0c4d0 0x55bc76d10a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d18380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52374/0 (socket says 172.21.15.97:52374) 2024-03-26T18:58:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2ab400640 1 -- 172.21.15.97:0/1542991921 learned_addr learned my addr 172.21.15.97:0/1542991921 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2ab400640 1 -- 172.21.15.97:0/1542991921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d0c4d0 msgr2=0x55bc76d10a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2ab400640 1 --2- 172.21.15.97:0/1542991921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d0c4d0 0x55bc76d10a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2ab400640 1 -- 172.21.15.97:0/1542991921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d06970 msgr2=0x55bc76d022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2ab400640 1 --2- 172.21.15.97:0/1542991921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d06970 0x55bc76d022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2ab400640 1 -- 172.21.15.97:0/1542991921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc76d06530 con 0x55bc76d07360 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2b1200640 1 --2- 172.21.15.97:0/1542991921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d0c4d0 0x55bc76d10a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2ab400640 1 --2- 172.21.15.97:0/1542991921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d18380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd2980043d0 tx=0x7fd29801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfd698eee44de7c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.694+0000 7fd2a0c00640 1 -- 172.21.15.97:0/1542991921 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2980044c0 con 0x55bc76d07360 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2a0c00640 1 -- 172.21.15.97:0/1542991921 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd298005500 con 0x55bc76d07360 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2a0c00640 1 -- 172.21.15.97:0/1542991921 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29801f680 con 0x55bc76d07360 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2b5e18740 1 -- 172.21.15.97:0/1542991921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 msgr2=0x55bc76d18380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/1542991921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d18380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd2980043d0 tx=0x7fd29801a000 comp rx=0 tx=0).stop 2024-03-26T18:58:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2b5e18740 1 -- 172.21.15.97:0/1542991921 shutdown_connections 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/1542991921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d0c4d0 0x55bc76d10a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/1542991921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d07360 0x55bc76d18380 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/1542991921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d06970 0x55bc76d022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2b5e18740 1 -- 172.21.15.97:0/1542991921 >> 172.21.15.97:0/1542991921 conn(0x55bc76d01ee0 msgr2=0x55bc76d1c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2b5e18740 1 -- 172.21.15.97:0/1542991921 shutdown_connections 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.695+0000 7fd2b5e18740 1 -- 172.21.15.97:0/1542991921 wait complete. 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.696+0000 7fd2b5e18740 1 -- start start 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.696+0000 7fd2b5e18740 1 Processor -- start 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.696+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d14510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.696+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d07360 0x55bc76d14a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.696+0000 7fd2b5e18740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d17480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.696+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc76d1eaf0 con 0x55bc76d07360 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.696+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bc76d1e420 con 0x55bc76d0c4d0 2024-03-26T18:58:12.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.696+0000 7fd2b5e18740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc76d1f1c0 con 0x55bc76d06970 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.696+0000 7fd2b5e18740 10 client.0 _ll_register_callbacks cb 0x7fd2a4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.696+0000 7fd2abe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d14510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d07360 0x55bc76d14a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2abe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d14510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37456/0 (socket says 172.21.15.97:37456) 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2abe00640 1 -- 172.21.15.97:0/2803670733 learned_addr learned my addr 172.21.15.97:0/2803670733 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2b1200640 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d17480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2abe00640 1 -- 172.21.15.97:0/2803670733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 msgr2=0x55bc76d17480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2abe00640 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d17480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2abe00640 1 -- 172.21.15.97:0/2803670733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d07360 msgr2=0x55bc76d14a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2abe00640 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d07360 0x55bc76d14a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2abe00640 1 -- 172.21.15.97:0/2803670733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc76d17ac0 con 0x55bc76d06970 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2b1200640 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d17480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2abe00640 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d14510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0bd2c0 tx=0x7fd29c041fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca9dd28d54699f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd296000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd296000640 1 -- 172.21.15.97:0/2803670733 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c04b040 con 0x55bc76d06970 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd296000640 1 -- 172.21.15.97:0/2803670733 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd29c0b9c30 con 0x55bc76d06970 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2b5e18740 1 -- 172.21.15.97:0/2803670733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd298005500 con 0x55bc76d06970 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd296000640 1 -- 172.21.15.97:0/2803670733 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd29c0bcf20 con 0x55bc76d06970 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2b5e18740 10 client.25030 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.697+0000 7fd2b5e18740 1 -- 172.21.15.97:0/2803670733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc76d83a10 con 0x55bc76d06970 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.698+0000 7fd296000640 1 -- 172.21.15.97:0/2803670733 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd298005500 con 0x55bc76d06970 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.698+0000 7fd296000640 1 -- 172.21.15.97:0/2803670733 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd29c0035f0 con 0x55bc76d06970 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.698+0000 7fd2b5e18740 10 client.25030 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.698+0000 7fd2b5e18740 10 client.25030 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.698+0000 7fd2b5e18740 20 client.25030 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.698+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x55bc76d88660 0x55bc76d87370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.698+0000 7fd2b5e18740 4 client.25030 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.698+0000 7fd2b5e18740 1 -- 172.21.15.97:0/2803670733 --> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] -- command(tid 0: {"prefix": "status"}) -- 0x55bc76c72a40 con 0x55bc76d88660 2024-03-26T18:58:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.698+0000 7fd2ab400640 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x55bc76d88660 0x55bc76d87370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd2ab400640 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x55bc76d88660 0x55bc76d87370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=883a587d3c27a0d server_cookie=3f1773d1bd0cb4b8 in_seq=0 out_seq=0 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd296000640 10 client.25030 ms_handle_connect on v2:172.21.15.97:6840/4165060179 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd296000640 1 -- 172.21.15.97:0/2803670733 <== mds.1 v2:172.21.15.97:6840/4165060179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55bc76c72a40 con 0x55bc76d88660 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd296000640 10 client.25030 handle_command_reply: tid=0 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd2b5e18740 1 client.25030 shutdown 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd2b5e18740 1 -- 172.21.15.97:0/2803670733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 msgr2=0x55bc76d14510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d14510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0bd2c0 tx=0x7fd29c041fb0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd2b5e18740 1 -- 172.21.15.97:0/2803670733 shutdown_connections 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x55bc76d88660 0x55bc76d87370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc76d0c4d0 0x55bc76d17480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc76d07360 0x55bc76d14a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd2b5e18740 1 --2- 172.21.15.97:0/2803670733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc76d06970 0x55bc76d14510 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd296000640 0 client.25030 ms_handle_reset on v2:172.21.15.97:6840/4165060179 2024-03-26T18:58:12.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.699+0000 7fd2b5e18740 1 -- 172.21.15.97:0/2803670733 >> 172.21.15.97:0/2803670733 conn(0x55bc76d01ee0 msgr2=0x55bc76d022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b5e18740 1 -- 172.21.15.97:0/2803670733 shutdown_connections 2024-03-26T18:58:12.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b5e18740 1 -- 172.21.15.97:0/2803670733 wait complete. 2024-03-26T18:58:12.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b5e18740 20 client.25030 trim_cache size 0 max 16384 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24569, 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 104.862887723, 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2024-03-26T18:58:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T18:58:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 118.165488337 2024-03-26T18:58:12.699 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:58:12.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b3200640 1 -- 172.21.15.97:0/925620456 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2900a2eb0 msgr2=0x7fd2900a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b3200640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2900a2eb0 0x7fd2900a5370 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55bc76d02e50 tx=0x7fd298005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:12.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b3200640 1 -- 172.21.15.97:0/925620456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac102420 msgr2=0x7fd2ac199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b3200640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac102420 0x7fd2ac199e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd29c02c260 tx=0x7fd29c02c670 comp rx=0 tx=0).stop 2024-03-26T18:58:12.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b3200640 1 -- 172.21.15.97:0/925620456 shutdown_connections 2024-03-26T18:58:12.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b3200640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd2900a2eb0 0x7fd2900a5370 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b3200640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2ac107680 0x7fd2ac1a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b3200640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2ac104d50 0x7fd2ac19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b3200640 1 --2- 172.21.15.97:0/925620456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac102420 0x7fd2ac199e50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.700+0000 7fd2b3200640 1 -- 172.21.15.97:0/925620456 >> 172.21.15.97:0/925620456 conn(0x7fd2ac0fbd70 msgr2=0x7fd2ac101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.701+0000 7fd2b3200640 1 -- 172.21.15.97:0/925620456 shutdown_connections 2024-03-26T18:58:12.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.701+0000 7fd2b3200640 1 -- 172.21.15.97:0/925620456 wait complete. 2024-03-26T18:58:12.717 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T18:58:12.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.926+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3615709782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d681017c0 msgr2=0x7f5d68101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.926+0000 7f5d6ca00640 1 --2- 172.21.15.97:0/3615709782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d681017c0 0x7f5d68101be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009900 tx=0x7f5d5802bc40 comp rx=0 tx=0).stop 2024-03-26T18:58:12.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.926+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3615709782 shutdown_connections 2024-03-26T18:58:12.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.926+0000 7f5d6ca00640 1 --2- 172.21.15.97:0/3615709782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d68103370 0x7f5d68109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.926+0000 7f5d6ca00640 1 --2- 172.21.15.97:0/3615709782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d681029b0 0x7f5d68102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.926+0000 7f5d6ca00640 1 --2- 172.21.15.97:0/3615709782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d681017c0 0x7f5d68101be0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.926+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3615709782 >> 172.21.15.97:0/3615709782 conn(0x7f5d680fcd30 msgr2=0x7f5d680ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.926+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3615709782 shutdown_connections 2024-03-26T18:58:12.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.926+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3615709782 wait complete. 2024-03-26T18:58:12.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.927+0000 7f5d6ca00640 1 Processor -- start 2024-03-26T18:58:12.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.927+0000 7f5d6ca00640 1 -- start start 2024-03-26T18:58:12.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d6ca00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d681017c0 0x7f5d68107540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d6ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d681029b0 0x7f5d68107a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d6ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d68103370 0x7f5d681040b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d6ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6810d460 con 0x7f5d68103370 2024-03-26T18:58:12.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d6ca00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6810d2e0 con 0x7f5d681017c0 2024-03-26T18:58:12.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d6ca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6810d5e0 con 0x7f5d681029b0 2024-03-26T18:58:12.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d67400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d681017c0 0x7f5d68107540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d681029b0 0x7f5d68107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d67e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d68103370 0x7f5d681040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d681029b0 0x7f5d68107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37478/0 (socket says 172.21.15.97:37478) 2024-03-26T18:58:12.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d67400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d681017c0 0x7f5d68107540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52406/0 (socket says 172.21.15.97:52406) 2024-03-26T18:58:12.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d66a00640 1 -- 172.21.15.97:0/3655356907 learned_addr learned my addr 172.21.15.97:0/3655356907 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d66a00640 1 -- 172.21.15.97:0/3655356907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d681017c0 msgr2=0x7f5d68107540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d66a00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d681017c0 0x7f5d68107540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d66a00640 1 -- 172.21.15.97:0/3655356907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d68103370 msgr2=0x7f5d681040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d66a00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d68103370 0x7f5d681040b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.928+0000 7f5d66a00640 1 -- 172.21.15.97:0/3655356907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d68104970 con 0x7f5d681029b0 2024-03-26T18:58:12.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.929+0000 7f5d66a00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d681029b0 0x7f5d68107a80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400ea60 tx=0x7f5d5400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.929+0000 7f5d5fe00640 1 -- 172.21.15.97:0/3655356907 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400cd20 con 0x7f5d681029b0 2024-03-26T18:58:12.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.929+0000 7f5d5fe00640 1 -- 172.21.15.97:0/3655356907 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d540035f0 con 0x7f5d681029b0 2024-03-26T18:58:12.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.929+0000 7f5d5fe00640 1 -- 172.21.15.97:0/3655356907 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54010690 con 0x7f5d681029b0 2024-03-26T18:58:12.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.929+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3655356907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d681a6160 con 0x7f5d681029b0 2024-03-26T18:58:12.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.929+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3655356907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d681a65c0 con 0x7f5d681029b0 2024-03-26T18:58:12.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.932+0000 7f5d5fe00640 1 -- 172.21.15.97:0/3655356907 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f5d540037a0 con 0x7f5d681029b0 2024-03-26T18:58:12.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d70d6b740 1 Processor -- start 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d5fe00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5d4c0a2fa0 0x7f5d4c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d70d6b740 1 -- start start 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d5fe00640 1 -- 172.21.15.97:0/3655356907 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5d540cf5f0 con 0x7f5d681029b0 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d67e00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5d4c0a2fa0 0x7f5d4c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52acf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 0x5583f52b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583f52111b0 con 0x5583f52b26b0 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5583f51b9b00 con 0x5583f52ad540 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.934+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5583f520d160 con 0x5583f52acb50 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d67400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d67400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52acf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37488/0 (socket says 172.21.15.97:37488) 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d67400640 1 -- 172.21.15.97:0/3019453396 learned_addr learned my addr 172.21.15.97:0/3019453396 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52422/0 (socket says 172.21.15.97:52422) 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d67e00640 1 --2- 172.21.15.97:0/3019453396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 0x5583f52b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d66a00640 1 -- 172.21.15.97:0/3019453396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 msgr2=0x5583f52acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d66a00640 1 --2- 172.21.15.97:0/3019453396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52acf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d66a00640 1 -- 172.21.15.97:0/3019453396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 msgr2=0x5583f52b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d66a00640 1 --2- 172.21.15.97:0/3019453396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 0x5583f52b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d66a00640 1 -- 172.21.15.97:0/3019453396 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f4f252d0 con 0x5583f52ad540 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d67e00640 1 --2- 172.21.15.97:0/3019453396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 0x5583f52b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d67e00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5d4c0a2fa0 0x7f5d4c0a5460 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d60005fd0 tx=0x7f5d60011810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d66a00640 1 --2- 172.21.15.97:0/3019453396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52b2040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400ea30 tx=0x7f5d5409b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e595e74b92b43bf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d53e00640 1 -- 172.21.15.97:0/3019453396 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5409bd50 con 0x5583f52ad540 2024-03-26T18:58:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d53e00640 1 -- 172.21.15.97:0/3019453396 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5409fea0 con 0x5583f52ad540 2024-03-26T18:58:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.935+0000 7f5d53e00640 1 -- 172.21.15.97:0/3019453396 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5409b1a0 con 0x5583f52ad540 2024-03-26T18:58:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.936+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3019453396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 msgr2=0x5583f52b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.936+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/3019453396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52b2040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400ea30 tx=0x7f5d5409b890 comp rx=0 tx=0).stop 2024-03-26T18:58:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.936+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3019453396 shutdown_connections 2024-03-26T18:58:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.936+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/3019453396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 0x5583f52b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.936+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/3019453396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52b2040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.936+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/3019453396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.936+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3019453396 >> 172.21.15.97:0/3019453396 conn(0x5583f52a80c0 msgr2=0x5583f52a84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.936+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3019453396 shutdown_connections 2024-03-26T18:58:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.936+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3019453396 wait complete. 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.937+0000 7f5d70d6b740 1 -- start start 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.937+0000 7f5d70d6b740 1 Processor -- start 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.937+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.937+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52ad540 0x5583f52b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.937+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52b26b0 0x5583f52be610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.937+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583f52b5600 con 0x5583f52ad540 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.937+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5583f51cced0 con 0x5583f52b26b0 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.937+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5583f52b5780 con 0x5583f52acb50 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d70d6b740 10 client.0 _ll_register_callbacks cb 0x5583f5208710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d67400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d67400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37492/0 (socket says 172.21.15.97:37492) 2024-03-26T18:58:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d67400640 1 -- 172.21.15.97:0/3969043704 learned_addr learned my addr 172.21.15.97:0/3969043704 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d66a00640 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52ad540 0x5583f52b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d67e00640 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52b26b0 0x5583f52be610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d67400640 1 -- 172.21.15.97:0/3969043704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52b26b0 msgr2=0x5583f52be610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d67400640 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52b26b0 0x5583f52be610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d67400640 1 -- 172.21.15.97:0/3969043704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52ad540 msgr2=0x5583f52b75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d67400640 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52ad540 0x5583f52b75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d67400640 1 -- 172.21.15.97:0/3969043704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f4fef6b0 con 0x5583f52acb50 2024-03-26T18:58:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d66a00640 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52ad540 0x5583f52b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d67400640 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b7090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009230 tx=0x7f5d58004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f538d2d4f668008 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.938+0000 7f5d50c00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:58:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.939+0000 7f5d50c00640 1 -- 172.21.15.97:0/3969043704 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58005d50 con 0x5583f52acb50 2024-03-26T18:58:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.939+0000 7f5d50c00640 1 -- 172.21.15.97:0/3969043704 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5d58005040 con 0x5583f52acb50 2024-03-26T18:58:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.939+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3969043704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5409fea0 con 0x5583f52acb50 2024-03-26T18:58:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.939+0000 7f5d50c00640 1 -- 172.21.15.97:0/3969043704 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5d58005240 con 0x5583f52acb50 2024-03-26T18:58:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.939+0000 7f5d70d6b740 10 client.25042 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.939+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3969043704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583f52c15d0 con 0x5583f52acb50 2024-03-26T18:58:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.939+0000 7f5d50c00640 1 -- 172.21.15.97:0/3969043704 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5409fea0 con 0x5583f52acb50 2024-03-26T18:58:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.940+0000 7f5d50c00640 1 -- 172.21.15.97:0/3969043704 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5d580035f0 con 0x5583f52acb50 2024-03-26T18:58:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.940+0000 7f5d70d6b740 10 client.25042 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.940+0000 7f5d70d6b740 10 client.25042 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T18:58:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.940+0000 7f5d70d6b740 20 client.25042 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.940+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5583f52c3c70 0x5583f52c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.940+0000 7f5d70d6b740 4 client.25042 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:58:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.940+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3969043704 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583f521e6f0 con 0x5583f52c3c70 2024-03-26T18:58:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.940+0000 7f5d66a00640 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5583f52c3c70 0x5583f52c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.941+0000 7f5d66a00640 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5583f52c3c70 0x5583f52c50e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbdc5f78929151de server_cookie=6988829e5c4e91d0 in_seq=0 out_seq=0 2024-03-26T18:58:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.941+0000 7f5d50c00640 10 client.25042 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:58:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.941+0000 7f5d50c00640 1 -- 172.21.15.97:0/3969043704 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5583f521e6f0 con 0x5583f52c3c70 2024-03-26T18:58:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.941+0000 7f5d50c00640 10 client.25042 handle_command_reply: tid=0 2024-03-26T18:58:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.941+0000 7f5d70d6b740 1 client.25042 shutdown 2024-03-26T18:58:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3969043704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 msgr2=0x5583f52b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b7090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009230 tx=0x7f5d58004270 comp rx=0 tx=0).stop 2024-03-26T18:58:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3969043704 shutdown_connections 2024-03-26T18:58:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5583f52c3c70 0x5583f52c50e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52b26b0 0x5583f52be610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d50c00640 0 client.25042 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:58:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52ad540 0x5583f52b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/3969043704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b7090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3969043704 >> 172.21.15.97:0/3969043704 conn(0x5583f52a80c0 msgr2=0x5583f52a72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3969043704 shutdown_connections 2024-03-26T18:58:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 1 -- 172.21.15.97:0/3969043704 wait complete. 2024-03-26T18:58:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.942+0000 7f5d70d6b740 20 client.25042 trim_cache size 0 max 16384 2024-03-26T18:58:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.973+0000 7f5d70d6b740 1 Processor -- start 2024-03-26T18:58:12.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.973+0000 7f5d70d6b740 1 -- start start 2024-03-26T18:58:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52acb50 0x5583f52b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52be5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52b26b0 0x5583f52b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583f52c3420 con 0x5583f52acb50 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5583f52c3af0 con 0x5583f52ad540 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5583f52c41c0 con 0x5583f52b26b0 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52acb50 0x5583f52b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52be5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d67e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52b26b0 0x5583f52b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52be5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52438/0 (socket says 172.21.15.97:52438) 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d66a00640 1 -- 172.21.15.97:0/453193931 learned_addr learned my addr 172.21.15.97:0/453193931 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d67e00640 1 -- 172.21.15.97:0/453193931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 msgr2=0x5583f52be5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d67e00640 1 --2- 172.21.15.97:0/453193931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52be5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d67e00640 1 -- 172.21.15.97:0/453193931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52acb50 msgr2=0x5583f52b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d67e00640 1 --2- 172.21.15.97:0/453193931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52acb50 0x5583f52b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.974+0000 7f5d67e00640 1 -- 172.21.15.97:0/453193931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f52a3660 con 0x5583f52b26b0 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.975+0000 7f5d67e00640 1 --2- 172.21.15.97:0/453193931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52b26b0 0x5583f52b6060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5d6001d830 tx=0x7f5d6001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=576db4cee93d911d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.975+0000 7f5d53400640 1 -- 172.21.15.97:0/453193931 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6002c070 con 0x5583f52b26b0 2024-03-26T18:58:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.975+0000 7f5d53400640 1 -- 172.21.15.97:0/453193931 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d60019050 con 0x5583f52b26b0 2024-03-26T18:58:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.975+0000 7f5d53400640 1 -- 172.21.15.97:0/453193931 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60017040 con 0x5583f52b26b0 2024-03-26T18:58:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.976+0000 7f5d70d6b740 1 -- 172.21.15.97:0/453193931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52b26b0 msgr2=0x5583f52b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.976+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/453193931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52b26b0 0x5583f52b6060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5d6001d830 tx=0x7f5d6001c000 comp rx=0 tx=0).stop 2024-03-26T18:58:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.976+0000 7f5d70d6b740 1 -- 172.21.15.97:0/453193931 shutdown_connections 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.976+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/453193931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52b26b0 0x5583f52b6060 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.976+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/453193931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52be5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.976+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/453193931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52acb50 0x5583f52b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.976+0000 7f5d70d6b740 1 -- 172.21.15.97:0/453193931 >> 172.21.15.97:0/453193931 conn(0x5583f52a80c0 msgr2=0x5583f52a84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.976+0000 7f5d70d6b740 1 -- 172.21.15.97:0/453193931 shutdown_connections 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.976+0000 7f5d70d6b740 1 -- 172.21.15.97:0/453193931 wait complete. 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.977+0000 7f5d70d6b740 1 -- start start 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.977+0000 7f5d70d6b740 1 Processor -- start 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.977+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.977+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.977+0000 7f5d70d6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 0x5583f52bd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.977+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583f52c4f60 con 0x5583f52b26b0 2024-03-26T18:58:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.977+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5583f52c4890 con 0x5583f52ad540 2024-03-26T18:58:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.977+0000 7f5d70d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5583f52c5630 con 0x5583f52acb50 2024-03-26T18:58:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d70d6b740 10 client.0 _ll_register_callbacks cb 0x7f5d58009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d67400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52ba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d67400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37516/0 (socket says 172.21.15.97:37516) 2024-03-26T18:58:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d67e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 0x5583f52bd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d67400640 1 -- 172.21.15.97:0/1213965558 learned_addr learned my addr 172.21.15.97:0/1213965558 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d67400640 1 -- 172.21.15.97:0/1213965558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 msgr2=0x5583f52ba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d67400640 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52ba4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d67400640 1 -- 172.21.15.97:0/1213965558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 msgr2=0x5583f52bd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d67400640 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 0x5583f52bd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d67400640 1 -- 172.21.15.97:0/1213965558 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f52bda90 con 0x5583f52acb50 2024-03-26T18:58:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d66a00640 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d67400640 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b9f90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d5802c7c0 tx=0x7f5d58003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9393d83c5065d9e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.978+0000 7f5d50c00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:58:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d50c00640 1 -- 172.21.15.97:0/1213965558 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58058070 con 0x5583f52acb50 2024-03-26T18:58:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d50c00640 1 -- 172.21.15.97:0/1213965558 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5d580037f0 con 0x5583f52acb50 2024-03-26T18:58:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d70d6b740 1 -- 172.21.15.97:0/1213965558 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d60019050 con 0x5583f52acb50 2024-03-26T18:58:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d50c00640 1 -- 172.21.15.97:0/1213965558 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5d58063070 con 0x5583f52acb50 2024-03-26T18:58:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d70d6b740 10 client.25048 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d70d6b740 1 -- 172.21.15.97:0/1213965558 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583f5329cd0 con 0x5583f52acb50 2024-03-26T18:58:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d50c00640 1 -- 172.21.15.97:0/1213965558 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d60019050 con 0x5583f52acb50 2024-03-26T18:58:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d50c00640 1 -- 172.21.15.97:0/1213965558 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5d580049a0 con 0x5583f52acb50 2024-03-26T18:58:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d70d6b740 10 client.25048 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d70d6b740 10 client.25048 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T18:58:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d70d6b740 20 client.25048 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.979+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5583f532e840 0x5583f532d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.980+0000 7f5d70d6b740 4 client.25048 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:58:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.980+0000 7f5d70d6b740 1 -- 172.21.15.97:0/1213965558 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "status"}) -- 0x5583f5218a40 con 0x5583f532e840 2024-03-26T18:58:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.980+0000 7f5d66a00640 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5583f532e840 0x5583f532d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.980+0000 7f5d66a00640 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5583f532e840 0x5583f532d550 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=178ea9a4d00e1ad server_cookie=7d4fb81c044354e0 in_seq=0 out_seq=0 2024-03-26T18:58:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.980+0000 7f5d50c00640 10 client.25048 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:58:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.980+0000 7f5d50c00640 1 -- 172.21.15.97:0/1213965558 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5583f5218a40 con 0x5583f532e840 2024-03-26T18:58:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.980+0000 7f5d50c00640 10 client.25048 handle_command_reply: tid=0 2024-03-26T18:58:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.980+0000 7f5d70d6b740 1 client.25048 shutdown 2024-03-26T18:58:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 1 -- 172.21.15.97:0/1213965558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 msgr2=0x5583f52b9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b9f90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d5802c7c0 tx=0x7f5d58003f00 comp rx=0 tx=0).stop 2024-03-26T18:58:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 1 -- 172.21.15.97:0/1213965558 shutdown_connections 2024-03-26T18:58:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5583f532e840 0x5583f532d550 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583f52b26b0 0x5583f52bd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583f52ad540 0x5583f52ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d50c00640 0 client.25048 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:58:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 1 --2- 172.21.15.97:0/1213965558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583f52acb50 0x5583f52b9f90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 1 -- 172.21.15.97:0/1213965558 >> 172.21.15.97:0/1213965558 conn(0x5583f52a80c0 msgr2=0x5583f52aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 1 -- 172.21.15.97:0/1213965558 shutdown_connections 2024-03-26T18:58:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 1 -- 172.21.15.97:0/1213965558 wait complete. 2024-03-26T18:58:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.981+0000 7f5d70d6b740 20 client.25048 trim_cache size 0 max 16384 2024-03-26T18:58:12.981 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:58:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:58:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-03-26T18:58:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24581, 2024-03-26T18:58:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-26T18:58:12.981 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-26T18:58:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-26T18:58:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 108.438723396, 2024-03-26T18:58:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-03-26T18:58:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2024-03-26T18:58:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T18:58:12.982 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 115.61098453300001 2024-03-26T18:58:12.982 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:58:12.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3655356907 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5d4c0a2fa0 msgr2=0x7f5d4c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5d4c0a2fa0 0x7f5d4c0a5460 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d60005fd0 tx=0x7f5d60011810 comp rx=0 tx=0).stop 2024-03-26T18:58:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3655356907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d681029b0 msgr2=0x7f5d68107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d681029b0 0x7f5d68107a80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400ea60 tx=0x7f5d5400ee60 comp rx=0 tx=0).stop 2024-03-26T18:58:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3655356907 shutdown_connections 2024-03-26T18:58:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5d4c0a2fa0 0x7f5d4c0a5460 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d68103370 0x7f5d681040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d681029b0 0x7f5d68107a80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 --2- 172.21.15.97:0/3655356907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d681017c0 0x7f5d68107540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:12.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3655356907 >> 172.21.15.97:0/3655356907 conn(0x7f5d680fcd30 msgr2=0x7f5d68073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:12.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3655356907 shutdown_connections 2024-03-26T18:58:12.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:12.982+0000 7f5d6ca00640 1 -- 172.21.15.97:0/3655356907 wait complete. 2024-03-26T18:58:12.995 INFO:tasks.cephadm:Setup complete, yielding 2024-03-26T18:58:12.995 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T18:58:13.005 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-03-26T18:58:13.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-26T18:58:13.456 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:58:14.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:14 smithi097 ceph-mon[31955]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-26T18:58:14.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:14 smithi144 ceph-mon[36053]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-26T18:58:14.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:14 smithi105 ceph-mon[36270]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.776+0000 7f1ba9c00700 1 -- 172.21.15.97:0/2599631443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ba41060d0 msgr2=0x7f1ba410c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.776+0000 7f1ba9c00700 1 --2- 172.21.15.97:0/2599631443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ba41060d0 0x7f1ba410c980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00b0d0 tx=0x7f1b9c00b3e0 comp rx=0 tx=0).stop 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.777+0000 7f1ba9c00700 1 -- 172.21.15.97:0/2599631443 shutdown_connections 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.777+0000 7f1ba9c00700 1 --2- 172.21.15.97:0/2599631443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ba41060d0 0x7f1ba410c980 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.777+0000 7f1ba9c00700 1 --2- 172.21.15.97:0/2599631443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba4104530 0x7f1ba4104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.777+0000 7f1ba9c00700 1 --2- 172.21.15.97:0/2599631443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba4105730 0x7f1ba4105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.777+0000 7f1ba9c00700 1 -- 172.21.15.97:0/2599631443 >> 172.21.15.97:0/2599631443 conn(0x7f1ba40ffad0 msgr2=0x7f1ba4101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.777+0000 7f1ba9c00700 1 -- 172.21.15.97:0/2599631443 shutdown_connections 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.777+0000 7f1ba9c00700 1 -- 172.21.15.97:0/2599631443 wait complete. 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.778+0000 7f1ba9c00700 1 Processor -- start 2024-03-26T18:58:15.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.778+0000 7f1ba9c00700 1 -- start start 2024-03-26T18:58:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba4104530 0x7f1ba419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9c00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba4105730 0x7f1ba419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ba41060d0 0x7f1ba41a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:15.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4110170 con 0x7f1ba4104530 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9c00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba410fff0 con 0x7f1ba41060d0 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9c00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41102f0 con 0x7f1ba4105730 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba4104530 0x7f1ba419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ba41060d0 0x7f1ba41a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ba41060d0 0x7f1ba41a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:52474/0 (socket says 172.21.15.97:52474) 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba4104530 0x7f1ba419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59426/0 (socket says 172.21.15.97:59426) 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9200700 1 -- 172.21.15.97:0/1117337757 learned_addr learned my addr 172.21.15.97:0/1117337757 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9200700 1 -- 172.21.15.97:0/1117337757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba4105730 msgr2=0x7f1ba419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9200700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba4105730 0x7f1ba419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9200700 1 -- 172.21.15.97:0/1117337757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba4104530 msgr2=0x7f1ba419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9200700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba4104530 0x7f1ba419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.779+0000 7f1ba9200700 1 -- 172.21.15.97:0/1117337757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba41a2380 con 0x7f1ba41060d0 2024-03-26T18:58:15.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.780+0000 7f1ba9200700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ba41060d0 0x7f1ba41a1ce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009fd0 tx=0x7f1b9c0079a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.780+0000 7f1ba0c00700 1 -- 172.21.15.97:0/1117337757 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c003c00 con 0x7f1ba41060d0 2024-03-26T18:58:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.780+0000 7f1ba0c00700 1 -- 172.21.15.97:0/1117337757 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9c022070 con 0x7f1ba41060d0 2024-03-26T18:58:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.780+0000 7f1ba0c00700 1 -- 172.21.15.97:0/1117337757 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c01b7b0 con 0x7f1ba41060d0 2024-03-26T18:58:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.780+0000 7f1ba9c00700 1 -- 172.21.15.97:0/1117337757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba41a2610 con 0x7f1ba41060d0 2024-03-26T18:58:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.780+0000 7f1ba9c00700 1 -- 172.21.15.97:0/1117337757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba41a2a10 con 0x7f1ba41060d0 2024-03-26T18:58:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.782+0000 7f1ba9c00700 1 -- 172.21.15.97:0/1117337757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba4069070 con 0x7f1ba41060d0 2024-03-26T18:58:15.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.785+0000 7f1ba0c00700 1 -- 172.21.15.97:0/1117337757 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f1b9c01d470 con 0x7f1ba41060d0 2024-03-26T18:58:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.786+0000 7f1ba0c00700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1b880a3030 0x7f1b880a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.786+0000 7f1ba0c00700 1 -- 172.21.15.97:0/1117337757 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1b9c00f020 con 0x7f1ba41060d0 2024-03-26T18:58:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.786+0000 7f1ba3e00700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1b880a3030 0x7f1b880a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.787+0000 7f1ba3e00700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1b880a3030 0x7f1b880a54f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4105590 tx=0x7f1b94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:15.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:15.790+0000 7f1ba0c00700 1 -- 172.21.15.97:0/1117337757 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b9c09cfb0 con 0x7f1ba41060d0 2024-03-26T18:58:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.046+0000 7f1ba9c00700 1 -- 172.21.15.97:0/1117337757 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1ba40627f0 con 0x7f1b880a3030 2024-03-26T18:58:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.048+0000 7f1ba0c00700 1 -- 172.21.15.97:0/1117337757 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1ba40627f0 con 0x7f1b880a3030 2024-03-26T18:58:16.041 INFO:teuthology.orchestra.run.smithi097.stdout:Backend: cephadm 2024-03-26T18:58:16.041 INFO:teuthology.orchestra.run.smithi097.stdout:Available: Yes 2024-03-26T18:58:16.041 INFO:teuthology.orchestra.run.smithi097.stdout:Paused: No 2024-03-26T18:58:16.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.051+0000 7f1ba9c00700 1 -- 172.21.15.97:0/1117337757 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1b880a3030 msgr2=0x7f1b880a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:16.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.051+0000 7f1ba9c00700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1b880a3030 0x7f1b880a54f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4105590 tx=0x7f1b94009260 comp rx=0 tx=0).stop 2024-03-26T18:58:16.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.051+0000 7f1ba9c00700 1 -- 172.21.15.97:0/1117337757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ba41060d0 msgr2=0x7f1ba41a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:16.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.051+0000 7f1ba9c00700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ba41060d0 0x7f1ba41a1ce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009fd0 tx=0x7f1b9c0079a0 comp rx=0 tx=0).stop 2024-03-26T18:58:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.052+0000 7f1ba9c00700 1 -- 172.21.15.97:0/1117337757 shutdown_connections 2024-03-26T18:58:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.052+0000 7f1ba9c00700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ba41060d0 0x7f1ba41a1ce0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.052+0000 7f1ba9c00700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba4104530 0x7f1ba419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.052+0000 7f1ba9c00700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1b880a3030 0x7f1b880a54f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.052+0000 7f1ba9c00700 1 --2- 172.21.15.97:0/1117337757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba4105730 0x7f1ba419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.052+0000 7f1ba9c00700 1 -- 172.21.15.97:0/1117337757 >> 172.21.15.97:0/1117337757 conn(0x7f1ba40ffad0 msgr2=0x7f1ba41089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.052+0000 7f1ba9c00700 1 -- 172.21.15.97:0/1117337757 shutdown_connections 2024-03-26T18:58:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:16.052+0000 7f1ba9c00700 1 -- 172.21.15.97:0/1117337757 wait complete. 2024-03-26T18:58:16.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:16 smithi097 ceph-mon[31955]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T18:58:16.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:16 smithi144 ceph-mon[36053]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T18:58:16.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:16 smithi105 ceph-mon[36270]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T18:58:16.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-26T18:58:17.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:17 smithi097 ceph-mon[31955]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:58:17.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:17 smithi144 ceph-mon[36053]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:17.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:17 smithi105 ceph-mon[36270]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:18.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:18 smithi097 ceph-mon[31955]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:18.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:18 smithi144 ceph-mon[36053]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:18.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:18 smithi105 ceph-mon[36270]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:19.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.522+0000 7fd3b3400700 1 -- 172.21.15.97:0/3892950660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac103600 msgr2=0x7fd3ac103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:19.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.522+0000 7fd3b3400700 1 --2- 172.21.15.97:0/3892950660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac103600 0x7fd3ac103a60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3980099c0 tx=0x7fd398009cd0 comp rx=0 tx=0).stop 2024-03-26T18:58:19.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.522+0000 7fd3b3400700 1 -- 172.21.15.97:0/3892950660 shutdown_connections 2024-03-26T18:58:19.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.522+0000 7fd3b3400700 1 --2- 172.21.15.97:0/3892950660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac103fa0 0x7fd3ac10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:19.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.522+0000 7fd3b3400700 1 --2- 172.21.15.97:0/3892950660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac102400 0x7fd3ac102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:19.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.522+0000 7fd3b3400700 1 --2- 172.21.15.97:0/3892950660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac103600 0x7fd3ac103a60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:19.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.522+0000 7fd3b3400700 1 -- 172.21.15.97:0/3892950660 >> 172.21.15.97:0/3892950660 conn(0x7fd3ac0fd960 msgr2=0x7fd3ac0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:19.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.522+0000 7fd3b3400700 1 -- 172.21.15.97:0/3892950660 shutdown_connections 2024-03-26T18:58:19.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.522+0000 7fd3b3400700 1 -- 172.21.15.97:0/3892950660 wait complete. 2024-03-26T18:58:19.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.523+0000 7fd3b3400700 1 Processor -- start 2024-03-26T18:58:19.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b3400700 1 -- start start 2024-03-26T18:58:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b3400700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac102400 0x7fd3ac076100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b3400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac103600 0x7fd3ac0747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b3400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac103fa0 0x7fd3ac074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b3400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10e020 con 0x7fd3ac103600 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b3400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10dea0 con 0x7fd3ac103fa0 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b3400700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10e1a0 con 0x7fd3ac102400 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b0a00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac102400 0x7fd3ac076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b0a00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac102400 0x7fd3ac076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37550/0 (socket says 172.21.15.97:37550) 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b0a00700 1 -- 172.21.15.97:0/440310000 learned_addr learned my addr 172.21.15.97:0/440310000 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.524+0000 7fd3b1400700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac103fa0 0x7fd3ac074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3abe00700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac103600 0x7fd3ac0747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3b0a00700 1 -- 172.21.15.97:0/440310000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac103fa0 msgr2=0x7fd3ac074ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3b0a00700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac103fa0 0x7fd3ac074ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3b0a00700 1 -- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac103600 msgr2=0x7fd3ac0747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3b0a00700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac103600 0x7fd3ac0747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3b0a00700 1 -- 172.21.15.97:0/440310000 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac0755b0 con 0x7fd3ac102400 2024-03-26T18:58:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3abe00700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac103600 0x7fd3ac0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:58:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3b0a00700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac102400 0x7fd3ac076100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000e970 tx=0x7fd3a000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3a9600700 1 -- 172.21.15.97:0/440310000 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000cbd0 con 0x7fd3ac102400 2024-03-26T18:58:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3a9600700 1 -- 172.21.15.97:0/440310000 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a0004500 con 0x7fd3ac102400 2024-03-26T18:58:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.525+0000 7fd3b3400700 1 -- 172.21.15.97:0/440310000 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac1a27c0 con 0x7fd3ac102400 2024-03-26T18:58:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.526+0000 7fd3b3400700 1 -- 172.21.15.97:0/440310000 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac1a2cd0 con 0x7fd3ac102400 2024-03-26T18:58:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.526+0000 7fd3a9600700 1 -- 172.21.15.97:0/440310000 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0003e30 con 0x7fd3ac102400 2024-03-26T18:58:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.527+0000 7fd3b3400700 1 -- 172.21.15.97:0/440310000 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38c005360 con 0x7fd3ac102400 2024-03-26T18:58:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.528+0000 7fd3a9600700 1 -- 172.21.15.97:0/440310000 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fd3a0004020 con 0x7fd3ac102400 2024-03-26T18:58:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.530+0000 7fd3a9600700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3940a2fb0 0x7fd3940a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.530+0000 7fd3a9600700 1 -- 172.21.15.97:0/440310000 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd3a00cf710 con 0x7fd3ac102400 2024-03-26T18:58:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.530+0000 7fd3abe00700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3940a2fb0 0x7fd3940a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.531+0000 7fd3abe00700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3940a2fb0 0x7fd3940a5470 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd398005cd0 tx=0x7fd398005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.533+0000 7fd3a9600700 1 -- 172.21.15.97:0/440310000 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd3a009c510 con 0x7fd3ac102400 2024-03-26T18:58:19.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.762+0000 7fd3b3400700 1 -- 172.21.15.97:0/440310000 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd38c000bf0 con 0x7fd3940a2fb0 2024-03-26T18:58:19.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.780+0000 7fd3a9600700 1 -- 172.21.15.97:0/440310000 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fd38c000bf0 con 0x7fd3940a2fb0 2024-03-26T18:58:19.773 INFO:teuthology.orchestra.run.smithi097.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-26T18:58:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:mds.a smithi097 running (2m) 24s ago 2m 17.0M - 19.0.0-2311-g36051606 497bb87bc5da 99cab503a0a8 2024-03-26T18:58:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:mds.b smithi105 running (2m) 54s ago 2m 26.0M - 19.0.0-2311-g36051606 497bb87bc5da ccc6801e72c3 2024-03-26T18:58:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:mds.c smithi144 running (2m) 54s ago 2m 19.5M - 19.0.0-2311-g36051606 497bb87bc5da a57bdb37035e 2024-03-26T18:58:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:mds.d smithi097 running (2m) 24s ago 2m 16.6M - 19.0.0-2311-g36051606 497bb87bc5da 3c4c8e413743 2024-03-26T18:58:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:mds.e smithi105 running (2m) 54s ago 2m 17.4M - 19.0.0-2311-g36051606 497bb87bc5da 2fff5cc0ee46 2024-03-26T18:58:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:mds.f smithi144 running (2m) 54s ago 2m 19.1M - 19.0.0-2311-g36051606 497bb87bc5da 81527c8e835f 2024-03-26T18:58:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:mds.g smithi097 running (2m) 24s ago 2m 12.4M - 19.0.0-2311-g36051606 497bb87bc5da 1446ef437b53 2024-03-26T18:58:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:mds.h smithi105 running (2m) 54s ago 2m 17.0M - 19.0.0-2311-g36051606 497bb87bc5da b609505c9761 2024-03-26T18:58:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:mds.i smithi144 running (2m) 54s ago 2m 17.5M - 19.0.0-2311-g36051606 497bb87bc5da a28535586c94 2024-03-26T18:58:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:mds.j smithi097 running (2m) 24s ago 2m 12.5M - 19.0.0-2311-g36051606 497bb87bc5da 6365380a10d2 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:mds.k smithi105 running (2m) 54s ago 2m 17.0M - 19.0.0-2311-g36051606 497bb87bc5da 5a839fe23511 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:mds.l smithi144 running (2m) 54s ago 2m 16.9M - 19.0.0-2311-g36051606 497bb87bc5da ca0f9421c468 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.x smithi097 *:9283,8765 running (10m) 24s ago 10m 514M - 19.0.0-2311-g36051606 497bb87bc5da ad6d5e2f3c87 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.y smithi105 *:8443,8765 running (8m) 54s ago 8m 454M - 19.0.0-2311-g36051606 497bb87bc5da 201c4d33d4db 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.z smithi144 *:8443,8765 running (8m) 54s ago 8m 451M - 19.0.0-2311-g36051606 497bb87bc5da 88fa8a76e5eb 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:mon.a smithi097 running (10m) 24s ago 10m 47.6M 2048M 19.0.0-2311-g36051606 497bb87bc5da 67e023dc813f 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:mon.b smithi105 running (8m) 54s ago 8m 43.1M 2048M 19.0.0-2311-g36051606 497bb87bc5da 5a28e055d823 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:mon.c smithi144 running (8m) 54s ago 8m 43.2M 2048M 19.0.0-2311-g36051606 497bb87bc5da a632f2217aa9 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 smithi097 running (8m) 24s ago 8m 57.4M 4272M 19.0.0-2311-g36051606 497bb87bc5da acf5a90a907e 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 smithi105 running (7m) 54s ago 7m 55.5M 4270M 19.0.0-2311-g36051606 497bb87bc5da 6857bb0ab775 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 smithi144 running (7m) 54s ago 7m 58.8M 4270M 19.0.0-2311-g36051606 497bb87bc5da 6769defc7394 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 smithi097 running (6m) 24s ago 6m 52.9M 4272M 19.0.0-2311-g36051606 497bb87bc5da 2e35504da461 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 smithi105 running (6m) 54s ago 6m 45.4M 4270M 19.0.0-2311-g36051606 497bb87bc5da d05565a46841 2024-03-26T18:58:19.775 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 smithi144 running (5m) 54s ago 5m 44.2M 4270M 19.0.0-2311-g36051606 497bb87bc5da db7b879727e0 2024-03-26T18:58:19.776 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 smithi097 running (5m) 24s ago 5m 41.0M 4272M 19.0.0-2311-g36051606 497bb87bc5da 5b2bb91e0a07 2024-03-26T18:58:19.776 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 smithi105 running (4m) 54s ago 4m 43.0M 4270M 19.0.0-2311-g36051606 497bb87bc5da 068bbc59e319 2024-03-26T18:58:19.776 INFO:teuthology.orchestra.run.smithi097.stdout:osd.8 smithi144 running (4m) 54s ago 4m 54.3M 4270M 19.0.0-2311-g36051606 497bb87bc5da 1bdf1962d786 2024-03-26T18:58:19.776 INFO:teuthology.orchestra.run.smithi097.stdout:osd.9 smithi097 running (3m) 24s ago 3m 50.0M 4272M 19.0.0-2311-g36051606 497bb87bc5da d7c9aae045a2 2024-03-26T18:58:19.776 INFO:teuthology.orchestra.run.smithi097.stdout:osd.10 smithi105 running (3m) 54s ago 3m 42.5M 4270M 19.0.0-2311-g36051606 497bb87bc5da 6dd29fcf6d1d 2024-03-26T18:58:19.776 INFO:teuthology.orchestra.run.smithi097.stdout:osd.11 smithi144 running (3m) 54s ago 3m 41.1M 4270M 19.0.0-2311-g36051606 497bb87bc5da 0b6f84205f00 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 -- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3940a2fb0 msgr2=0x7fd3940a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3940a2fb0 0x7fd3940a5470 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd398005cd0 tx=0x7fd398005c40 comp rx=0 tx=0).stop 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 -- 172.21.15.97:0/440310000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac102400 msgr2=0x7fd3ac076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac102400 0x7fd3ac076100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000e970 tx=0x7fd3a000ec80 comp rx=0 tx=0).stop 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 -- 172.21.15.97:0/440310000 shutdown_connections 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac103fa0 0x7fd3ac074ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3940a2fb0 0x7fd3940a5470 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac102400 0x7fd3ac076100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 --2- 172.21.15.97:0/440310000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac103600 0x7fd3ac0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 -- 172.21.15.97:0/440310000 >> 172.21.15.97:0/440310000 conn(0x7fd3ac0fd960 msgr2=0x7fd3ac106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 -- 172.21.15.97:0/440310000 shutdown_connections 2024-03-26T18:58:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:19.783+0000 7fd3b3400700 1 -- 172.21.15.97:0/440310000 wait complete. 2024-03-26T18:58:20.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:20 smithi097 ceph-mon[31955]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:20.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:20 smithi144 ceph-mon[36053]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:20 smithi105 ceph-mon[36270]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:20.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-26T18:58:21.162 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:58:21.311 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:21 smithi097 ceph-mon[31955]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:21.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:21 smithi144 ceph-mon[36053]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:21.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:21 smithi105 ceph-mon[36270]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:22.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:22 smithi097 ceph-mon[31955]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:22.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:22 smithi144 ceph-mon[36053]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:22.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:22 smithi105 ceph-mon[36270]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.390+0000 7f6899c00700 1 -- 172.21.15.97:0/3307686479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68941061b0 msgr2=0x7f689410ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.390+0000 7f6899c00700 1 --2- 172.21.15.97:0/3307686479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68941061b0 0x7f689410ca60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f688c009fc0 tx=0x7f688c00b330 comp rx=0 tx=0).stop 2024-03-26T18:58:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.390+0000 7f6899c00700 1 -- 172.21.15.97:0/3307686479 shutdown_connections 2024-03-26T18:58:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.390+0000 7f6899c00700 1 --2- 172.21.15.97:0/3307686479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68941061b0 0x7f689410ca60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.390+0000 7f6899c00700 1 --2- 172.21.15.97:0/3307686479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6894104610 0x7f6894104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.390+0000 7f6899c00700 1 --2- 172.21.15.97:0/3307686479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6894105810 0x7f6894105c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.390+0000 7f6899c00700 1 -- 172.21.15.97:0/3307686479 >> 172.21.15.97:0/3307686479 conn(0x7f68940ffb70 msgr2=0x7f6894101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.390+0000 7f6899c00700 1 -- 172.21.15.97:0/3307686479 shutdown_connections 2024-03-26T18:58:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.390+0000 7f6899c00700 1 -- 172.21.15.97:0/3307686479 wait complete. 2024-03-26T18:58:23.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.392+0000 7f6899c00700 1 Processor -- start 2024-03-26T18:58:23.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.392+0000 7f6899c00700 1 -- start start 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.392+0000 7f6899c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6894104610 0x7f689419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.392+0000 7f6899c00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6894105810 0x7f689419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.392+0000 7f6899c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68941061b0 0x7f68941a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6899c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6894110220 con 0x7f6894104610 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6899c00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f68941100a0 con 0x7f68941061b0 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6899c00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68941103a0 con 0x7f6894105810 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6892a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6894104610 0x7f689419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6892000700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6894105810 0x7f689419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6892a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6894104610 0x7f689419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60026/0 (socket says 172.21.15.97:60026) 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6892a00700 1 -- 172.21.15.97:0/3545294764 learned_addr learned my addr 172.21.15.97:0/3545294764 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6893400700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68941061b0 0x7f68941a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6892000700 1 -- 172.21.15.97:0/3545294764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68941061b0 msgr2=0x7f68941a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6892000700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68941061b0 0x7f68941a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6892000700 1 -- 172.21.15.97:0/3545294764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6894104610 msgr2=0x7f689419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6892000700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6894104610 0x7f689419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.393+0000 7f6892000700 1 -- 172.21.15.97:0/3545294764 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68941a23e0 con 0x7f6894105810 2024-03-26T18:58:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.394+0000 7f6892000700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6894105810 0x7f689419acc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f688000e9f0 tx=0x7f688000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.394+0000 7f688b400700 1 -- 172.21.15.97:0/3545294764 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688000c9c0 con 0x7f6894105810 2024-03-26T18:58:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.394+0000 7f688b400700 1 -- 172.21.15.97:0/3545294764 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880010430 con 0x7f6894105810 2024-03-26T18:58:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.394+0000 7f688b400700 1 -- 172.21.15.97:0/3545294764 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880018680 con 0x7f6894105810 2024-03-26T18:58:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.394+0000 7f6899c00700 1 -- 172.21.15.97:0/3545294764 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68941a26d0 con 0x7f6894105810 2024-03-26T18:58:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.394+0000 7f6899c00700 1 -- 172.21.15.97:0/3545294764 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68941a2c60 con 0x7f6894105810 2024-03-26T18:58:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.395+0000 7f6899c00700 1 -- 172.21.15.97:0/3545294764 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6894069070 con 0x7f6894105810 2024-03-26T18:58:23.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.397+0000 7f688b400700 1 -- 172.21.15.97:0/3545294764 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f68800105e0 con 0x7f6894105810 2024-03-26T18:58:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.398+0000 7f688b400700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f68780a3030 0x7f68780a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.398+0000 7f688b400700 1 -- 172.21.15.97:0/3545294764 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f68800cf370 con 0x7f6894105810 2024-03-26T18:58:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.398+0000 7f6892a00700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f68780a3030 0x7f68780a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.399+0000 7f6892a00700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f68780a3030 0x7f68780a54f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6884005950 tx=0x7f6884009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:23.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.402+0000 7f688b400700 1 -- 172.21.15.97:0/3545294764 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f688009c170 con 0x7f6894105810 2024-03-26T18:58:23.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.653+0000 7f6899c00700 1 -- 172.21.15.97:0/3545294764 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f68940627f0 con 0x7f68780a3030 2024-03-26T18:58:23.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.661+0000 7f688b400700 1 -- 172.21.15.97:0/3545294764 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f68940627f0 con 0x7f68780a3030 2024-03-26T18:58:23.655 INFO:teuthology.orchestra.run.smithi097.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-26T18:58:23.656 INFO:teuthology.orchestra.run.smithi097.stdout:mds.all 12/12 58s ago 2m smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-03-26T18:58:23.656 INFO:teuthology.orchestra.run.smithi097.stdout:mgr 3/3 58s ago 8m smithi097=x;smithi105=y;smithi144=z;count:3 2024-03-26T18:58:23.656 INFO:teuthology.orchestra.run.smithi097.stdout:mon 3/3 58s ago 9m smithi097:172.21.15.97=a;smithi105:172.21.15.105=b;smithi144:172.21.15.144=c;count:3 2024-03-26T18:58:23.656 INFO:teuthology.orchestra.run.smithi097.stdout:osd 12 58s ago - 2024-03-26T18:58:23.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.664+0000 7f6899c00700 1 -- 172.21.15.97:0/3545294764 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f68780a3030 msgr2=0x7f68780a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:23.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.664+0000 7f6899c00700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f68780a3030 0x7f68780a54f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6884005950 tx=0x7f6884009260 comp rx=0 tx=0).stop 2024-03-26T18:58:23.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.664+0000 7f6899c00700 1 -- 172.21.15.97:0/3545294764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6894105810 msgr2=0x7f689419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:23.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.664+0000 7f6899c00700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6894105810 0x7f689419acc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f688000e9f0 tx=0x7f688000ed00 comp rx=0 tx=0).stop 2024-03-26T18:58:23.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.665+0000 7f6899c00700 1 -- 172.21.15.97:0/3545294764 shutdown_connections 2024-03-26T18:58:23.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.665+0000 7f6899c00700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68941061b0 0x7f68941a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:23.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.665+0000 7f6899c00700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6894104610 0x7f689419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:23.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.665+0000 7f6899c00700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f68780a3030 0x7f68780a54f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:23.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.665+0000 7f6899c00700 1 --2- 172.21.15.97:0/3545294764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6894105810 0x7f689419acc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:23.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.665+0000 7f6899c00700 1 -- 172.21.15.97:0/3545294764 >> 172.21.15.97:0/3545294764 conn(0x7f68940ffb70 msgr2=0x7f6894108a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:23.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.665+0000 7f6899c00700 1 -- 172.21.15.97:0/3545294764 shutdown_connections 2024-03-26T18:58:23.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:23.665+0000 7f6899c00700 1 -- 172.21.15.97:0/3545294764 wait complete. 2024-03-26T18:58:24.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:24 smithi097 ceph-mon[31955]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:24 smithi144 ceph-mon[36053]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:24.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:24 smithi105 ceph-mon[36270]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:24.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-26T18:58:24.924 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:58:25.320 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:25 smithi097 ceph-mon[31955]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:25.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:25 smithi144 ceph-mon[36053]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:25.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:25 smithi105 ceph-mon[36270]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:26.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:26 smithi097 ceph-mon[31955]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:58:26.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:26 smithi144 ceph-mon[36053]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:58:26.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:26 smithi105 ceph-mon[36270]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:58:27.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.184+0000 7f199d200700 1 -- 172.21.15.97:0/3859097433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1998074e10 msgr2=0x7f1998075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:27.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.184+0000 7f199d200700 1 --2- 172.21.15.97:0/3859097433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1998074e10 0x7f1998075290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1980009990 tx=0x7f1980009ca0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.184+0000 7f199d200700 1 -- 172.21.15.97:0/3859097433 shutdown_connections 2024-03-26T18:58:27.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.184+0000 7f199d200700 1 --2- 172.21.15.97:0/3859097433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199810e9e0 0x7f1998110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.184+0000 7f199d200700 1 --2- 172.21.15.97:0/3859097433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19980763e0 0x7f1998074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.184+0000 7f199d200700 1 --2- 172.21.15.97:0/3859097433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1998074e10 0x7f1998075290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.184+0000 7f199d200700 1 -- 172.21.15.97:0/3859097433 >> 172.21.15.97:0/3859097433 conn(0x7f19980fd960 msgr2=0x7f19980ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.184+0000 7f199d200700 1 -- 172.21.15.97:0/3859097433 shutdown_connections 2024-03-26T18:58:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.184+0000 7f199d200700 1 -- 172.21.15.97:0/3859097433 wait complete. 2024-03-26T18:58:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.186+0000 7f199d200700 1 Processor -- start 2024-03-26T18:58:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.186+0000 7f199d200700 1 -- start start 2024-03-26T18:58:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f199d200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1998074e10 0x7f199819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f199d200700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19980763e0 0x7f199819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f199d200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199810e9e0 0x7f19981a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f199d200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1998114630 con 0x7f1998074e10 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f199d200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f19981144b0 con 0x7f199810e9e0 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f199d200700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19981147b0 con 0x7f19980763e0 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f198fe00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19980763e0 0x7f199819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f1995e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1998074e10 0x7f199819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f198fe00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19980763e0 0x7f199819f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:47774/0 (socket says 172.21.15.97:47774) 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f198fe00700 1 -- 172.21.15.97:0/331368546 learned_addr learned my addr 172.21.15.97:0/331368546 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f1996800700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199810e9e0 0x7f19981a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f1995e00700 1 -- 172.21.15.97:0/331368546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19980763e0 msgr2=0x7f199819f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f1995e00700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19980763e0 0x7f199819f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f1995e00700 1 -- 172.21.15.97:0/331368546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199810e9e0 msgr2=0x7f19981a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f1995e00700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199810e9e0 0x7f19981a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.187+0000 7f1995e00700 1 -- 172.21.15.97:0/331368546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19981a6910 con 0x7f1998074e10 2024-03-26T18:58:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.188+0000 7f1995e00700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1998074e10 0x7f199819ec90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f198800e9c0 tx=0x7f198800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:27.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.188+0000 7f198ea00700 1 -- 172.21.15.97:0/331368546 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198800c9c0 con 0x7f1998074e10 2024-03-26T18:58:27.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.188+0000 7f198ea00700 1 -- 172.21.15.97:0/331368546 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198800cb60 con 0x7f1998074e10 2024-03-26T18:58:27.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.188+0000 7f198ea00700 1 -- 172.21.15.97:0/331368546 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988018680 con 0x7f1998074e10 2024-03-26T18:58:27.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.188+0000 7f199d200700 1 -- 172.21.15.97:0/331368546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19981a6c00 con 0x7f1998074e10 2024-03-26T18:58:27.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.188+0000 7f199d200700 1 -- 172.21.15.97:0/331368546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19981a70f0 con 0x7f1998074e10 2024-03-26T18:58:27.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.190+0000 7f199d200700 1 -- 172.21.15.97:0/331368546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1998069070 con 0x7f1998074e10 2024-03-26T18:58:27.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.191+0000 7f198ea00700 1 -- 172.21.15.97:0/331368546 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f1988010430 con 0x7f1998074e10 2024-03-26T18:58:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.193+0000 7f198ea00700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f197c0a2fb0 0x7f197c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.193+0000 7f198ea00700 1 -- 172.21.15.97:0/331368546 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f19880cf510 con 0x7f1998074e10 2024-03-26T18:58:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.193+0000 7f198fe00700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f197c0a2fb0 0x7f197c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.194+0000 7f198fe00700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f197c0a2fb0 0x7f197c0a5470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f19981a01a0 tx=0x7f1980005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:27.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.198+0000 7f198ea00700 1 -- 172.21.15.97:0/331368546 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f198809c310 con 0x7f1998074e10 2024-03-26T18:58:27.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.478+0000 7f199d200700 1 -- 172.21.15.97:0/331368546 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f19980628b0 con 0x7f197c0a2fb0 2024-03-26T18:58:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.481+0000 7f198ea00700 1 -- 172.21.15.97:0/331368546 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f19980628b0 con 0x7f197c0a2fb0 2024-03-26T18:58:27.474 INFO:teuthology.orchestra.run.smithi097.stdout:HOST ADDR LABELS STATUS 2024-03-26T18:58:27.474 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 172.21.15.97 2024-03-26T18:58:27.474 INFO:teuthology.orchestra.run.smithi097.stdout:smithi105 172.21.15.105 2024-03-26T18:58:27.475 INFO:teuthology.orchestra.run.smithi097.stdout:smithi144 172.21.15.144 2024-03-26T18:58:27.475 INFO:teuthology.orchestra.run.smithi097.stdout:3 hosts in cluster 2024-03-26T18:58:27.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.484+0000 7f199d200700 1 -- 172.21.15.97:0/331368546 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f197c0a2fb0 msgr2=0x7f197c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:27.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.484+0000 7f199d200700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f197c0a2fb0 0x7f197c0a5470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f19981a01a0 tx=0x7f1980005fd0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.484+0000 7f199d200700 1 -- 172.21.15.97:0/331368546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1998074e10 msgr2=0x7f199819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:27.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.484+0000 7f199d200700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1998074e10 0x7f199819ec90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f198800e9c0 tx=0x7f198800ecd0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.484+0000 7f199d200700 1 -- 172.21.15.97:0/331368546 shutdown_connections 2024-03-26T18:58:27.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.484+0000 7f199d200700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199810e9e0 0x7f19981a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.484+0000 7f199d200700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1998074e10 0x7f199819ec90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.484+0000 7f199d200700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f197c0a2fb0 0x7f197c0a5470 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.484+0000 7f199d200700 1 --2- 172.21.15.97:0/331368546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19980763e0 0x7f199819f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:27.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.484+0000 7f199d200700 1 -- 172.21.15.97:0/331368546 >> 172.21.15.97:0/331368546 conn(0x7f19980fd960 msgr2=0x7f1998101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:27.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.485+0000 7f199d200700 1 -- 172.21.15.97:0/331368546 shutdown_connections 2024-03-26T18:58:27.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:27.485+0000 7f199d200700 1 -- 172.21.15.97:0/331368546 wait complete. 2024-03-26T18:58:28.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:28 smithi097 ceph-mon[31955]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:28.354 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-26T18:58:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:28 smithi144 ceph-mon[36053]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:28.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:28 smithi105 ceph-mon[36270]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:58:29.329 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:29 smithi097 ceph-mon[31955]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:29.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:29 smithi144 ceph-mon[36053]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:29.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:29 smithi105 ceph-mon[36270]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:30.216 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:30 smithi097 ceph-mon[31955]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:30 smithi144 ceph-mon[36053]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:30.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:30 smithi105 ceph-mon[36270]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.095+0000 7fe226200700 1 -- 172.21.15.97:0/1466331607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe220105830 msgr2=0x7fe220105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.095+0000 7fe226200700 1 --2- 172.21.15.97:0/1466331607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe220105830 0x7fe220105c90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009990 tx=0x7fe20c009ca0 comp rx=0 tx=0).stop 2024-03-26T18:58:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.095+0000 7fe226200700 1 -- 172.21.15.97:0/1466331607 shutdown_connections 2024-03-26T18:58:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.095+0000 7fe226200700 1 --2- 172.21.15.97:0/1466331607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2201061d0 0x7fe22010ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.095+0000 7fe226200700 1 --2- 172.21.15.97:0/1466331607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe220104630 0x7fe220104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.095+0000 7fe226200700 1 --2- 172.21.15.97:0/1466331607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe220105830 0x7fe220105c90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.095+0000 7fe226200700 1 -- 172.21.15.97:0/1466331607 >> 172.21.15.97:0/1466331607 conn(0x7fe2200ffbd0 msgr2=0x7fe220102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.095+0000 7fe226200700 1 -- 172.21.15.97:0/1466331607 shutdown_connections 2024-03-26T18:58:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.095+0000 7fe226200700 1 -- 172.21.15.97:0/1466331607 wait complete. 2024-03-26T18:58:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.097+0000 7fe226200700 1 Processor -- start 2024-03-26T18:58:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.097+0000 7fe226200700 1 -- start start 2024-03-26T18:58:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.097+0000 7fe226200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe220104630 0x7fe22019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.097+0000 7fe226200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220105830 0x7fe22019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe226200700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201061d0 0x7fe2201a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe226200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe220110250 con 0x7fe220104630 2024-03-26T18:58:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe21ea00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220105830 0x7fe22019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe21ea00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220105830 0x7fe22019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44314/0 (socket says 172.21.15.97:44314) 2024-03-26T18:58:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe21ea00700 1 -- 172.21.15.97:0/3904906760 learned_addr learned my addr 172.21.15.97:0/3904906760 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe21f400700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe220104630 0x7fe22019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe226200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201100d0 con 0x7fe220105830 2024-03-26T18:58:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201103d0 con 0x7fe2201061d0 2024-03-26T18:58:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe21fe00700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201061d0 0x7fe2201a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:31.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe21ea00700 1 -- 172.21.15.97:0/3904906760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201061d0 msgr2=0x7fe2201a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:31.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe21ea00700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201061d0 0x7fe2201a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:31.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe21ea00700 1 -- 172.21.15.97:0/3904906760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe220104630 msgr2=0x7fe22019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:31.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe21ea00700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe220104630 0x7fe22019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:31.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.098+0000 7fe21ea00700 1 -- 172.21.15.97:0/3904906760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2201a2490 con 0x7fe220105830 2024-03-26T18:58:31.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.099+0000 7fe21ea00700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220105830 0x7fe22019ad70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006010 tx=0x7fe20c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:31.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.099+0000 7fe217e00700 1 -- 172.21.15.97:0/3904906760 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c027070 con 0x7fe220105830 2024-03-26T18:58:31.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.099+0000 7fe217e00700 1 -- 172.21.15.97:0/3904906760 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c00fdb0 con 0x7fe220105830 2024-03-26T18:58:31.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.099+0000 7fe217e00700 1 -- 172.21.15.97:0/3904906760 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c017650 con 0x7fe220105830 2024-03-26T18:58:31.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.099+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2201a2720 con 0x7fe220105830 2024-03-26T18:58:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.099+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2201a2b20 con 0x7fe220105830 2024-03-26T18:58:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.100+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe220069070 con 0x7fe220105830 2024-03-26T18:58:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.102+0000 7fe217e00700 1 -- 172.21.15.97:0/3904906760 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fe20c02a070 con 0x7fe220105830 2024-03-26T18:58:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.103+0000 7fe217e00700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe2040a3030 0x7fe2040a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.103+0000 7fe217e00700 1 -- 172.21.15.97:0/3904906760 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe20c0d04e0 con 0x7fe220105830 2024-03-26T18:58:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.104+0000 7fe21f400700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe2040a3030 0x7fe2040a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:31.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.104+0000 7fe21f400700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe2040a3030 0x7fe2040a54f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe220105690 tx=0x7fe210009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.107+0000 7fe217e00700 1 -- 172.21.15.97:0/3904906760 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe20c05e740 con 0x7fe220105830 2024-03-26T18:58:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.362+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe2200628b0 con 0x7fe2040a3030 2024-03-26T18:58:31.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.365+0000 7fe217e00700 1 -- 172.21.15.97:0/3904906760 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe2200628b0 con 0x7fe2040a3030 2024-03-26T18:58:31.359 INFO:teuthology.orchestra.run.smithi097.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-26T18:58:31.359 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-26T18:58:31.359 INFO:teuthology.orchestra.run.smithi097.stdout:smithi105 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400TR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-26T18:58:31.359 INFO:teuthology.orchestra.run.smithi097.stdout:smithi144 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050085400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-26T18:58:31.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.368+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe2040a3030 msgr2=0x7fe2040a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:31.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.368+0000 7fe226200700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe2040a3030 0x7fe2040a54f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe220105690 tx=0x7fe210009260 comp rx=0 tx=0).stop 2024-03-26T18:58:31.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.368+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220105830 msgr2=0x7fe22019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:31.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.368+0000 7fe226200700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220105830 0x7fe22019ad70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006010 tx=0x7fe20c00f690 comp rx=0 tx=0).stop 2024-03-26T18:58:31.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.368+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 shutdown_connections 2024-03-26T18:58:31.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.368+0000 7fe226200700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe2040a3030 0x7fe2040a54f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:31.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.368+0000 7fe226200700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201061d0 0x7fe2201a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:31.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.368+0000 7fe226200700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe220104630 0x7fe22019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:31.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.368+0000 7fe226200700 1 --2- 172.21.15.97:0/3904906760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220105830 0x7fe22019ad70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:31.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.368+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 >> 172.21.15.97:0/3904906760 conn(0x7fe2200ffbd0 msgr2=0x7fe220108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:31.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.369+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 shutdown_connections 2024-03-26T18:58:31.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:31.369+0000 7fe226200700 1 -- 172.21.15.97:0/3904906760 wait complete. 2024-03-26T18:58:32.091 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T18:58:32.103 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-03-26T18:58:32.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-26T18:58:32.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:32 smithi097 ceph-mon[31955]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:32 smithi144 ceph-mon[36053]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:32.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:32 smithi105 ceph-mon[36270]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:32.534 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:58:33.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:33 smithi097 ceph-mon[31955]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:33.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:33 smithi144 ceph-mon[36053]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:33.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:33 smithi105 ceph-mon[36270]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:34.355 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:34 smithi097 ceph-mon[31955]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:34.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:34 smithi144 ceph-mon[36053]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:34.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:34 smithi105 ceph-mon[36270]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:34.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.635+0000 7f6749a00700 1 -- 172.21.15.97:0/846822515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67440763e0 msgr2=0x7f6744074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:34.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.635+0000 7f6749a00700 1 --2- 172.21.15.97:0/846822515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67440763e0 0x7f6744074840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6734009990 tx=0x7f6734009ca0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.635+0000 7f6749a00700 1 -- 172.21.15.97:0/846822515 shutdown_connections 2024-03-26T18:58:34.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.635+0000 7f6749a00700 1 --2- 172.21.15.97:0/846822515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674410e9e0 0x7f6744110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.635+0000 7f6749a00700 1 --2- 172.21.15.97:0/846822515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67440763e0 0x7f6744074840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.635+0000 7f6749a00700 1 --2- 172.21.15.97:0/846822515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6744074e10 0x7f6744075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.635+0000 7f6749a00700 1 -- 172.21.15.97:0/846822515 >> 172.21.15.97:0/846822515 conn(0x7f67440fd960 msgr2=0x7f67440ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:34.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.635+0000 7f6749a00700 1 -- 172.21.15.97:0/846822515 shutdown_connections 2024-03-26T18:58:34.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.635+0000 7f6749a00700 1 -- 172.21.15.97:0/846822515 wait complete. 2024-03-26T18:58:34.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.636+0000 7f6749a00700 1 Processor -- start 2024-03-26T18:58:34.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6749a00700 1 -- start start 2024-03-26T18:58:34.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6749a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744074e10 0x7f674419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:34.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6749a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440763e0 0x7f674419f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:34.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6749a00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674410e9e0 0x7f67441a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:34.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6749a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6744114610 con 0x7f67440763e0 2024-03-26T18:58:34.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6749a00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6744114490 con 0x7f6744074e10 2024-03-26T18:58:34.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6749a00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6744114790 con 0x7f674410e9e0 2024-03-26T18:58:34.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6742a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744074e10 0x7f674419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:34.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6742a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744074e10 0x7f674419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44334/0 (socket says 172.21.15.97:44334) 2024-03-26T18:58:34.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6742a00700 1 -- 172.21.15.97:0/3147597217 learned_addr learned my addr 172.21.15.97:0/3147597217 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:34.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6742000700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440763e0 0x7f674419f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:34.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.637+0000 7f6743400700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674410e9e0 0x7f67441a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:34.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f6742a00700 1 -- 172.21.15.97:0/3147597217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674410e9e0 msgr2=0x7f67441a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:34.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f6742a00700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674410e9e0 0x7f67441a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f6742a00700 1 -- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440763e0 msgr2=0x7f674419f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:34.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f6742a00700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440763e0 0x7f674419f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f6742a00700 1 -- 172.21.15.97:0/3147597217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67441a68f0 con 0x7f6744074e10 2024-03-26T18:58:34.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f6742000700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440763e0 0x7f674419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:34.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f6742a00700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744074e10 0x7f674419ec70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f673400b7b0 tx=0x7f673400bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:34.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f673b400700 1 -- 172.21.15.97:0/3147597217 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734026070 con 0x7f6744074e10 2024-03-26T18:58:34.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f673b400700 1 -- 172.21.15.97:0/3147597217 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f673400f460 con 0x7f6744074e10 2024-03-26T18:58:34.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f673b400700 1 -- 172.21.15.97:0/3147597217 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734021670 con 0x7f6744074e10 2024-03-26T18:58:34.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.638+0000 7f6749a00700 1 -- 172.21.15.97:0/3147597217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67441a6b80 con 0x7f6744074e10 2024-03-26T18:58:34.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.639+0000 7f6749a00700 1 -- 172.21.15.97:0/3147597217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67441a7030 con 0x7f6744074e10 2024-03-26T18:58:34.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.640+0000 7f6749a00700 1 -- 172.21.15.97:0/3147597217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6744069070 con 0x7f6744074e10 2024-03-26T18:58:34.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.642+0000 7f673b400700 1 -- 172.21.15.97:0/3147597217 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f6734029070 con 0x7f6744074e10 2024-03-26T18:58:34.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.643+0000 7f673b400700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f67280a3030 0x7f67280a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:34.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.643+0000 7f673b400700 1 -- 172.21.15.97:0/3147597217 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f67340d05f0 con 0x7f6744074e10 2024-03-26T18:58:34.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.643+0000 7f6742000700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f67280a3030 0x7f67280a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:34.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.644+0000 7f6742000700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f67280a3030 0x7f67280a54f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67441a0180 tx=0x7f6730009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:34.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.645+0000 7f673b400700 1 -- 172.21.15.97:0/3147597217 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6734013070 con 0x7f6744074e10 2024-03-26T18:58:34.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.935+0000 7f6749a00700 1 -- 172.21.15.97:0/3147597217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f674402dc30 con 0x7f6744074e10 2024-03-26T18:58:34.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.935+0000 7f673b400700 1 -- 172.21.15.97:0/3147597217 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3092 (secure 0 0 0) 0x7f673409d480 con 0x7f6744074e10 2024-03-26T18:58:34.930 INFO:teuthology.orchestra.run.smithi097.stdout:e26 2024-03-26T18:58:34.930 INFO:teuthology.orchestra.run.smithi097.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T18:58:34.930 INFO:teuthology.orchestra.run.smithi097.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:legacy client fscid: 1 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:Filesystem 'cephfs' (1) 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:fs_name cephfs 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 26 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-03-26T18:56:23.642405+0000 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2024-03-26T18:56:31.851339+0000 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:tableserver 0 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:root 0 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:session_timeout 300 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:session_autoclose 300 2024-03-26T18:58:34.931 INFO:teuthology.orchestra.run.smithi097.stdout:max_file_size 1099511627776 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:max_xattr_size 65536 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:required_client_features {} 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:last_failure 0 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:last_failure_osd_epoch 0 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:max_mds 5 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:in 0,1,2,3,4 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:up {0=24581,1=24569,2=24535,3=24494,4=24481} 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:failed 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:damaged 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:stopped 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:data_pools [3] 2024-03-26T18:58:34.932 INFO:teuthology.orchestra.run.smithi097.stdout:metadata_pool 2 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:inline_data disabled 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:balancer 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:bal_rank_mask -1 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:standby_count_wanted 0 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:qdb_cluster leader: 24581 members: 24569,24581,24535,24494,24481 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b{0:24581} state up:active seq 3 addr [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.h{0:24524} state up:standby-replay seq 1 addr [v2:172.21.15.105:6836/1343808654,v1:172.21.15.105:6837/1343808654] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a{1:24569} state up:active seq 5 addr [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c{1:24511} state up:standby-replay seq 1 addr [v2:172.21.15.144:6836/1138636767,v1:172.21.15.144:6837/1138636767] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.i{2:24535} state up:active seq 7 addr [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.f{2:24499} state up:standby-replay seq 1 addr [v2:172.21.15.144:6834/411039946,v1:172.21.15.144:6835/411039946] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.e{3:24494} state up:active seq 10 addr [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.933 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.k{3:24446} state up:standby-replay seq 1 addr [v2:172.21.15.105:6832/254387341,v1:172.21.15.105:6833/254387341] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.934 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d{4:24481} state up:active seq 11 addr [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.934 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.l{4:24439} state up:standby-replay seq 1 addr [v2:172.21.15.144:6832/3099308706,v1:172.21.15.144:6833/3099308706] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:34.934 INFO:teuthology.orchestra.run.smithi097.stdout:Standby daemons: 2024-03-26T18:58:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:34.934 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.j{-1:14385} state up:standby seq 1 addr [v2:172.21.15.97:6834/1485859624,v1:172.21.15.97:6835/1485859624] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.934 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.g{-1:14445} state up:standby seq 1 addr [v2:172.21.15.97:6838/940320803,v1:172.21.15.97:6839/940320803] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:58:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 -- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f67280a3030 msgr2=0x7f67280a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f67280a3030 0x7f67280a54f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67441a0180 tx=0x7f6730009260 comp rx=0 tx=0).stop 2024-03-26T18:58:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 -- 172.21.15.97:0/3147597217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744074e10 msgr2=0x7f674419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744074e10 0x7f674419ec70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f673400b7b0 tx=0x7f673400bbb0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 -- 172.21.15.97:0/3147597217 shutdown_connections 2024-03-26T18:58:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f67280a3030 0x7f67280a54f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674410e9e0 0x7f67441a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744074e10 0x7f674419ec70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 --2- 172.21.15.97:0/3147597217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440763e0 0x7f674419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 -- 172.21.15.97:0/3147597217 >> 172.21.15.97:0/3147597217 conn(0x7f67440fd960 msgr2=0x7f6744101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.939+0000 7f6749a00700 1 -- 172.21.15.97:0/3147597217 shutdown_connections 2024-03-26T18:58:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:34.940+0000 7f6749a00700 1 -- 172.21.15.97:0/3147597217 wait complete. 2024-03-26T18:58:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-03-26T18:58:35.320 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:35 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3147597217' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T18:58:35.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:35 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3147597217' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T18:58:35.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:35 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3147597217' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T18:58:35.780 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-26T18:58:36.213 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:58:36.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:36 smithi097 ceph-mon[31955]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:58:36.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:36 smithi144 ceph-mon[36053]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:58:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:36 smithi105 ceph-mon[36270]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:58:38.208 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:38 smithi097 ceph-mon[31955]: pgmap v381: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:38 smithi144 ceph-mon[36053]: pgmap v381: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:38.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:38 smithi105 ceph-mon[36270]: pgmap v381: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.481+0000 7f159f200700 1 -- 172.21.15.97:0/1681802140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598102400 msgr2=0x7f1598102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.481+0000 7f159f200700 1 --2- 172.21.15.97:0/1681802140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598102400 0x7f1598102820 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1588009990 tx=0x7f1588009ca0 comp rx=0 tx=0).stop 2024-03-26T18:58:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.481+0000 7f159f200700 1 -- 172.21.15.97:0/1681802140 shutdown_connections 2024-03-26T18:58:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.481+0000 7f159f200700 1 --2- 172.21.15.97:0/1681802140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1598103fa0 0x7f159810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.481+0000 7f159f200700 1 --2- 172.21.15.97:0/1681802140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598102400 0x7f1598102820 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.481+0000 7f159f200700 1 --2- 172.21.15.97:0/1681802140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103600 0x7f1598103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:38.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.481+0000 7f159f200700 1 -- 172.21.15.97:0/1681802140 >> 172.21.15.97:0/1681802140 conn(0x7f15980fd960 msgr2=0x7f15980ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:38.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.481+0000 7f159f200700 1 -- 172.21.15.97:0/1681802140 shutdown_connections 2024-03-26T18:58:38.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.481+0000 7f159f200700 1 -- 172.21.15.97:0/1681802140 wait complete. 2024-03-26T18:58:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.482+0000 7f159f200700 1 Processor -- start 2024-03-26T18:58:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f159f200700 1 -- start start 2024-03-26T18:58:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f159f200700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598102400 0x7f1598198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f159f200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103600 0x7f1598198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f159f200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1598103fa0 0x7f159819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f159f200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f159810dfc0 con 0x7f1598103600 2024-03-26T18:58:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f159f200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f159810de40 con 0x7f1598103fa0 2024-03-26T18:58:38.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f159f200700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f159810e140 con 0x7f1598102400 2024-03-26T18:58:38.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f1597e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598102400 0x7f1598198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:38.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f159d200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1598103fa0 0x7f159819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:38.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f1597400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103600 0x7f1598198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:38.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f1597e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598102400 0x7f1598198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:49702/0 (socket says 172.21.15.97:49702) 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.483+0000 7f1597e00700 1 -- 172.21.15.97:0/230620543 learned_addr learned my addr 172.21.15.97:0/230620543 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f1597e00700 1 -- 172.21.15.97:0/230620543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1598103fa0 msgr2=0x7f159819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f1597e00700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1598103fa0 0x7f159819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f1597e00700 1 -- 172.21.15.97:0/230620543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103600 msgr2=0x7f1598198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f1597e00700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103600 0x7f1598198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f1597e00700 1 -- 172.21.15.97:0/230620543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15981a0270 con 0x7f1598102400 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f159d200700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1598103fa0 0x7f159819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f1597e00700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598102400 0x7f1598198610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1588000c00 tx=0x7f158800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f1594c00700 1 -- 172.21.15.97:0/230620543 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588026070 con 0x7f1598102400 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f1594c00700 1 -- 172.21.15.97:0/230620543 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f158800fda0 con 0x7f1598102400 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f159f200700 1 -- 172.21.15.97:0/230620543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15980745e0 con 0x7f1598102400 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.484+0000 7f159f200700 1 -- 172.21.15.97:0/230620543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1598074b10 con 0x7f1598102400 2024-03-26T18:58:38.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.485+0000 7f1594c00700 1 -- 172.21.15.97:0/230620543 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588017650 con 0x7f1598102400 2024-03-26T18:58:38.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.486+0000 7f159f200700 1 -- 172.21.15.97:0/230620543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1578005360 con 0x7f1598102400 2024-03-26T18:58:38.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.488+0000 7f1594c00700 1 -- 172.21.15.97:0/230620543 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f1588029070 con 0x7f1598102400 2024-03-26T18:58:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.490+0000 7f1594c00700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f157c0a3030 0x7f157c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.490+0000 7f1594c00700 1 -- 172.21.15.97:0/230620543 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f15880d0280 con 0x7f1598102400 2024-03-26T18:58:38.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.490+0000 7f1597400700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f157c0a3030 0x7f157c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:38.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.491+0000 7f1597400700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f157c0a3030 0x7f157c0a54f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1584009c60 tx=0x7f1584009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:38.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.493+0000 7f1594c00700 1 -- 172.21.15.97:0/230620543 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f158805d740 con 0x7f1598102400 2024-03-26T18:58:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.781+0000 7f159f200700 1 -- 172.21.15.97:0/230620543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f1578005e60 con 0x7f1598102400 2024-03-26T18:58:38.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.782+0000 7f1594c00700 1 -- 172.21.15.97:0/230620543 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5249 (secure 0 0 0) 0x7f1588013070 con 0x7f1598102400 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 87 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:fsid 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-03-26T18:47:23.209624+0000 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2024-03-26T18:57:30.061624+0000 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:crush_version 26 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:full_ratio 0.95 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:backfillfull_ratio 0.9 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:nearfull_ratio 0.85 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:require_min_compat_client luminous 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:min_compat_client luminous 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:require_osd_release squid 2024-03-26T18:58:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:stretch_mode_enabled false 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:max_osd 12 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6802/932099107,v1:172.21.15.97:6803/932099107] [v2:172.21.15.97:6804/932099107,v1:172.21.15.97:6805/932099107] exists,up bb36143c-3d96-4a50-af1b-e1c3cdbd5681 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6800/3472523969,v1:172.21.15.105:6801/3472523969] [v2:172.21.15.105:6802/3472523969,v1:172.21.15.105:6803/3472523969] exists,up a895a67a-5554-4bad-ab17-6b6971466aae 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/837505140,v1:172.21.15.144:6801/837505140] [v2:172.21.15.144:6802/837505140,v1:172.21.15.144:6803/837505140] exists,up 9f46098e-3956-46ba-8a63-14e6557ef35f 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6810/1821717710,v1:172.21.15.97:6811/1821717710] [v2:172.21.15.97:6812/1821717710,v1:172.21.15.97:6813/1821717710] exists,up 0937aa7b-00ea-417f-b74f-442ca05ac527 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6808/4057159020,v1:172.21.15.105:6809/4057159020] [v2:172.21.15.105:6810/4057159020,v1:172.21.15.105:6811/4057159020] exists,up 3f20c598-4cd5-40d8-befd-e5e50d759bc6 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/356775844,v1:172.21.15.144:6809/356775844] [v2:172.21.15.144:6810/356775844,v1:172.21.15.144:6811/356775844] exists,up d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6818/35447762,v1:172.21.15.97:6819/35447762] [v2:172.21.15.97:6820/35447762,v1:172.21.15.97:6821/35447762] exists,up 9b32d8dc-0311-47c2-a2c4-4e324b70eefd 2024-03-26T18:58:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6816/1025195601,v1:172.21.15.105:6817/1025195601] [v2:172.21.15.105:6818/1025195601,v1:172.21.15.105:6819/1025195601] exists,up cbf3ac56-47cf-4b95-ba5f-5d8e72185128 2024-03-26T18:58:38.780 INFO:teuthology.orchestra.run.smithi097.stdout:osd.8 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6816/3722681651,v1:172.21.15.144:6817/3722681651] [v2:172.21.15.144:6818/3722681651,v1:172.21.15.144:6819/3722681651] exists,up c94dd159-fde0-4285-9b5d-e837afcb8f7f 2024-03-26T18:58:38.780 INFO:teuthology.orchestra.run.smithi097.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6826/2932879599,v1:172.21.15.97:6827/2932879599] [v2:172.21.15.97:6828/2932879599,v1:172.21.15.97:6829/2932879599] exists,up d179a4d5-7751-4d6b-a273-97807334dbe7 2024-03-26T18:58:38.780 INFO:teuthology.orchestra.run.smithi097.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6824/4281982959,v1:172.21.15.105:6825/4281982959] [v2:172.21.15.105:6826/4281982959,v1:172.21.15.105:6827/4281982959] exists,up 6e5fd1b9-0abc-48b2-991a-2e07f7606bfe 2024-03-26T18:58:38.780 INFO:teuthology.orchestra.run.smithi097.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6824/2295652267,v1:172.21.15.144:6825/2295652267] [v2:172.21.15.144:6826/2295652267,v1:172.21.15.144:6827/2295652267] exists,up 2347144e-d823-4816-ac8c-65722dd9cc62 2024-03-26T18:58:38.780 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.0 [8,4] 2024-03-26T18:58:38.780 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.a [2,11] 2024-03-26T18:58:38.781 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.10 [8,11] 2024-03-26T18:58:38.781 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.1d [2,4] 2024-03-26T18:58:38.781 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.10 [5,9] 2024-03-26T18:58:38.781 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.34 [4,9] 2024-03-26T18:58:38.781 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.3c [4,9] 2024-03-26T18:58:38.781 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/668477340 expires 2024-03-27T18:48:28.768371+0000 2024-03-26T18:58:38.782 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/2962857505 expires 2024-03-27T18:48:28.768371+0000 2024-03-26T18:58:38.782 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6800/350003366 expires 2024-03-27T18:47:55.672991+0000 2024-03-26T18:58:38.782 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6800/3462603610 expires 2024-03-27T18:48:28.768371+0000 2024-03-26T18:58:38.782 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3308112091 expires 2024-03-27T18:47:55.672991+0000 2024-03-26T18:58:38.782 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6801/3462603610 expires 2024-03-27T18:48:28.768371+0000 2024-03-26T18:58:38.782 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/1051395080 expires 2024-03-27T18:48:28.768371+0000 2024-03-26T18:58:38.782 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3851585874 expires 2024-03-27T18:47:55.672991+0000 2024-03-26T18:58:38.782 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6801/350003366 expires 2024-03-27T18:47:55.672991+0000 2024-03-26T18:58:38.782 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/2299096028 expires 2024-03-27T18:47:55.672991+0000 2024-03-26T18:58:38.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 -- 172.21.15.97:0/230620543 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f157c0a3030 msgr2=0x7f157c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:38.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f157c0a3030 0x7f157c0a54f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1584009c60 tx=0x7f1584009260 comp rx=0 tx=0).stop 2024-03-26T18:58:38.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 -- 172.21.15.97:0/230620543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598102400 msgr2=0x7f1598198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:38.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598102400 0x7f1598198610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1588000c00 tx=0x7f158800f690 comp rx=0 tx=0).stop 2024-03-26T18:58:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 -- 172.21.15.97:0/230620543 shutdown_connections 2024-03-26T18:58:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1598103fa0 0x7f159819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f157c0a3030 0x7f157c0a54f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598102400 0x7f1598198610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 --2- 172.21.15.97:0/230620543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103600 0x7f1598198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 -- 172.21.15.97:0/230620543 >> 172.21.15.97:0/230620543 conn(0x7f15980fd960 msgr2=0x7f1598106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 -- 172.21.15.97:0/230620543 shutdown_connections 2024-03-26T18:58:38.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:38.786+0000 7f159f200700 1 -- 172.21.15.97:0/230620543 wait complete. 2024-03-26T18:58:39.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:39 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/230620543' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T18:58:39.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:39 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/230620543' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T18:58:39.568 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:39 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/230620543' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T18:58:39.620 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-26T18:58:39.632 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:58:39.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.856+0000 7fcb64a00640 1 -- 172.21.15.97:0/2643692266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb60102230 msgr2=0x7fcb60104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.856+0000 7fcb64a00640 1 --2- 172.21.15.97:0/2643692266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb60102230 0x7fcb60104620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009900 tx=0x7fcb5002bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.856+0000 7fcb64a00640 1 -- 172.21.15.97:0/2643692266 shutdown_connections 2024-03-26T18:58:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.856+0000 7fcb64a00640 1 --2- 172.21.15.97:0/2643692266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb60107490 0x7fcb60109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.856+0000 7fcb64a00640 1 --2- 172.21.15.97:0/2643692266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb60104b60 0x7fcb60106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.856+0000 7fcb64a00640 1 --2- 172.21.15.97:0/2643692266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb60102230 0x7fcb60104620 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.856+0000 7fcb64a00640 1 -- 172.21.15.97:0/2643692266 >> 172.21.15.97:0/2643692266 conn(0x7fcb600fbba0 msgr2=0x7fcb600fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.856+0000 7fcb64a00640 1 -- 172.21.15.97:0/2643692266 shutdown_connections 2024-03-26T18:58:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.857+0000 7fcb64a00640 1 -- 172.21.15.97:0/2643692266 wait complete. 2024-03-26T18:58:39.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.858+0000 7fcb64a00640 1 Processor -- start 2024-03-26T18:58:39.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.858+0000 7fcb64a00640 1 -- start start 2024-03-26T18:58:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.858+0000 7fcb64a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb60102230 0x7fcb60101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb64a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb60104b60 0x7fcb601001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb64a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb60107490 0x7fcb60100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb64a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6010d3f0 con 0x7fcb60104b60 2024-03-26T18:58:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb64a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6010d270 con 0x7fcb60107490 2024-03-26T18:58:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb64a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6010d570 con 0x7fcb60102230 2024-03-26T18:58:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb5f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb60107490 0x7fcb60100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb5f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb60107490 0x7fcb60100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44380/0 (socket says 172.21.15.97:44380) 2024-03-26T18:58:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb5f200640 1 -- 172.21.15.97:0/1130512516 learned_addr learned my addr 172.21.15.97:0/1130512516 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:39.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb5de00640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb60104b60 0x7fcb601001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:39.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb5f200640 1 -- 172.21.15.97:0/1130512516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb60102230 msgr2=0x7fcb60101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:39.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb5f200640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb60102230 0x7fcb60101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:39.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb5f200640 1 -- 172.21.15.97:0/1130512516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb60104b60 msgr2=0x7fcb601001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb5f200640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb60104b60 0x7fcb601001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.859+0000 7fcb5f200640 1 -- 172.21.15.97:0/1130512516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb60100fb0 con 0x7fcb60107490 2024-03-26T18:58:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.860+0000 7fcb5f200640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb60107490 0x7fcb60100720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800eb10 tx=0x7fcb5800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.860+0000 7fcb57400640 1 -- 172.21.15.97:0/1130512516 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58004b20 con 0x7fcb60107490 2024-03-26T18:58:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.860+0000 7fcb57400640 1 -- 172.21.15.97:0/1130512516 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb58004cc0 con 0x7fcb60107490 2024-03-26T18:58:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.860+0000 7fcb64a00640 1 -- 172.21.15.97:0/1130512516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb601a1b30 con 0x7fcb60107490 2024-03-26T18:58:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.860+0000 7fcb57400640 1 -- 172.21.15.97:0/1130512516 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58010760 con 0x7fcb60107490 2024-03-26T18:58:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.860+0000 7fcb64a00640 1 -- 172.21.15.97:0/1130512516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb601a20d0 con 0x7fcb60107490 2024-03-26T18:58:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.862+0000 7fcb64a00640 1 -- 172.21.15.97:0/1130512516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb600fd690 con 0x7fcb60107490 2024-03-26T18:58:39.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.865+0000 7fcb57400640 1 -- 172.21.15.97:0/1130512516 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fcb58005020 con 0x7fcb60107490 2024-03-26T18:58:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.866+0000 7fcb57400640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb440a2fa0 0x7fcb440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.867+0000 7fcb57400640 1 -- 172.21.15.97:0/1130512516 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcb580cfad0 con 0x7fcb60107490 2024-03-26T18:58:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.867+0000 7fcb5e800640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb440a2fa0 0x7fcb440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.867+0000 7fcb5e800640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb440a2fa0 0x7fcb440a5460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcb500057c0 tx=0x7fcb50000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:39.870+0000 7fcb57400640 1 -- 172.21.15.97:0/1130512516 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcb5809c930 con 0x7fcb60107490 2024-03-26T18:58:40.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.113+0000 7fcb64a00640 1 -- 172.21.15.97:0/1130512516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb600624c0 con 0x7fcb60107490 2024-03-26T18:58:40.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.115+0000 7fcb57400640 1 -- 172.21.15.97:0/1130512516 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22581 (secure 0 0 0) 0x7fcb5809c230 con 0x7fcb60107490 2024-03-26T18:58:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:40.109 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:57:30.061624+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:58:40.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 -- 172.21.15.97:0/1130512516 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb440a2fa0 msgr2=0x7fcb440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb440a2fa0 0x7fcb440a5460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcb500057c0 tx=0x7fcb50000bc0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 -- 172.21.15.97:0/1130512516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb60107490 msgr2=0x7fcb60100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb60107490 0x7fcb60100720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800eb10 tx=0x7fcb5800c2d0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 -- 172.21.15.97:0/1130512516 shutdown_connections 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcb440a2fa0 0x7fcb440a5460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb60107490 0x7fcb60100720 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb60104b60 0x7fcb601001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 --2- 172.21.15.97:0/1130512516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb60102230 0x7fcb60101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 -- 172.21.15.97:0/1130512516 >> 172.21.15.97:0/1130512516 conn(0x7fcb600fbba0 msgr2=0x7fcb60102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 -- 172.21.15.97:0/1130512516 shutdown_connections 2024-03-26T18:58:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.118+0000 7fcb64a00640 1 -- 172.21.15.97:0/1130512516 wait complete. 2024-03-26T18:58:40.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T18:51:11.459500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T18:56:22.069048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T18:56:23.116030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T18:58:40.135 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:58:40.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.344+0000 7ff8f8a00640 1 -- 172.21.15.97:0/4285923739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8f41075a0 msgr2=0x7ff8f41099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:40.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.344+0000 7ff8f8a00640 1 --2- 172.21.15.97:0/4285923739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8f41075a0 0x7ff8f41099f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009ca0 tx=0x7ff8ec02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.344+0000 7ff8f8a00640 1 -- 172.21.15.97:0/4285923739 shutdown_connections 2024-03-26T18:58:40.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.344+0000 7ff8f8a00640 1 --2- 172.21.15.97:0/4285923739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8f41075a0 0x7ff8f41099f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.344+0000 7ff8f8a00640 1 --2- 172.21.15.97:0/4285923739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4104c70 0x7ff8f4107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.344+0000 7ff8f8a00640 1 --2- 172.21.15.97:0/4285923739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f4102340 0x7ff8f4104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.344+0000 7ff8f8a00640 1 -- 172.21.15.97:0/4285923739 >> 172.21.15.97:0/4285923739 conn(0x7ff8f40fbd30 msgr2=0x7ff8f40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:40.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.344+0000 7ff8f8a00640 1 -- 172.21.15.97:0/4285923739 shutdown_connections 2024-03-26T18:58:40.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.344+0000 7ff8f8a00640 1 -- 172.21.15.97:0/4285923739 wait complete. 2024-03-26T18:58:40.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.345+0000 7ff8f8a00640 1 Processor -- start 2024-03-26T18:58:40.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.345+0000 7ff8f8a00640 1 -- start start 2024-03-26T18:58:40.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.345+0000 7ff8f8a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8f4102340 0x7ff8f4199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:40.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4104c70 0x7ff8f419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:40.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f41075a0 0x7ff8f41a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:40.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f8a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f410d560 con 0x7ff8f41075a0 2024-03-26T18:58:40.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f8a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f410d3e0 con 0x7ff8f4104c70 2024-03-26T18:58:40.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f8a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f410d6e0 con 0x7ff8f4102340 2024-03-26T18:58:40.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8f4102340 0x7ff8f4199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8f4102340 0x7ff8f4199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:49718/0 (socket says 172.21.15.97:49718) 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1e00640 1 -- 172.21.15.97:0/1429352057 learned_addr learned my addr 172.21.15.97:0/1429352057 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4104c70 0x7ff8f419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f2800640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f41075a0 0x7ff8f41a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1e00640 1 -- 172.21.15.97:0/1429352057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4104c70 msgr2=0x7ff8f419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1e00640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4104c70 0x7ff8f419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1e00640 1 -- 172.21.15.97:0/1429352057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f41075a0 msgr2=0x7ff8f41a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1e00640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f41075a0 0x7ff8f41a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1e00640 1 -- 172.21.15.97:0/1429352057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f41a19c0 con 0x7ff8f4102340 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1400640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4104c70 0x7ff8f419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.346+0000 7ff8f1e00640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8f4102340 0x7ff8f4199db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400e9f0 tx=0x7ff8e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.347+0000 7ff8eaa00640 1 -- 172.21.15.97:0/1429352057 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400cd80 con 0x7ff8f4102340 2024-03-26T18:58:40.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.347+0000 7ff8eaa00640 1 -- 172.21.15.97:0/1429352057 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e4010910 con 0x7ff8f4102340 2024-03-26T18:58:40.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.347+0000 7ff8f8a00640 1 -- 172.21.15.97:0/1429352057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f41a1cb0 con 0x7ff8f4102340 2024-03-26T18:58:40.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.347+0000 7ff8eaa00640 1 -- 172.21.15.97:0/1429352057 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4018990 con 0x7ff8f4102340 2024-03-26T18:58:40.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.347+0000 7ff8f8a00640 1 -- 172.21.15.97:0/1429352057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f41a21a0 con 0x7ff8f4102340 2024-03-26T18:58:40.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.348+0000 7ff8f8a00640 1 -- 172.21.15.97:0/1429352057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f4109090 con 0x7ff8f4102340 2024-03-26T18:58:40.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.350+0000 7ff8eaa00640 1 -- 172.21.15.97:0/1429352057 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7ff8e4010ac0 con 0x7ff8f4102340 2024-03-26T18:58:40.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.352+0000 7ff8eaa00640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8d80a2fa0 0x7ff8d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:40.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.352+0000 7ff8eaa00640 1 -- 172.21.15.97:0/1429352057 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff8e40cf710 con 0x7ff8f4102340 2024-03-26T18:58:40.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.352+0000 7ff8f1400640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8d80a2fa0 0x7ff8d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.352+0000 7ff8f1400640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8d80a2fa0 0x7ff8d80a5460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff8f406a9f0 tx=0x7ff8e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:40.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.355+0000 7ff8eaa00640 1 -- 172.21.15.97:0/1429352057 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff8e409bec0 con 0x7ff8f4102340 2024-03-26T18:58:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:40 smithi144 ceph-mon[36053]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:40 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1130512516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:40 smithi105 ceph-mon[36270]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:40 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1130512516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:40.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.551+0000 7ff8f8a00640 1 -- 172.21.15.97:0/1429352057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8f40625d0 con 0x7ff8f4102340 2024-03-26T18:58:40.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.551+0000 7ff8eaa00640 1 -- 172.21.15.97:0/1429352057 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7ff8e409ff60 con 0x7ff8f4102340 2024-03-26T18:58:40.545 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-26T18:58:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 -- 172.21.15.97:0/1429352057 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8d80a2fa0 msgr2=0x7ff8d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8d80a2fa0 0x7ff8d80a5460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff8f406a9f0 tx=0x7ff8e0005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 -- 172.21.15.97:0/1429352057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8f4102340 msgr2=0x7ff8f4199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8f4102340 0x7ff8f4199db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400e9f0 tx=0x7ff8e400eeb0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 -- 172.21.15.97:0/1429352057 shutdown_connections 2024-03-26T18:58:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8d80a2fa0 0x7ff8d80a5460 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8f41075a0 0x7ff8f41a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4104c70 0x7ff8f419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 --2- 172.21.15.97:0/1429352057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8f4102340 0x7ff8f4199db0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 -- 172.21.15.97:0/1429352057 >> 172.21.15.97:0/1429352057 conn(0x7ff8f40fbd30 msgr2=0x7ff8f41019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 -- 172.21.15.97:0/1429352057 shutdown_connections 2024-03-26T18:58:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.554+0000 7ff8f8a00640 1 -- 172.21.15.97:0/1429352057 wait complete. 2024-03-26T18:58:40.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:40 smithi097 ceph-mon[31955]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:40.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:40 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1130512516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:40.567 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:58:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.773+0000 7f7efbe00640 1 -- 172.21.15.97:0/1363218918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ef41019b0 msgr2=0x7f7ef4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.773+0000 7f7efbe00640 1 --2- 172.21.15.97:0/1363218918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ef41019b0 0x7f7ef4101e30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009900 tx=0x7f7ee002bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.773+0000 7f7efbe00640 1 -- 172.21.15.97:0/1363218918 shutdown_connections 2024-03-26T18:58:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.773+0000 7f7efbe00640 1 --2- 172.21.15.97:0/1363218918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef4102370 0x7f7ef4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.773+0000 7f7efbe00640 1 --2- 172.21.15.97:0/1363218918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ef41019b0 0x7f7ef4101e30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.773+0000 7f7efbe00640 1 --2- 172.21.15.97:0/1363218918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ef41007c0 0x7f7ef4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.773+0000 7f7efbe00640 1 -- 172.21.15.97:0/1363218918 >> 172.21.15.97:0/1363218918 conn(0x7f7ef40fbd30 msgr2=0x7f7ef40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.773+0000 7f7efbe00640 1 -- 172.21.15.97:0/1363218918 shutdown_connections 2024-03-26T18:58:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.773+0000 7f7efbe00640 1 -- 172.21.15.97:0/1363218918 wait complete. 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.774+0000 7f7efbe00640 1 Processor -- start 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.774+0000 7f7efbe00640 1 -- start start 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7efbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ef41007c0 0x7f7ef4104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7efbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef41019b0 0x7f7ef4102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7efbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ef4102370 0x7f7ef41032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7efbe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef410c440 con 0x7f7ef41019b0 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7efbe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef410c2c0 con 0x7f7ef41007c0 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7efbe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef410c5c0 con 0x7f7ef4102370 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7ef9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ef41007c0 0x7f7ef4104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7ef9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ef41007c0 0x7f7ef4104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44428/0 (socket says 172.21.15.97:44428) 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7ef9400640 1 -- 172.21.15.97:0/3201766305 learned_addr learned my addr 172.21.15.97:0/3201766305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7ef8a00640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef41019b0 0x7f7ef4102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7ef9e00640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ef4102370 0x7f7ef41032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7ef9400640 1 -- 172.21.15.97:0/3201766305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ef4102370 msgr2=0x7f7ef41032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7ef9400640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ef4102370 0x7f7ef41032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7ef9400640 1 -- 172.21.15.97:0/3201766305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef41019b0 msgr2=0x7f7ef4102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7ef9400640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef41019b0 0x7f7ef4102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.775+0000 7f7ef9400640 1 -- 172.21.15.97:0/3201766305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef4103b60 con 0x7f7ef41007c0 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.776+0000 7f7ef9400640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ef41007c0 0x7f7ef4104750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800e9c0 tx=0x7f7ee800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.776+0000 7f7ef2000640 1 -- 172.21.15.97:0/3201766305 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800cd80 con 0x7f7ef41007c0 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.776+0000 7f7ef2000640 1 -- 172.21.15.97:0/3201766305 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee80035f0 con 0x7f7ef41007c0 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.776+0000 7f7efbe00640 1 -- 172.21.15.97:0/3201766305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef41a1d80 con 0x7f7ef41007c0 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.776+0000 7f7ef2000640 1 -- 172.21.15.97:0/3201766305 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8010690 con 0x7f7ef41007c0 2024-03-26T18:58:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.776+0000 7f7efbe00640 1 -- 172.21.15.97:0/3201766305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef41a2190 con 0x7f7ef41007c0 2024-03-26T18:58:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.777+0000 7f7efbe00640 1 -- 172.21.15.97:0/3201766305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed4005290 con 0x7f7ef41007c0 2024-03-26T18:58:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.780+0000 7f7ef2000640 1 -- 172.21.15.97:0/3201766305 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f7ee8004000 con 0x7f7ef41007c0 2024-03-26T18:58:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.782+0000 7f7ef2000640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7edc0a2fa0 0x7f7edc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.782+0000 7f7ef2000640 1 -- 172.21.15.97:0/3201766305 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7ee80cf6b0 con 0x7f7ef41007c0 2024-03-26T18:58:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.783+0000 7f7ef8a00640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7edc0a2fa0 0x7f7edc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.783+0000 7f7ef8a00640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7edc0a2fa0 0x7f7edc0a5460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ee002c7b0 tx=0x7f7ee0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:40.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:40.785+0000 7f7ef2000640 1 -- 172.21.15.97:0/3201766305 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7ee809c4b0 con 0x7f7ef41007c0 2024-03-26T18:58:41.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.086+0000 7f7efbe00640 1 -- 172.21.15.97:0/3201766305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7ed4005d90 con 0x7f7ef41007c0 2024-03-26T18:58:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.086+0000 7f7ef2000640 1 -- 172.21.15.97:0/3201766305 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f7ee809bdb0 con 0x7f7ef41007c0 2024-03-26T18:58:41.080 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-26T18:58:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 -- 172.21.15.97:0/3201766305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7edc0a2fa0 msgr2=0x7f7edc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7edc0a2fa0 0x7f7edc0a5460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ee002c7b0 tx=0x7f7ee0005fd0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 -- 172.21.15.97:0/3201766305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ef41007c0 msgr2=0x7f7ef4104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ef41007c0 0x7f7ef4104750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800e9c0 tx=0x7f7ee800ee80 comp rx=0 tx=0).stop 2024-03-26T18:58:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 -- 172.21.15.97:0/3201766305 shutdown_connections 2024-03-26T18:58:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7edc0a2fa0 0x7f7edc0a5460 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ef4102370 0x7f7ef41032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef41019b0 0x7f7ef4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 --2- 172.21.15.97:0/3201766305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ef41007c0 0x7f7ef4104750 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 -- 172.21.15.97:0/3201766305 >> 172.21.15.97:0/3201766305 conn(0x7f7ef40fbd30 msgr2=0x7f7ef40fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 -- 172.21.15.97:0/3201766305 shutdown_connections 2024-03-26T18:58:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.089+0000 7f7efbe00640 1 -- 172.21.15.97:0/3201766305 wait complete. 2024-03-26T18:58:41.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:58:41.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.322+0000 7f0775800640 1 -- 172.21.15.97:0/3858829751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0770102220 msgr2=0x7f0770104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:41.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.322+0000 7f0775800640 1 --2- 172.21.15.97:0/3858829751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0770102220 0x7f0770104610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0760009900 tx=0x7f076002bc40 comp rx=0 tx=0).stop 2024-03-26T18:58:41.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.322+0000 7f0775800640 1 -- 172.21.15.97:0/3858829751 shutdown_connections 2024-03-26T18:58:41.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.322+0000 7f0775800640 1 --2- 172.21.15.97:0/3858829751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0770107480 0x7f07701098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.322+0000 7f0775800640 1 --2- 172.21.15.97:0/3858829751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0770104b50 0x7f0770106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.322+0000 7f0775800640 1 --2- 172.21.15.97:0/3858829751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0770102220 0x7f0770104610 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.322+0000 7f0775800640 1 -- 172.21.15.97:0/3858829751 >> 172.21.15.97:0/3858829751 conn(0x7f07700fbc30 msgr2=0x7f07700fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.322+0000 7f0775800640 1 -- 172.21.15.97:0/3858829751 shutdown_connections 2024-03-26T18:58:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.322+0000 7f0775800640 1 -- 172.21.15.97:0/3858829751 wait complete. 2024-03-26T18:58:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.323+0000 7f0775800640 1 Processor -- start 2024-03-26T18:58:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f0775800640 1 -- start start 2024-03-26T18:58:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f0775800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0770102220 0x7f0770199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f0775800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0770104b50 0x7f077019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f0775800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0770107480 0x7f07701a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f0775800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f077010d490 con 0x7f0770104b50 2024-03-26T18:58:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f0775800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f077010d310 con 0x7f0770102220 2024-03-26T18:58:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f0775800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f077010d610 con 0x7f0770107480 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f076fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0770102220 0x7f0770199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f076fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0770102220 0x7f0770199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:43954/0 (socket says 172.21.15.97:43954) 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f076fe00640 1 -- 172.21.15.97:0/451937274 learned_addr learned my addr 172.21.15.97:0/451937274 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f0774e00640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0770107480 0x7f07701a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.324+0000 7f076f400640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0770104b50 0x7f077019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f076fe00640 1 -- 172.21.15.97:0/451937274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0770107480 msgr2=0x7f07701a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f076fe00640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0770107480 0x7f07701a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f076fe00640 1 -- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0770104b50 msgr2=0x7f077019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f076fe00640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0770104b50 0x7f077019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f076fe00640 1 -- 172.21.15.97:0/451937274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07701a1910 con 0x7f0770102220 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f076f400640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0770104b50 0x7f077019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f076fe00640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0770102220 0x7f0770199d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f076002c8f0 tx=0x7f0760005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f076cc00640 1 -- 172.21.15.97:0/451937274 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760041070 con 0x7f0770102220 2024-03-26T18:58:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f076cc00640 1 -- 172.21.15.97:0/451937274 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07600054e0 con 0x7f0770102220 2024-03-26T18:58:41.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f0775800640 1 -- 172.21.15.97:0/451937274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07701a1ba0 con 0x7f0770102220 2024-03-26T18:58:41.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f076cc00640 1 -- 172.21.15.97:0/451937274 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760034440 con 0x7f0770102220 2024-03-26T18:58:41.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.325+0000 7f0775800640 1 -- 172.21.15.97:0/451937274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07701a2060 con 0x7f0770102220 2024-03-26T18:58:41.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.327+0000 7f0775800640 1 -- 172.21.15.97:0/451937274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0770108fd0 con 0x7f0770102220 2024-03-26T18:58:41.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.328+0000 7f076cc00640 1 -- 172.21.15.97:0/451937274 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f0760044070 con 0x7f0770102220 2024-03-26T18:58:41.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.330+0000 7f076cc00640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07540a2f20 0x7f07540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:41.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.330+0000 7f076cc00640 1 -- 172.21.15.97:0/451937274 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f07600f1160 con 0x7f0770102220 2024-03-26T18:58:41.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.331+0000 7f076f400640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07540a2f20 0x7f07540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:41.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.331+0000 7f076f400640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07540a2f20 0x7f07540a53e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f075c0058c0 tx=0x7f075c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:41.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.337+0000 7f076cc00640 1 -- 172.21.15.97:0/451937274 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f07600bdf60 con 0x7f0770102220 2024-03-26T18:58:41.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:41 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1429352057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:41.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:41 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3201766305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:41 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1429352057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:41.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:41 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3201766305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:41.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:41 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1429352057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:41.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:41 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3201766305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:41.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.652+0000 7f0775800640 1 -- 172.21.15.97:0/451937274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f077002d240 con 0x7f0770102220 2024-03-26T18:58:41.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.653+0000 7f076cc00640 1 -- 172.21.15.97:0/451937274 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f07600bd860 con 0x7f0770102220 2024-03-26T18:58:41.647 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-26T18:58:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 -- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07540a2f20 msgr2=0x7f07540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07540a2f20 0x7f07540a53e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f075c0058c0 tx=0x7f075c005850 comp rx=0 tx=0).stop 2024-03-26T18:58:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 -- 172.21.15.97:0/451937274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0770102220 msgr2=0x7f0770199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0770102220 0x7f0770199d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f076002c8f0 tx=0x7f0760005e40 comp rx=0 tx=0).stop 2024-03-26T18:58:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 -- 172.21.15.97:0/451937274 shutdown_connections 2024-03-26T18:58:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f07540a2f20 0x7f07540a53e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0770107480 0x7f07701a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0770104b50 0x7f077019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 --2- 172.21.15.97:0/451937274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0770102220 0x7f0770199d00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 -- 172.21.15.97:0/451937274 >> 172.21.15.97:0/451937274 conn(0x7f07700fbc30 msgr2=0x7f0770101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:41.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 -- 172.21.15.97:0/451937274 shutdown_connections 2024-03-26T18:58:41.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.656+0000 7f0775800640 1 -- 172.21.15.97:0/451937274 wait complete. 2024-03-26T18:58:41.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:58:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.880+0000 7f3d0f200640 1 -- 172.21.15.97:0/3790689153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d080758a0 msgr2=0x7f3d08073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.880+0000 7f3d0f200640 1 --2- 172.21.15.97:0/3790689153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d080758a0 0x7f3d08073cf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009900 tx=0x7f3cfc02bcd0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.880+0000 7f3d0f200640 1 -- 172.21.15.97:0/3790689153 shutdown_connections 2024-03-26T18:58:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.880+0000 7f3d0f200640 1 --2- 172.21.15.97:0/3790689153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d081095b0 0x7f3d0810ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.880+0000 7f3d0f200640 1 --2- 172.21.15.97:0/3790689153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d080742c0 0x7f3d08074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.880+0000 7f3d0f200640 1 --2- 172.21.15.97:0/3790689153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d080758a0 0x7f3d08073cf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.880+0000 7f3d0f200640 1 -- 172.21.15.97:0/3790689153 >> 172.21.15.97:0/3790689153 conn(0x7f3d080fcc90 msgr2=0x7f3d080ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.880+0000 7f3d0f200640 1 -- 172.21.15.97:0/3790689153 shutdown_connections 2024-03-26T18:58:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.880+0000 7f3d0f200640 1 -- 172.21.15.97:0/3790689153 wait complete. 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.881+0000 7f3d0f200640 1 Processor -- start 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0f200640 1 -- start start 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d080742c0 0x7f3d08199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d080758a0 0x7f3d0819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d081095b0 0x7f3d081a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0810f5e0 con 0x7f3d081095b0 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0f200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0810f460 con 0x7f3d080742c0 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0810f760 con 0x7f3d080758a0 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d080742c0 0x7f3d08199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d080758a0 0x7f3d0819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d080742c0 0x7f3d08199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:43970/0 (socket says 172.21.15.97:43970) 2024-03-26T18:58:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0de00640 1 -- 172.21.15.97:0/3182837190 learned_addr learned my addr 172.21.15.97:0/3182837190 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.882+0000 7f3d0e800640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d081095b0 0x7f3d081a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d0de00640 1 -- 172.21.15.97:0/3182837190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d080758a0 msgr2=0x7f3d0819a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d0de00640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d080758a0 0x7f3d0819a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d0de00640 1 -- 172.21.15.97:0/3182837190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d081095b0 msgr2=0x7f3d081a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d0de00640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d081095b0 0x7f3d081a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d0de00640 1 -- 172.21.15.97:0/3182837190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d081a1810 con 0x7f3d080742c0 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d0de00640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d080742c0 0x7f3d08199bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc02c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d02a00640 1 -- 172.21.15.97:0/3182837190 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc043020 con 0x7f3d080742c0 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d02a00640 1 -- 172.21.15.97:0/3182837190 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cfc0054e0 con 0x7f3d080742c0 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d0f200640 1 -- 172.21.15.97:0/3182837190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d081a1aa0 con 0x7f3d080742c0 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d02a00640 1 -- 172.21.15.97:0/3182837190 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc003c20 con 0x7f3d080742c0 2024-03-26T18:58:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.883+0000 7f3d0f200640 1 -- 172.21.15.97:0/3182837190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d081a2040 con 0x7f3d080742c0 2024-03-26T18:58:41.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.884+0000 7f3d0f200640 1 -- 172.21.15.97:0/3182837190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d08108370 con 0x7f3d080742c0 2024-03-26T18:58:41.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.886+0000 7f3d02a00640 1 -- 172.21.15.97:0/3182837190 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f3cfc046020 con 0x7f3d080742c0 2024-03-26T18:58:41.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.887+0000 7f3d02a00640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3cf00a2fa0 0x7f3cf00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:41.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.888+0000 7f3d02a00640 1 -- 172.21.15.97:0/3182837190 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3cfc0f14b0 con 0x7f3d080742c0 2024-03-26T18:58:41.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.888+0000 7f3d0d400640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3cf00a2fa0 0x7f3cf00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:41.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.888+0000 7f3d0d400640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3cf00a2fa0 0x7f3cf00a5460 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3d0819b100 tx=0x7f3cf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:41.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:41.891+0000 7f3d02a00640 1 -- 172.21.15.97:0/3182837190 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3cfc04d020 con 0x7f3d080742c0 2024-03-26T18:58:42.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.189+0000 7f3d0f200640 1 -- 172.21.15.97:0/3182837190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d080624c0 con 0x7f3d080742c0 2024-03-26T18:58:42.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.190+0000 7f3d02a00640 1 -- 172.21.15.97:0/3182837190 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11935 (secure 0 0 0) 0x7f3cfc0be2b0 con 0x7f3d080742c0 2024-03-26T18:58:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:42.184 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T18:56:23.642405+0000","modified":"2024-03-26T18:56:31.851339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24535,"mds_3":24494,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2391073232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2391073232},{"type":"v1","addr":"172.21.15.105:6839","nonce":2391073232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24535,24494,24481]},"id":1}]} 2024-03-26T18:58:42.184 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 -- 172.21.15.97:0/3182837190 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3cf00a2fa0 msgr2=0x7f3cf00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3cf00a2fa0 0x7f3cf00a5460 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3d0819b100 tx=0x7f3cf8005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 -- 172.21.15.97:0/3182837190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d080742c0 msgr2=0x7f3d08199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d080742c0 0x7f3d08199bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc02c6f0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 -- 172.21.15.97:0/3182837190 shutdown_connections 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3cf00a2fa0 0x7f3cf00a5460 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d081095b0 0x7f3d081a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d080758a0 0x7f3d0819a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 --2- 172.21.15.97:0/3182837190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d080742c0 0x7f3d08199bf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 -- 172.21.15.97:0/3182837190 >> 172.21.15.97:0/3182837190 conn(0x7f3d080fcc90 msgr2=0x7f3d08105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 -- 172.21.15.97:0/3182837190 shutdown_connections 2024-03-26T18:58:42.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.194+0000 7f3d0f200640 1 -- 172.21.15.97:0/3182837190 wait complete. 2024-03-26T18:58:42.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:58:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:42 smithi144 ceph-mon[36053]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:42 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/451937274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:42.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.436+0000 7f9eabe00640 1 -- 172.21.15.97:0/3044751174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac1074b0 msgr2=0x7f9eac1098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:42.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.436+0000 7f9eabe00640 1 --2- 172.21.15.97:0/3044751174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac1074b0 0x7f9eac1098a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009ca0 tx=0x7f9ea402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.436+0000 7f9eabe00640 1 -- 172.21.15.97:0/3044751174 shutdown_connections 2024-03-26T18:58:42.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.436+0000 7f9eabe00640 1 --2- 172.21.15.97:0/3044751174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac1074b0 0x7f9eac1098a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.436+0000 7f9eabe00640 1 --2- 172.21.15.97:0/3044751174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eac104b80 0x7f9eac106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.436+0000 7f9eabe00640 1 --2- 172.21.15.97:0/3044751174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eac102250 0x7f9eac104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.436+0000 7f9eabe00640 1 -- 172.21.15.97:0/3044751174 >> 172.21.15.97:0/3044751174 conn(0x7f9eac0fbba0 msgr2=0x7f9eac0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:42.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.436+0000 7f9eabe00640 1 -- 172.21.15.97:0/3044751174 shutdown_connections 2024-03-26T18:58:42.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.436+0000 7f9eabe00640 1 -- 172.21.15.97:0/3044751174 wait complete. 2024-03-26T18:58:42.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.437+0000 7f9eabe00640 1 Processor -- start 2024-03-26T18:58:42.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.437+0000 7f9eabe00640 1 -- start start 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eabe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eac102250 0x7f9eac199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eabe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac104b80 0x7f9eac19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eabe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eac1074b0 0x7f9eac1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eabe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac10d3a0 con 0x7f9eac102250 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eabe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac10d220 con 0x7f9eac1074b0 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eabe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac10d520 con 0x7f9eac104b80 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eaaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eac102250 0x7f9eac199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eaaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eac102250 0x7f9eac199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36728/0 (socket says 172.21.15.97:36728) 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eaa000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac104b80 0x7f9eac19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eaaa00640 1 -- 172.21.15.97:0/1073518153 learned_addr learned my addr 172.21.15.97:0/1073518153 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eab400640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eac1074b0 0x7f9eac1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eaaa00640 1 -- 172.21.15.97:0/1073518153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac104b80 msgr2=0x7f9eac19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eaaa00640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac104b80 0x7f9eac19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eaaa00640 1 -- 172.21.15.97:0/1073518153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eac1074b0 msgr2=0x7f9eac1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:42.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eaaa00640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eac1074b0 0x7f9eac1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eaaa00640 1 -- 172.21.15.97:0/1073518153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac1a1840 con 0x7f9eac102250 2024-03-26T18:58:42.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eab400640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eac1074b0 0x7f9eac1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:42.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.438+0000 7f9eaa000640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac104b80 0x7f9eac19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:42.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.439+0000 7f9eaaa00640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eac102250 0x7f9eac199c30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00e9f0 tx=0x7f9e9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:42.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.439+0000 7f9ea3400640 1 -- 172.21.15.97:0/1073518153 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00cd80 con 0x7f9eac102250 2024-03-26T18:58:42.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.439+0000 7f9ea3400640 1 -- 172.21.15.97:0/1073518153 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c010910 con 0x7f9eac102250 2024-03-26T18:58:42.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.439+0000 7f9eabe00640 1 -- 172.21.15.97:0/1073518153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac1a1b30 con 0x7f9eac102250 2024-03-26T18:58:42.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.439+0000 7f9ea3400640 1 -- 172.21.15.97:0/1073518153 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c018990 con 0x7f9eac102250 2024-03-26T18:58:42.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.440+0000 7f9eabe00640 1 -- 172.21.15.97:0/1073518153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac1a20d0 con 0x7f9eac102250 2024-03-26T18:58:42.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.441+0000 7f9eabe00640 1 -- 172.21.15.97:0/1073518153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eac108fa0 con 0x7f9eac102250 2024-03-26T18:58:42.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.443+0000 7f9ea3400640 1 -- 172.21.15.97:0/1073518153 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f9e9c010ac0 con 0x7f9eac102250 2024-03-26T18:58:42.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.444+0000 7f9ea3400640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e900a2fa0 0x7f9e900a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:42.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.444+0000 7f9ea3400640 1 -- 172.21.15.97:0/1073518153 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9e9c0cf6b0 con 0x7f9eac102250 2024-03-26T18:58:42.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.444+0000 7f9eaa000640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e900a2fa0 0x7f9e900a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:42.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.445+0000 7f9eaa000640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e900a2fa0 0x7f9e900a5460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e980095f0 tx=0x7f9e98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.450+0000 7f9ea3400640 1 -- 172.21.15.97:0/1073518153 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9e9c09bec0 con 0x7f9eac102250 2024-03-26T18:58:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:42 smithi105 ceph-mon[36270]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:42 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/451937274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:42.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:42 smithi097 ceph-mon[31955]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:42.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:42 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/451937274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.706+0000 7f9eabe00640 1 -- 172.21.15.97:0/1073518153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9eac0624f0 con 0x7f9eac102250 2024-03-26T18:58:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.708+0000 7f9ea3400640 1 -- 172.21.15.97:0/1073518153 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22581 (secure 0 0 0) 0x7f9e9c09ff60 con 0x7f9eac102250 2024-03-26T18:58:42.702 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:42.702 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:57:30.061624+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:58:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 -- 172.21.15.97:0/1073518153 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e900a2fa0 msgr2=0x7f9e900a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e900a2fa0 0x7f9e900a5460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e980095f0 tx=0x7f9e98005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 -- 172.21.15.97:0/1073518153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eac102250 msgr2=0x7f9eac199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eac102250 0x7f9eac199c30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00e9f0 tx=0x7f9e9c00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 -- 172.21.15.97:0/1073518153 shutdown_connections 2024-03-26T18:58:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e900a2fa0 0x7f9e900a5460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eac1074b0 0x7f9eac1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac104b80 0x7f9eac19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 --2- 172.21.15.97:0/1073518153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eac102250 0x7f9eac199c30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 -- 172.21.15.97:0/1073518153 >> 172.21.15.97:0/1073518153 conn(0x7f9eac0fbba0 msgr2=0x7f9eac101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.710+0000 7f9eabe00640 1 -- 172.21.15.97:0/1073518153 shutdown_connections 2024-03-26T18:58:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.711+0000 7f9eabe00640 1 -- 172.21.15.97:0/1073518153 wait complete. 2024-03-26T18:58:42.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T18:51:11.459500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T18:56:22.069048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T18:56:23.116030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T18:58:42.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:58:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.939+0000 7f6088000640 1 -- 172.21.15.97:0/3278359216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6080103b00 msgr2=0x7f6080103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.939+0000 7f6088000640 1 --2- 172.21.15.97:0/3278359216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6080103b00 0x7f6080103f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6074009ca0 tx=0x7f607402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.939+0000 7f6088000640 1 -- 172.21.15.97:0/3278359216 shutdown_connections 2024-03-26T18:58:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.939+0000 7f6088000640 1 --2- 172.21.15.97:0/3278359216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60801056b0 0x7f608010bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.939+0000 7f6088000640 1 --2- 172.21.15.97:0/3278359216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080104cf0 0x7f6080105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.939+0000 7f6088000640 1 --2- 172.21.15.97:0/3278359216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6080103b00 0x7f6080103f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.939+0000 7f6088000640 1 -- 172.21.15.97:0/3278359216 >> 172.21.15.97:0/3278359216 conn(0x7f60800ff070 msgr2=0x7f60801014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:42.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.939+0000 7f6088000640 1 -- 172.21.15.97:0/3278359216 shutdown_connections 2024-03-26T18:58:42.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.939+0000 7f6088000640 1 -- 172.21.15.97:0/3278359216 wait complete. 2024-03-26T18:58:42.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.940+0000 7f6088000640 1 Processor -- start 2024-03-26T18:58:42.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.940+0000 7f6088000640 1 -- start start 2024-03-26T18:58:42.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.940+0000 7f6088000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6080103b00 0x7f6080199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.940+0000 7f6088000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6080104cf0 0x7f608019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.940+0000 7f6088000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60801056b0 0x7f60801a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.940+0000 7f6088000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f608010f750 con 0x7f60801056b0 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.940+0000 7f6088000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f608010f5d0 con 0x7f6080103b00 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.940+0000 7f6088000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f608010f8d0 con 0x7f6080104cf0 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.940+0000 7f6085600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6080103b00 0x7f6080199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.940+0000 7f6085600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6080103b00 0x7f6080199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44008/0 (socket says 172.21.15.97:44008) 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6085600640 1 -- 172.21.15.97:0/2979632010 learned_addr learned my addr 172.21.15.97:0/2979632010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6086000640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60801056b0 0x7f60801a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6084c00640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6080104cf0 0x7f608019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6085600640 1 -- 172.21.15.97:0/2979632010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6080104cf0 msgr2=0x7f608019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6085600640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6080104cf0 0x7f608019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6085600640 1 -- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60801056b0 msgr2=0x7f60801a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6085600640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60801056b0 0x7f60801a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6085600640 1 -- 172.21.15.97:0/2979632010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60801a1a50 con 0x7f6080103b00 2024-03-26T18:58:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6086000640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60801056b0 0x7f60801a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6085600640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6080103b00 0x7f6080199e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6074000f80 tx=0x7f6074032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f607e000640 1 -- 172.21.15.97:0/2979632010 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074041020 con 0x7f6080103b00 2024-03-26T18:58:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f6088000640 1 -- 172.21.15.97:0/2979632010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60801a1ce0 con 0x7f6080103b00 2024-03-26T18:58:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f607e000640 1 -- 172.21.15.97:0/2979632010 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6074034cd0 con 0x7f6080103b00 2024-03-26T18:58:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.941+0000 7f607e000640 1 -- 172.21.15.97:0/2979632010 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074039040 con 0x7f6080103b00 2024-03-26T18:58:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.942+0000 7f6088000640 1 -- 172.21.15.97:0/2979632010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60801a21a0 con 0x7f6080103b00 2024-03-26T18:58:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.943+0000 7f6088000640 1 -- 172.21.15.97:0/2979632010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608010aea0 con 0x7f6080103b00 2024-03-26T18:58:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.944+0000 7f607e000640 1 -- 172.21.15.97:0/2979632010 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f6074044020 con 0x7f6080103b00 2024-03-26T18:58:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.946+0000 7f607e000640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f60680a2fa0 0x7f60680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.946+0000 7f607e000640 1 -- 172.21.15.97:0/2979632010 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f60740f1200 con 0x7f6080103b00 2024-03-26T18:58:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.946+0000 7f6084c00640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f60680a2fa0 0x7f60680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.947+0000 7f6084c00640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f60680a2fa0 0x7f60680a5460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f608019b340 tx=0x7f606c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:42.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:42.954+0000 7f607e000640 1 -- 172.21.15.97:0/2979632010 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f607404b020 con 0x7f6080103b00 2024-03-26T18:58:43.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.166+0000 7f6088000640 1 -- 172.21.15.97:0/2979632010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6080062560 con 0x7f6080103b00 2024-03-26T18:58:43.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.166+0000 7f607e000640 1 -- 172.21.15.97:0/2979632010 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f60740348f0 con 0x7f6080103b00 2024-03-26T18:58:43.160 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 -- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f60680a2fa0 msgr2=0x7f60680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f60680a2fa0 0x7f60680a5460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f608019b340 tx=0x7f606c005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 -- 172.21.15.97:0/2979632010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6080103b00 msgr2=0x7f6080199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6080103b00 0x7f6080199e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6074000f80 tx=0x7f6074032ec0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 -- 172.21.15.97:0/2979632010 shutdown_connections 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f60680a2fa0 0x7f60680a5460 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60801056b0 0x7f60801a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6080104cf0 0x7f608019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 --2- 172.21.15.97:0/2979632010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6080103b00 0x7f6080199e30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 -- 172.21.15.97:0/2979632010 >> 172.21.15.97:0/2979632010 conn(0x7f60800ff070 msgr2=0x7f6080108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 -- 172.21.15.97:0/2979632010 shutdown_connections 2024-03-26T18:58:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.169+0000 7f6088000640 1 -- 172.21.15.97:0/2979632010 wait complete. 2024-03-26T18:58:43.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:58:43.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.389+0000 7f5dd8200640 1 -- 172.21.15.97:0/940796089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0104cd0 msgr2=0x7f5dd0105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.389+0000 7f5dd8200640 1 --2- 172.21.15.97:0/940796089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0104cd0 0x7f5dd0105150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009900 tx=0x7f5dbc02bc40 comp rx=0 tx=0).stop 2024-03-26T18:58:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.389+0000 7f5dd8200640 1 -- 172.21.15.97:0/940796089 shutdown_connections 2024-03-26T18:58:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.390+0000 7f5dd8200640 1 --2- 172.21.15.97:0/940796089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd0105690 0x7f5dd010bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.390+0000 7f5dd8200640 1 --2- 172.21.15.97:0/940796089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0104cd0 0x7f5dd0105150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.390+0000 7f5dd8200640 1 --2- 172.21.15.97:0/940796089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dd0103ae0 0x7f5dd0103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.390+0000 7f5dd8200640 1 -- 172.21.15.97:0/940796089 >> 172.21.15.97:0/940796089 conn(0x7f5dd00ff070 msgr2=0x7f5dd0101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.390+0000 7f5dd8200640 1 -- 172.21.15.97:0/940796089 shutdown_connections 2024-03-26T18:58:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.390+0000 7f5dd8200640 1 -- 172.21.15.97:0/940796089 wait complete. 2024-03-26T18:58:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.391+0000 7f5dd8200640 1 Processor -- start 2024-03-26T18:58:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.391+0000 7f5dd8200640 1 -- start start 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd8200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dd0103ae0 0x7f5dd0199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd8200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd0104cd0 0x7f5dd019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd8200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0105690 0x7f5dd01a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd8200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd010f7b0 con 0x7f5dd0105690 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd8200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd010f630 con 0x7f5dd0103ae0 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd8200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd010f930 con 0x7f5dd0104cd0 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd5800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dd0103ae0 0x7f5dd0199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd5800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dd0103ae0 0x7f5dd0199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44030/0 (socket says 172.21.15.97:44030) 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd5800640 1 -- 172.21.15.97:0/954048978 learned_addr learned my addr 172.21.15.97:0/954048978 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd6200640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0105690 0x7f5dd01a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd4e00640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd0104cd0 0x7f5dd019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd5800640 1 -- 172.21.15.97:0/954048978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd0104cd0 msgr2=0x7f5dd019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd5800640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd0104cd0 0x7f5dd019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd5800640 1 -- 172.21.15.97:0/954048978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0105690 msgr2=0x7f5dd01a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:43.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd5800640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0105690 0x7f5dd01a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.392+0000 7f5dd5800640 1 -- 172.21.15.97:0/954048978 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd01a19e0 con 0x7f5dd0103ae0 2024-03-26T18:58:43.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.393+0000 7f5dd5800640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dd0103ae0 0x7f5dd0199dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400e9c0 tx=0x7f5dc400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:43.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.393+0000 7f5dce000640 1 -- 172.21.15.97:0/954048978 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc400cde0 con 0x7f5dd0103ae0 2024-03-26T18:58:43.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.393+0000 7f5dce000640 1 -- 172.21.15.97:0/954048978 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc4010910 con 0x7f5dd0103ae0 2024-03-26T18:58:43.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.393+0000 7f5dd8200640 1 -- 172.21.15.97:0/954048978 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd01a1cd0 con 0x7f5dd0103ae0 2024-03-26T18:58:43.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.393+0000 7f5dce000640 1 -- 172.21.15.97:0/954048978 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4018a00 con 0x7f5dd0103ae0 2024-03-26T18:58:43.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.393+0000 7f5dd8200640 1 -- 172.21.15.97:0/954048978 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd01a2190 con 0x7f5dd0103ae0 2024-03-26T18:58:43.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.394+0000 7f5dd8200640 1 -- 172.21.15.97:0/954048978 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd010ada0 con 0x7f5dd0103ae0 2024-03-26T18:58:43.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.397+0000 7f5dce000640 1 -- 172.21.15.97:0/954048978 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f5dc401f020 con 0x7f5dd0103ae0 2024-03-26T18:58:43.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.398+0000 7f5dce000640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5db80a2fa0 0x7f5db80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:43.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.398+0000 7f5dce000640 1 -- 172.21.15.97:0/954048978 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5dc40cf6c0 con 0x7f5dd0103ae0 2024-03-26T18:58:43.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.399+0000 7f5dd4e00640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5db80a2fa0 0x7f5db80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:43.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.399+0000 7f5dd4e00640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5db80a2fa0 0x7f5db80a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc02c250 tx=0x7f5dbc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:43.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.402+0000 7f5dce000640 1 -- 172.21.15.97:0/954048978 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5dc401e2c0 con 0x7f5dd0103ae0 2024-03-26T18:58:43.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:43 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3182837190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:43.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:43 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1073518153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:43 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3182837190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:43.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:43 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1073518153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:43.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:43 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3182837190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:43.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:43 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1073518153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:43.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.675+0000 7f5dd8200640 1 -- 172.21.15.97:0/954048978 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5dd002d240 con 0x7f5dd0103ae0 2024-03-26T18:58:43.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.676+0000 7f5dce000640 1 -- 172.21.15.97:0/954048978 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f5dd002d240 con 0x7f5dd0103ae0 2024-03-26T18:58:43.670 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-26T18:58:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 -- 172.21.15.97:0/954048978 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5db80a2fa0 msgr2=0x7f5db80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5db80a2fa0 0x7f5db80a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc02c250 tx=0x7f5dbc005fd0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 -- 172.21.15.97:0/954048978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dd0103ae0 msgr2=0x7f5dd0199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dd0103ae0 0x7f5dd0199dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400e9c0 tx=0x7f5dc400ee80 comp rx=0 tx=0).stop 2024-03-26T18:58:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 -- 172.21.15.97:0/954048978 shutdown_connections 2024-03-26T18:58:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5db80a2fa0 0x7f5db80a5460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0105690 0x7f5dd01a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd0104cd0 0x7f5dd019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 --2- 172.21.15.97:0/954048978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dd0103ae0 0x7f5dd0199dc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 -- 172.21.15.97:0/954048978 >> 172.21.15.97:0/954048978 conn(0x7f5dd00ff070 msgr2=0x7f5dd0107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:43.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 -- 172.21.15.97:0/954048978 shutdown_connections 2024-03-26T18:58:43.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.679+0000 7f5dd8200640 1 -- 172.21.15.97:0/954048978 wait complete. 2024-03-26T18:58:43.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:58:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.898+0000 7fcc6a800640 1 -- 172.21.15.97:0/714763647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc641007e0 msgr2=0x7fcc64100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.898+0000 7fcc6a800640 1 --2- 172.21.15.97:0/714763647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc641007e0 0x7fcc64100c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009900 tx=0x7fcc5402bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.898+0000 7fcc6a800640 1 -- 172.21.15.97:0/714763647 shutdown_connections 2024-03-26T18:58:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.898+0000 7fcc6a800640 1 --2- 172.21.15.97:0/714763647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc64102390 0x7fcc64108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.898+0000 7fcc6a800640 1 --2- 172.21.15.97:0/714763647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc641019d0 0x7fcc64101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.898+0000 7fcc6a800640 1 --2- 172.21.15.97:0/714763647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc641007e0 0x7fcc64100c00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.899+0000 7fcc6a800640 1 -- 172.21.15.97:0/714763647 >> 172.21.15.97:0/714763647 conn(0x7fcc640fbd50 msgr2=0x7fcc640fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.899+0000 7fcc6a800640 1 -- 172.21.15.97:0/714763647 shutdown_connections 2024-03-26T18:58:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.899+0000 7fcc6a800640 1 -- 172.21.15.97:0/714763647 wait complete. 2024-03-26T18:58:43.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc6a800640 1 Processor -- start 2024-03-26T18:58:43.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc6a800640 1 -- start start 2024-03-26T18:58:43.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc6a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc641007e0 0x7fcc64199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:43.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc6a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc641019d0 0x7fcc6419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:43.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc6a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc64102390 0x7fcc641a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:43.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc6a800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410c4b0 con 0x7fcc641019d0 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc6a800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410c330 con 0x7fcc641007e0 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc6a800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410c630 con 0x7fcc64102390 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc63400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc641007e0 0x7fcc64199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc63e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc64102390 0x7fcc641a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc63400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc641007e0 0x7fcc64199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44044/0 (socket says 172.21.15.97:44044) 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc63400640 1 -- 172.21.15.97:0/3259699859 learned_addr learned my addr 172.21.15.97:0/3259699859 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.900+0000 7fcc62a00640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc641019d0 0x7fcc6419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc63400640 1 -- 172.21.15.97:0/3259699859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc64102390 msgr2=0x7fcc641a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc63400640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc64102390 0x7fcc641a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc63400640 1 -- 172.21.15.97:0/3259699859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc641019d0 msgr2=0x7fcc6419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc63400640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc641019d0 0x7fcc6419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc63400640 1 -- 172.21.15.97:0/3259699859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc641a1aa0 con 0x7fcc641007e0 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc63400640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc641007e0 0x7fcc64199e80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcc54000c00 tx=0x7fcc54004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc5be00640 1 -- 172.21.15.97:0/3259699859 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54043020 con 0x7fcc641007e0 2024-03-26T18:58:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc5be00640 1 -- 172.21.15.97:0/3259699859 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc540054e0 con 0x7fcc641007e0 2024-03-26T18:58:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc6a800640 1 -- 172.21.15.97:0/3259699859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc641a1d30 con 0x7fcc641007e0 2024-03-26T18:58:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc5be00640 1 -- 172.21.15.97:0/3259699859 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54003d00 con 0x7fcc641007e0 2024-03-26T18:58:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.901+0000 7fcc6a800640 1 -- 172.21.15.97:0/3259699859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc641a21d0 con 0x7fcc641007e0 2024-03-26T18:58:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.902+0000 7fcc6a800640 1 -- 172.21.15.97:0/3259699859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc641a23a0 con 0x7fcc641007e0 2024-03-26T18:58:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.904+0000 7fcc5be00640 1 -- 172.21.15.97:0/3259699859 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fcc54046020 con 0x7fcc641007e0 2024-03-26T18:58:43.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.906+0000 7fcc5be00640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcc480a2f20 0x7fcc480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:43.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.907+0000 7fcc5be00640 1 -- 172.21.15.97:0/3259699859 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcc540f1430 con 0x7fcc641007e0 2024-03-26T18:58:43.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.907+0000 7fcc62a00640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcc480a2f20 0x7fcc480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:43.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.907+0000 7fcc62a00640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcc480a2f20 0x7fcc480a53e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc6419b390 tx=0x7fcc50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:43.909+0000 7fcc5be00640 1 -- 172.21.15.97:0/3259699859 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcc5404d020 con 0x7fcc641007e0 2024-03-26T18:58:44.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.168+0000 7fcc6a800640 1 -- 172.21.15.97:0/3259699859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcc641a2700 con 0x7fcc641007e0 2024-03-26T18:58:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.168+0000 7fcc5be00640 1 -- 172.21.15.97:0/3259699859 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fcc540be230 con 0x7fcc641007e0 2024-03-26T18:58:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 -- 172.21.15.97:0/3259699859 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcc480a2f20 msgr2=0x7fcc480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcc480a2f20 0x7fcc480a53e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc6419b390 tx=0x7fcc50005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 -- 172.21.15.97:0/3259699859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc641007e0 msgr2=0x7fcc64199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc641007e0 0x7fcc64199e80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcc54000c00 tx=0x7fcc54004970 comp rx=0 tx=0).stop 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 -- 172.21.15.97:0/3259699859 shutdown_connections 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcc480a2f20 0x7fcc480a53e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc64102390 0x7fcc641a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc641019d0 0x7fcc6419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 --2- 172.21.15.97:0/3259699859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc641007e0 0x7fcc64199e80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 -- 172.21.15.97:0/3259699859 >> 172.21.15.97:0/3259699859 conn(0x7fcc640fbd50 msgr2=0x7fcc640fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 -- 172.21.15.97:0/3259699859 shutdown_connections 2024-03-26T18:58:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.171+0000 7fcc6a800640 1 -- 172.21.15.97:0/3259699859 wait complete. 2024-03-26T18:58:44.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:58:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.399+0000 7f9f3ea00640 1 -- 172.21.15.97:0/1140682557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38107620 msgr2=0x7f9f38109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.399+0000 7f9f3ea00640 1 --2- 172.21.15.97:0/1140682557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38107620 0x7f9f38109a10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009be0 tx=0x7f9f3002bef0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.399+0000 7f9f3ea00640 1 -- 172.21.15.97:0/1140682557 shutdown_connections 2024-03-26T18:58:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.399+0000 7f9f3ea00640 1 --2- 172.21.15.97:0/1140682557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38107620 0x7f9f38109a10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.399+0000 7f9f3ea00640 1 --2- 172.21.15.97:0/1140682557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f38104cf0 0x7f9f381070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.399+0000 7f9f3ea00640 1 --2- 172.21.15.97:0/1140682557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f381023c0 0x7f9f381047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.399+0000 7f9f3ea00640 1 -- 172.21.15.97:0/1140682557 >> 172.21.15.97:0/1140682557 conn(0x7f9f380fbd70 msgr2=0x7f9f380fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.399+0000 7f9f3ea00640 1 -- 172.21.15.97:0/1140682557 shutdown_connections 2024-03-26T18:58:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.399+0000 7f9f3ea00640 1 -- 172.21.15.97:0/1140682557 wait complete. 2024-03-26T18:58:44.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.400+0000 7f9f3ea00640 1 Processor -- start 2024-03-26T18:58:44.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.400+0000 7f9f3ea00640 1 -- start start 2024-03-26T18:58:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f3ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f381023c0 0x7f9f38199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f3ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38104cf0 0x7f9f3819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f3ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38107620 0x7f9f381a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f3ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810d5b0 con 0x7f9f38104cf0 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f3ea00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810d430 con 0x7f9f381023c0 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f3ea00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810d730 con 0x7f9f38107620 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f37e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f381023c0 0x7f9f38199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f37e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f381023c0 0x7f9f38199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44068/0 (socket says 172.21.15.97:44068) 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f37e00640 1 -- 172.21.15.97:0/3061806692 learned_addr learned my addr 172.21.15.97:0/3061806692 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f37400640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38104cf0 0x7f9f3819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38107620 0x7f9f381a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f37e00640 1 -- 172.21.15.97:0/3061806692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38107620 msgr2=0x7f9f381a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f37e00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38107620 0x7f9f381a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f37e00640 1 -- 172.21.15.97:0/3061806692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38104cf0 msgr2=0x7f9f3819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f37e00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38104cf0 0x7f9f3819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.401+0000 7f9f37e00640 1 -- 172.21.15.97:0/3061806692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f381a19e0 con 0x7f9f381023c0 2024-03-26T18:58:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.402+0000 7f9f37e00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f381023c0 0x7f9f38199dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800e9c0 tx=0x7f9f2800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.402+0000 7f9f34c00640 1 -- 172.21.15.97:0/3061806692 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800ccc0 con 0x7f9f381023c0 2024-03-26T18:58:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.402+0000 7f9f34c00640 1 -- 172.21.15.97:0/3061806692 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f28005500 con 0x7f9f381023c0 2024-03-26T18:58:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.402+0000 7f9f34c00640 1 -- 172.21.15.97:0/3061806692 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f28010430 con 0x7f9f381023c0 2024-03-26T18:58:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.402+0000 7f9f3ea00640 1 -- 172.21.15.97:0/3061806692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f381a1cd0 con 0x7f9f381023c0 2024-03-26T18:58:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.402+0000 7f9f3ea00640 1 -- 172.21.15.97:0/3061806692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f381a21c0 con 0x7f9f381023c0 2024-03-26T18:58:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.403+0000 7f9f3ea00640 1 -- 172.21.15.97:0/3061806692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f38109110 con 0x7f9f381023c0 2024-03-26T18:58:44.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.406+0000 7f9f34c00640 1 -- 172.21.15.97:0/3061806692 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f9f280046c0 con 0x7f9f381023c0 2024-03-26T18:58:44.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.407+0000 7f9f34c00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a2fa0 0x7f9f1c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:44.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.407+0000 7f9f34c00640 1 -- 172.21.15.97:0/3061806692 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9f280cf670 con 0x7f9f381023c0 2024-03-26T18:58:44.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.407+0000 7f9f37400640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a2fa0 0x7f9f1c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:44.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.408+0000 7f9f37400640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a2fa0 0x7f9f1c0a5460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9f240095f0 tx=0x7f9f24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:44.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.411+0000 7f9f34c00640 1 -- 172.21.15.97:0/3061806692 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9f2809c470 con 0x7f9f381023c0 2024-03-26T18:58:44.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:44 smithi105 ceph-mon[36270]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:44.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:44 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2979632010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:44.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:44 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/954048978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:44.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:44 smithi097 ceph-mon[31955]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:44.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:44 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2979632010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:44.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:44 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/954048978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:44.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:44 smithi144 ceph-mon[36053]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:44.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:44 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2979632010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:58:44.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:44 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/954048978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:58:44.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.667+0000 7f9f3ea00640 1 -- 172.21.15.97:0/3061806692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f3802d240 con 0x7f9f381023c0 2024-03-26T18:58:44.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.668+0000 7f9f34c00640 1 -- 172.21.15.97:0/3061806692 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11935 (secure 0 0 0) 0x7f9f2809bd70 con 0x7f9f381023c0 2024-03-26T18:58:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:44.662 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T18:56:23.642405+0000","modified":"2024-03-26T18:56:31.851339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24535,"mds_3":24494,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1343808654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1343808654},{"type":"v1","addr":"172.21.15.105:6837","nonce":1343808654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2391073232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2391073232},{"type":"v1","addr":"172.21.15.105:6839","nonce":2391073232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24535,24494,24481]},"id":1}]} 2024-03-26T18:58:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-03-26T18:58:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.672+0000 7f9f3ea00640 1 -- 172.21.15.97:0/3061806692 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a2fa0 msgr2=0x7f9f1c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.672+0000 7f9f3ea00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a2fa0 0x7f9f1c0a5460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9f240095f0 tx=0x7f9f24005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.672+0000 7f9f3ea00640 1 -- 172.21.15.97:0/3061806692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f381023c0 msgr2=0x7f9f38199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.672+0000 7f9f3ea00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f381023c0 0x7f9f38199dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800e9c0 tx=0x7f9f2800ee80 comp rx=0 tx=0).stop 2024-03-26T18:58:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.672+0000 7f9f3ea00640 1 -- 172.21.15.97:0/3061806692 shutdown_connections 2024-03-26T18:58:44.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.672+0000 7f9f3ea00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a2fa0 0x7f9f1c0a5460 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.672+0000 7f9f3ea00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38107620 0x7f9f381a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.672+0000 7f9f3ea00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38104cf0 0x7f9f3819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.672+0000 7f9f3ea00640 1 --2- 172.21.15.97:0/3061806692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f381023c0 0x7f9f38199dc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.672+0000 7f9f3ea00640 1 -- 172.21.15.97:0/3061806692 >> 172.21.15.97:0/3061806692 conn(0x7f9f380fbd70 msgr2=0x7f9f38101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:44.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.673+0000 7f9f3ea00640 1 -- 172.21.15.97:0/3061806692 shutdown_connections 2024-03-26T18:58:44.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.673+0000 7f9f3ea00640 1 -- 172.21.15.97:0/3061806692 wait complete. 2024-03-26T18:58:44.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:58:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.908+0000 7f1961400640 1 -- 172.21.15.97:0/2251315236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195c101720 msgr2=0x7f195c101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.908+0000 7f1961400640 1 --2- 172.21.15.97:0/2251315236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195c101720 0x7f195c101b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f194c009900 tx=0x7f194c02bc40 comp rx=0 tx=0).stop 2024-03-26T18:58:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.908+0000 7f1961400640 1 -- 172.21.15.97:0/2251315236 shutdown_connections 2024-03-26T18:58:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.908+0000 7f1961400640 1 --2- 172.21.15.97:0/2251315236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c1032d0 0x7f195c109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.908+0000 7f1961400640 1 --2- 172.21.15.97:0/2251315236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f195c102910 0x7f195c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.908+0000 7f1961400640 1 --2- 172.21.15.97:0/2251315236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195c101720 0x7f195c101b40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.908+0000 7f1961400640 1 -- 172.21.15.97:0/2251315236 >> 172.21.15.97:0/2251315236 conn(0x7f195c0fcc90 msgr2=0x7f195c0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.908+0000 7f1961400640 1 -- 172.21.15.97:0/2251315236 shutdown_connections 2024-03-26T18:58:44.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.908+0000 7f1961400640 1 -- 172.21.15.97:0/2251315236 wait complete. 2024-03-26T18:58:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1961400640 1 Processor -- start 2024-03-26T18:58:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1961400640 1 -- start start 2024-03-26T18:58:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1961400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c101720 0x7f195c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1961400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195c102910 0x7f195c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1961400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f195c1032d0 0x7f195c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1961400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f195c10d3c0 con 0x7f195c1032d0 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1961400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f195c10d240 con 0x7f195c102910 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1961400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f195c10d540 con 0x7f195c101720 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1953e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195c102910 0x7f195c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f195be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c101720 0x7f195c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1953e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195c102910 0x7f195c197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44098/0 (socket says 172.21.15.97:44098) 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.909+0000 7f1953e00640 1 -- 172.21.15.97:0/2133635182 learned_addr learned my addr 172.21.15.97:0/2133635182 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1960a00640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f195c1032d0 0x7f195c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1953e00640 1 -- 172.21.15.97:0/2133635182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c101720 msgr2=0x7f195c197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1953e00640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c101720 0x7f195c197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1953e00640 1 -- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f195c1032d0 msgr2=0x7f195c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1953e00640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f195c1032d0 0x7f195c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1953e00640 1 -- 172.21.15.97:0/2133635182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c19f650 con 0x7f195c102910 2024-03-26T18:58:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1960a00640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f195c1032d0 0x7f195c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1953e00640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195c102910 0x7f195c197f70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f194400e9f0 tx=0x7f194400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1959600640 1 -- 172.21.15.97:0/2133635182 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194400cd80 con 0x7f195c102910 2024-03-26T18:58:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1959600640 1 -- 172.21.15.97:0/2133635182 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1944010910 con 0x7f195c102910 2024-03-26T18:58:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1961400640 1 -- 172.21.15.97:0/2133635182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195c19f940 con 0x7f195c102910 2024-03-26T18:58:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.910+0000 7f1959600640 1 -- 172.21.15.97:0/2133635182 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944018990 con 0x7f195c102910 2024-03-26T18:58:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.911+0000 7f1961400640 1 -- 172.21.15.97:0/2133635182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195c19fee0 con 0x7f195c102910 2024-03-26T18:58:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.912+0000 7f1961400640 1 -- 172.21.15.97:0/2133635182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f195c108a40 con 0x7f195c102910 2024-03-26T18:58:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.913+0000 7f1959600640 1 -- 172.21.15.97:0/2133635182 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f1944010ac0 con 0x7f195c102910 2024-03-26T18:58:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.915+0000 7f1959600640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19400a2fa0 0x7f19400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:44.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.915+0000 7f1959600640 1 -- 172.21.15.97:0/2133635182 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f19440cf730 con 0x7f195c102910 2024-03-26T18:58:44.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.915+0000 7f195be00640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19400a2fa0 0x7f19400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:44.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.916+0000 7f195be00640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19400a2fa0 0x7f19400a5460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f194c02c180 tx=0x7f194c02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:44.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:44.918+0000 7f1959600640 1 -- 172.21.15.97:0/2133635182 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f194409c530 con 0x7f195c102910 2024-03-26T18:58:45.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.120+0000 7f1961400640 1 -- 172.21.15.97:0/2133635182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f195c0624c0 con 0x7f195c102910 2024-03-26T18:58:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.121+0000 7f1959600640 1 -- 172.21.15.97:0/2133635182 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22581 (secure 0 0 0) 0x7f194409be30 con 0x7f195c102910 2024-03-26T18:58:45.115 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:58:45.116 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T18:57:30.061624+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:58:45.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 -- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19400a2fa0 msgr2=0x7f19400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19400a2fa0 0x7f19400a5460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f194c02c180 tx=0x7f194c02cf90 comp rx=0 tx=0).stop 2024-03-26T18:58:45.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 -- 172.21.15.97:0/2133635182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195c102910 msgr2=0x7f195c197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195c102910 0x7f195c197f70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f194400e9f0 tx=0x7f194400eeb0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 -- 172.21.15.97:0/2133635182 shutdown_connections 2024-03-26T18:58:45.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19400a2fa0 0x7f19400a5460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f195c1032d0 0x7f195c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195c102910 0x7f195c197f70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 --2- 172.21.15.97:0/2133635182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c101720 0x7f195c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 -- 172.21.15.97:0/2133635182 >> 172.21.15.97:0/2133635182 conn(0x7f195c0fcc90 msgr2=0x7f195c105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 -- 172.21.15.97:0/2133635182 shutdown_connections 2024-03-26T18:58:45.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.124+0000 7f1961400640 1 -- 172.21.15.97:0/2133635182 wait complete. 2024-03-26T18:58:45.136 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T18:56:23.642405+0000', 'modified': '2024-03-26T18:56:31.851339+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24581, 'mds_1': 24569, 'mds_2': 24535, 'mds_3': 24494, 'mds_4': 24481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6833/3099308706', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3099308706}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3099308706}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6833/254387341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6832', 'nonce': 254387341}, {'type': 'v1', 'addr': '172.21.15.105:6833', 'nonce': 254387341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'd', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.97:6837/4124660285', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 4124660285}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 4124660285}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.105:6835/1538692215', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 1538692215}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 1538692215}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/411039946', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 411039946}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 411039946}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6837/1138636767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 1138636767}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 1138636767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6837/1343808654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 1343808654}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1343808654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.144:6839/994192292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 994192292}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 994192292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'a', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.97:6841/4165060179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6840', 'nonce': 4165060179}, {'type': 'v1', 'addr': '172.21.15.97:6841', 'nonce': 4165060179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6839/2391073232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 2391073232}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 2391073232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24569, 24581, 24535, 24494, 24481]} 2024-03-26T18:58:45.137 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-26T18:58:45.137 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-26T18:58:45.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.346+0000 7f9da8e00640 1 -- 172.21.15.97:0/3519986515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da41017c0 msgr2=0x7f9da4101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.346+0000 7f9da8e00640 1 --2- 172.21.15.97:0/3519986515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da41017c0 0x7f9da4101be0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009900 tx=0x7f9d9402bc50 comp rx=0 tx=0).stop 2024-03-26T18:58:45.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.346+0000 7f9da8e00640 1 -- 172.21.15.97:0/3519986515 shutdown_connections 2024-03-26T18:58:45.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.346+0000 7f9da8e00640 1 --2- 172.21.15.97:0/3519986515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da4103370 0x7f9da4109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.346+0000 7f9da8e00640 1 --2- 172.21.15.97:0/3519986515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da41029b0 0x7f9da4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.346+0000 7f9da8e00640 1 --2- 172.21.15.97:0/3519986515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da41017c0 0x7f9da4101be0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.346+0000 7f9da8e00640 1 -- 172.21.15.97:0/3519986515 >> 172.21.15.97:0/3519986515 conn(0x7f9da40fcd30 msgr2=0x7f9da40ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.346+0000 7f9da8e00640 1 -- 172.21.15.97:0/3519986515 shutdown_connections 2024-03-26T18:58:45.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.346+0000 7f9da8e00640 1 -- 172.21.15.97:0/3519986515 wait complete. 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.347+0000 7f9da8e00640 1 Processor -- start 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.347+0000 7f9da8e00640 1 -- start start 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da8e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da41017c0 0x7f9da4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da41029b0 0x7f9da4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da8e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da4103370 0x7f9da419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da8e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9da410d460 con 0x7f9da41029b0 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da8e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9da410d2e0 con 0x7f9da41017c0 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da8e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9da410d5e0 con 0x7f9da4103370 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da41017c0 0x7f9da4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da41017c0 0x7f9da4197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44104/0 (socket says 172.21.15.97:44104) 2024-03-26T18:58:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da3400640 1 -- 172.21.15.97:0/1072689354 learned_addr learned my addr 172.21.15.97:0/1072689354 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da2a00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da41029b0 0x7f9da4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da3e00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da4103370 0x7f9da419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da3400640 1 -- 172.21.15.97:0/1072689354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da4103370 msgr2=0x7f9da419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da3400640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da4103370 0x7f9da419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da3400640 1 -- 172.21.15.97:0/1072689354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da41029b0 msgr2=0x7f9da4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da3400640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da41029b0 0x7f9da4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.348+0000 7f9da3400640 1 -- 172.21.15.97:0/1072689354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da419f730 con 0x7f9da41017c0 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.349+0000 7f9da3400640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da41017c0 0x7f9da4197b10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9d94005ab0 tx=0x7f9d9402c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.349+0000 7f9d9be00640 1 -- 172.21.15.97:0/1072689354 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94041070 con 0x7f9da41017c0 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.349+0000 7f9d9be00640 1 -- 172.21.15.97:0/1072689354 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d94004450 con 0x7f9da41017c0 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.349+0000 7f9da8e00640 1 -- 172.21.15.97:0/1072689354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da419f9c0 con 0x7f9da41017c0 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.349+0000 7f9d9be00640 1 -- 172.21.15.97:0/1072689354 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94034440 con 0x7f9da41017c0 2024-03-26T18:58:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.349+0000 7f9da8e00640 1 -- 172.21.15.97:0/1072689354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da419ff10 con 0x7f9da41017c0 2024-03-26T18:58:45.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.352+0000 7f9dacd22740 1 Processor -- start 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9dacd22740 1 -- start start 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 0x5567b4424380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4424950 0x5567b4429450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4429ac0 0x5567b442bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9dacd22740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5567b43841b0 con 0x5567b4423f60 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9dacd22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5567b432cb00 con 0x5567b4429ac0 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9dacd22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567b4380160 con 0x5567b4424950 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9da2a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4429ac0 0x5567b442bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9da2a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4429ac0 0x5567b442bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44112/0 (socket says 172.21.15.97:44112) 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 0x5567b4424380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.353+0000 7f9da2a00640 1 -- 172.21.15.97:0/471022456 learned_addr learned my addr 172.21.15.97:0/471022456 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.354+0000 7f9da2a00640 1 -- 172.21.15.97:0/471022456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4424950 msgr2=0x5567b4429450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:58:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.354+0000 7f9da2a00640 1 --2- 172.21.15.97:0/471022456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4424950 0x5567b4429450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.354+0000 7f9da2a00640 1 -- 172.21.15.97:0/471022456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 msgr2=0x5567b4424380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.354+0000 7f9da2a00640 1 --2- 172.21.15.97:0/471022456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 0x5567b4424380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.354+0000 7f9da2a00640 1 -- 172.21.15.97:0/471022456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b40982d0 con 0x5567b4429ac0 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.354+0000 7f9d9be00640 1 -- 172.21.15.97:0/1072689354 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f9d94044070 con 0x7f9da41017c0 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.354+0000 7f9da2a00640 1 --2- 172.21.15.97:0/471022456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4429ac0 0x5567b442bf50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009c90 tx=0x7f9d90011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe5707b88e1fbd81 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.354+0000 7f9d98c00640 1 -- 172.21.15.97:0/471022456 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9001e070 con 0x5567b4429ac0 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.354+0000 7f9d98c00640 1 -- 172.21.15.97:0/471022456 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90019b10 con 0x5567b4429ac0 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.354+0000 7f9d98c00640 1 -- 172.21.15.97:0/471022456 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000eac0 con 0x5567b4429ac0 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.355+0000 7f9dacd22740 1 -- 172.21.15.97:0/471022456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4429ac0 msgr2=0x5567b442bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.355+0000 7f9dacd22740 1 --2- 172.21.15.97:0/471022456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4429ac0 0x5567b442bf50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009c90 tx=0x7f9d90011810 comp rx=0 tx=0).stop 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.355+0000 7f9dacd22740 1 -- 172.21.15.97:0/471022456 shutdown_connections 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.355+0000 7f9dacd22740 1 --2- 172.21.15.97:0/471022456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4429ac0 0x5567b442bf50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.355+0000 7f9dacd22740 1 --2- 172.21.15.97:0/471022456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4424950 0x5567b4429450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.355+0000 7f9dacd22740 1 --2- 172.21.15.97:0/471022456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 0x5567b4424380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.355+0000 7f9dacd22740 1 -- 172.21.15.97:0/471022456 >> 172.21.15.97:0/471022456 conn(0x5567b441f4d0 msgr2=0x5567b441f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.355+0000 7f9dacd22740 1 -- 172.21.15.97:0/471022456 shutdown_connections 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.355+0000 7f9dacd22740 1 -- 172.21.15.97:0/471022456 wait complete. 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.356+0000 7f9dacd22740 1 -- start start 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.356+0000 7f9d9be00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9d880a2eb0 0x7f9d880a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.356+0000 7f9dacd22740 1 Processor -- start 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.356+0000 7f9d9be00640 1 -- 172.21.15.97:0/1072689354 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9d940f0f40 con 0x7f9da41017c0 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.356+0000 7f9da3e00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9d880a2eb0 0x7f9d880a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4423f60 0x5567b442e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4424950 0x5567b442ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4429ac0 0x5567b4435a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9dacd22740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5567b442ca70 con 0x5567b4429ac0 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9dacd22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5567b433fed0 con 0x5567b4423f60 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9dacd22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567b442cbf0 con 0x5567b4424950 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9dacd22740 10 client.0 _ll_register_callbacks cb 0x5567b437b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9da3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4423f60 0x5567b442e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9da3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4423f60 0x5567b442e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44116/0 (socket says 172.21.15.97:44116) 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9da3400640 1 -- 172.21.15.97:0/1726805941 learned_addr learned my addr 172.21.15.97:0/1726805941 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9da3e00640 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4429ac0 0x5567b4435a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9da2a00640 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4424950 0x5567b442ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9da3400640 1 -- 172.21.15.97:0/1726805941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4424950 msgr2=0x5567b442ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.357+0000 7f9da3400640 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4424950 0x5567b442ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9da3e00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9d880a2eb0 0x7f9d880a5370 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5567b44251b0 tx=0x7f9d9c0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9da3400640 1 -- 172.21.15.97:0/1726805941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4429ac0 msgr2=0x5567b4435a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9da3400640 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4429ac0 0x5567b4435a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9da3400640 1 -- 172.21.15.97:0/1726805941 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b41626b0 con 0x5567b4423f60 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9da2a00640 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4424950 0x5567b442ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9da3400640 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4423f60 0x5567b442e500 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d940bd3f0 tx=0x7f9d940bd230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17f4acd2a12a4838 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9d8d600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:58:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9d8d600640 1 -- 172.21.15.97:0/1726805941 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9404b040 con 0x5567b4423f60 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9d8d600640 1 -- 172.21.15.97:0/1726805941 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9d94003680 con 0x5567b4423f60 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9dacd22740 1 -- 172.21.15.97:0/1726805941 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d90019b10 con 0x5567b4423f60 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9d8d600640 1 -- 172.21.15.97:0/1726805941 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d940c1790 con 0x5567b4423f60 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9dacd22740 10 client.25184 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.358+0000 7f9dacd22740 1 -- 172.21.15.97:0/1726805941 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567b4438a60 con 0x5567b4423f60 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.359+0000 7f9d8d600640 1 -- 172.21.15.97:0/1726805941 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90019b10 con 0x5567b4423f60 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.359+0000 7f9d8d600640 1 -- 172.21.15.97:0/1726805941 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9d940bc5b0 con 0x5567b4423f60 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.359+0000 7f9dacd22740 10 client.25184 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.359+0000 7f9dacd22740 10 client.25184 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.359+0000 7f9dacd22740 20 client.25184 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.359+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5567b443b040 0x5567b443c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.359+0000 7f9dacd22740 4 client.25184 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.359+0000 7f9dacd22740 1 -- 172.21.15.97:0/1726805941 --> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567b43916f0 con 0x5567b443b040 2024-03-26T18:58:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.359+0000 7f9da2a00640 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5567b443b040 0x5567b443c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.360+0000 7f9da2a00640 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5567b443b040 0x5567b443c4b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca554c25a2a2935f server_cookie=2a30cc77aabcef9e in_seq=0 out_seq=0 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.360+0000 7f9d8d600640 10 client.25184 ms_handle_connect on v2:172.21.15.97:6836/4124660285 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.360+0000 7f9d8d600640 1 -- 172.21.15.97:0/1726805941 <== mds.4 v2:172.21.15.97:6836/4124660285 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5567b43916f0 con 0x5567b443b040 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.360+0000 7f9d8d600640 10 client.25184 handle_command_reply: tid=0 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 client.25184 shutdown 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 -- 172.21.15.97:0/1726805941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4423f60 msgr2=0x5567b442e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4423f60 0x5567b442e500 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d940bd3f0 tx=0x7f9d940bd230 comp rx=0 tx=0).stop 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 -- 172.21.15.97:0/1726805941 shutdown_connections 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5567b443b040 0x5567b443c4b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4429ac0 0x5567b4435a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4424950 0x5567b442ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1726805941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4423f60 0x5567b442e500 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9d8d600640 0 client.25184 ms_handle_reset on v2:172.21.15.97:6836/4124660285 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 -- 172.21.15.97:0/1726805941 >> 172.21.15.97:0/1726805941 conn(0x5567b441f4d0 msgr2=0x5567b441a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 -- 172.21.15.97:0/1726805941 shutdown_connections 2024-03-26T18:58:45.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 1 -- 172.21.15.97:0/1726805941 wait complete. 2024-03-26T18:58:45.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.361+0000 7f9dacd22740 20 client.25184 trim_cache size 0 max 16384 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9dacd22740 1 Processor -- start 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9dacd22740 1 -- start start 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4423f60 0x5567b442cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4435a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4429ac0 0x5567b4425eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9dacd22740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5567b443a7f0 con 0x5567b4429ac0 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9dacd22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5567b443aec0 con 0x5567b4424950 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9dacd22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567b443b590 con 0x5567b4423f60 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9da2a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4435a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9da3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4423f60 0x5567b442cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9da2a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4435a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44120/0 (socket says 172.21.15.97:44120) 2024-03-26T18:58:45.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4429ac0 0x5567b4425eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.381+0000 7f9da2a00640 1 -- 172.21.15.97:0/1850163069 learned_addr learned my addr 172.21.15.97:0/1850163069 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.382+0000 7f9da3400640 1 -- 172.21.15.97:0/1850163069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 msgr2=0x5567b4435a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.382+0000 7f9da3400640 1 --2- 172.21.15.97:0/1850163069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4435a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.382+0000 7f9da3400640 1 -- 172.21.15.97:0/1850163069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4429ac0 msgr2=0x5567b4425eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.382+0000 7f9da3400640 1 --2- 172.21.15.97:0/1850163069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4429ac0 0x5567b4425eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.382+0000 7f9da3400640 1 -- 172.21.15.97:0/1850163069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b442f680 con 0x5567b4423f60 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.382+0000 7f9da2a00640 1 --2- 172.21.15.97:0/1850163069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4435a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.382+0000 7f9da3400640 1 --2- 172.21.15.97:0/1850163069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4423f60 0x5567b442cff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d94036040 tx=0x7f9d940a8050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8686ee6fef0691e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.382+0000 7f9d8fe00640 1 -- 172.21.15.97:0/1850163069 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9404b040 con 0x5567b4423f60 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.382+0000 7f9d8fe00640 1 -- 172.21.15.97:0/1850163069 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d940c5dd0 con 0x5567b4423f60 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.382+0000 7f9d8fe00640 1 -- 172.21.15.97:0/1850163069 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d940c2ac0 con 0x5567b4423f60 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.383+0000 7f9dacd22740 1 -- 172.21.15.97:0/1850163069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4423f60 msgr2=0x5567b442cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.383+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1850163069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4423f60 0x5567b442cff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d94036040 tx=0x7f9d940a8050 comp rx=0 tx=0).stop 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.383+0000 7f9dacd22740 1 -- 172.21.15.97:0/1850163069 shutdown_connections 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.383+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1850163069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4429ac0 0x5567b4425eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.383+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1850163069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4435a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.383+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1850163069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4423f60 0x5567b442cff0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.383+0000 7f9dacd22740 1 -- 172.21.15.97:0/1850163069 >> 172.21.15.97:0/1850163069 conn(0x5567b441f4d0 msgr2=0x5567b44253c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.383+0000 7f9dacd22740 1 -- 172.21.15.97:0/1850163069 shutdown_connections 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.383+0000 7f9dacd22740 1 -- 172.21.15.97:0/1850163069 wait complete. 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.383+0000 7f9dacd22740 1 -- start start 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9dacd22740 1 Processor -- start 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 0x5567b4431240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4431780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9dacd22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4429ac0 0x5567b4434680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9dacd22740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5567b443c330 con 0x5567b4423f60 2024-03-26T18:58:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9dacd22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5567b443bc60 con 0x5567b4424950 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9dacd22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567b443ca00 con 0x5567b4429ac0 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9dacd22740 10 client.0 _ll_register_callbacks cb 0x7f9d940b9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9da3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 0x5567b4431240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4429ac0 0x5567b4434680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9da2a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4431780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9da3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 0x5567b4431240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36846/0 (socket says 172.21.15.97:36846) 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.384+0000 7f9da3400640 1 -- 172.21.15.97:0/1284797655 learned_addr learned my addr 172.21.15.97:0/1284797655 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9da3400640 1 -- 172.21.15.97:0/1284797655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4429ac0 msgr2=0x5567b4434680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9da3400640 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4429ac0 0x5567b4434680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9da3400640 1 -- 172.21.15.97:0/1284797655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 msgr2=0x5567b4431780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9da3400640 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4431780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9da3400640 1 -- 172.21.15.97:0/1284797655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b4434cf0 con 0x5567b4423f60 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9da2a00640 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4431780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9da3e00640 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4429ac0 0x5567b4434680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9da3400640 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 0x5567b4431240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d940bca00 tx=0x7f9d94003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a25da420dc6376 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9d8d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9d8d600640 1 -- 172.21.15.97:0/1284797655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9404b070 con 0x5567b4423f60 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9d8d600640 1 -- 172.21.15.97:0/1284797655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9d940b9440 con 0x5567b4423f60 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9dacd22740 1 -- 172.21.15.97:0/1284797655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5567b438ba40 con 0x5567b4423f60 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9d8d600640 1 -- 172.21.15.97:0/1284797655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d940c2cc0 con 0x5567b4423f60 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9dacd22740 10 client.15057 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.385+0000 7f9dacd22740 1 -- 172.21.15.97:0/1284797655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567b44a0db0 con 0x5567b4423f60 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.386+0000 7f9d8d600640 1 -- 172.21.15.97:0/1284797655 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5567b438ba40 con 0x5567b4423f60 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.386+0000 7f9d8d600640 1 -- 172.21.15.97:0/1284797655 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9d940c1160 con 0x5567b4423f60 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.386+0000 7f9dacd22740 10 client.15057 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.386+0000 7f9dacd22740 10 client.15057 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.386+0000 7f9dacd22740 20 client.15057 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.386+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5567b44a5380 0x5567b44a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.386+0000 7f9dacd22740 4 client.15057 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:58:45.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.386+0000 7f9dacd22740 1 -- 172.21.15.97:0/1284797655 --> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] -- command(tid 0: {"prefix": "status"}) -- 0x5567b43098c0 con 0x5567b44a5380 2024-03-26T18:58:45.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.386+0000 7f9da2a00640 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5567b44a5380 0x5567b44a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9da2a00640 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5567b44a5380 0x5567b44a1e90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d292e6968a79137d server_cookie=6c2ec3c333b45ee in_seq=0 out_seq=0 2024-03-26T18:58:45.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9d8d600640 10 client.15057 ms_handle_connect on v2:172.21.15.97:6836/4124660285 2024-03-26T18:58:45.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9d8d600640 1 -- 172.21.15.97:0/1284797655 <== mds.4 v2:172.21.15.97:6836/4124660285 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5567b43098c0 con 0x5567b44a5380 2024-03-26T18:58:45.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9d8d600640 10 client.15057 handle_command_reply: tid=0 2024-03-26T18:58:45.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9dacd22740 1 client.15057 shutdown 2024-03-26T18:58:45.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9dacd22740 1 -- 172.21.15.97:0/1284797655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 msgr2=0x5567b4431240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 0x5567b4431240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d940bca00 tx=0x7f9d94003f90 comp rx=0 tx=0).stop 2024-03-26T18:58:45.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9dacd22740 1 -- 172.21.15.97:0/1284797655 shutdown_connections 2024-03-26T18:58:45.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.97:6836/4124660285,v1:172.21.15.97:6837/4124660285] conn(0x5567b44a5380 0x5567b44a1e90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567b4429ac0 0x5567b4434680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9d8d600640 0 client.15057 ms_handle_reset on v2:172.21.15.97:6836/4124660285 2024-03-26T18:58:45.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567b4424950 0x5567b4431780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9dacd22740 1 --2- 172.21.15.97:0/1284797655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567b4423f60 0x5567b4431240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.387+0000 7f9dacd22740 1 -- 172.21.15.97:0/1284797655 >> 172.21.15.97:0/1284797655 conn(0x5567b441f4d0 msgr2=0x5567b4425eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.388+0000 7f9dacd22740 1 -- 172.21.15.97:0/1284797655 shutdown_connections 2024-03-26T18:58:45.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.388+0000 7f9dacd22740 1 -- 172.21.15.97:0/1284797655 wait complete. 2024-03-26T18:58:45.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.388+0000 7f9dacd22740 20 client.15057 trim_cache size 0 max 16384 2024-03-26T18:58:45.386 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:58:45.386 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 4, 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24481, 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 134.56415401699999, 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 173.41036117600001 2024-03-26T18:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:58:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.388+0000 7f9da8e00640 1 -- 172.21.15.97:0/1072689354 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9d880a2eb0 msgr2=0x7f9d880a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.388+0000 7f9da8e00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9d880a2eb0 0x7f9d880a5370 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5567b44251b0 tx=0x7f9d9c0096a0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.389+0000 7f9da8e00640 1 -- 172.21.15.97:0/1072689354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da41017c0 msgr2=0x7f9da4197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.389+0000 7f9da8e00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da41017c0 0x7f9da4197b10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9d94005ab0 tx=0x7f9d9402c6e0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.389+0000 7f9da8e00640 1 -- 172.21.15.97:0/1072689354 shutdown_connections 2024-03-26T18:58:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.389+0000 7f9da8e00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9d880a2eb0 0x7f9d880a5370 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.389+0000 7f9da8e00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da4103370 0x7f9da419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.389+0000 7f9da8e00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da41029b0 0x7f9da4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.389+0000 7f9da8e00640 1 --2- 172.21.15.97:0/1072689354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da41017c0 0x7f9da4197b10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.389+0000 7f9da8e00640 1 -- 172.21.15.97:0/1072689354 >> 172.21.15.97:0/1072689354 conn(0x7f9da40fcd30 msgr2=0x7f9da4105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.389+0000 7f9da8e00640 1 -- 172.21.15.97:0/1072689354 shutdown_connections 2024-03-26T18:58:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.389+0000 7f9da8e00640 1 -- 172.21.15.97:0/1072689354 wait complete. 2024-03-26T18:58:45.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:45 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3259699859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:45.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:45 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3061806692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:45.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:45 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2133635182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:45.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-26T18:58:45.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:45 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3259699859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:45.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:45 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3061806692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:45.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:45 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2133635182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:45.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.621+0000 7fc561200640 1 -- 172.21.15.97:0/297996525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc55c102ae0 msgr2=0x7fc55c102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.621+0000 7fc561200640 1 --2- 172.21.15.97:0/297996525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc55c102ae0 0x7fc55c102f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc544009860 tx=0x7fc54402bb70 comp rx=0 tx=0).stop 2024-03-26T18:58:45.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.621+0000 7fc561200640 1 -- 172.21.15.97:0/297996525 shutdown_connections 2024-03-26T18:58:45.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.621+0000 7fc561200640 1 --2- 172.21.15.97:0/297996525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc55c1034a0 0x7fc55c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.621+0000 7fc561200640 1 --2- 172.21.15.97:0/297996525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc55c102ae0 0x7fc55c102f60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.621+0000 7fc561200640 1 --2- 172.21.15.97:0/297996525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc55c1018f0 0x7fc55c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.621+0000 7fc561200640 1 -- 172.21.15.97:0/297996525 >> 172.21.15.97:0/297996525 conn(0x7fc55c0fce60 msgr2=0x7fc55c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.622+0000 7fc561200640 1 -- 172.21.15.97:0/297996525 shutdown_connections 2024-03-26T18:58:45.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.622+0000 7fc561200640 1 -- 172.21.15.97:0/297996525 wait complete. 2024-03-26T18:58:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.622+0000 7fc561200640 1 Processor -- start 2024-03-26T18:58:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc561200640 1 -- start start 2024-03-26T18:58:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc561200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc55c1018f0 0x7fc55c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc561200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc55c102ae0 0x7fc55c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc561200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc55c1034a0 0x7fc55c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc561200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c10d550 con 0x7fc55c1034a0 2024-03-26T18:58:45.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc561200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c10d3d0 con 0x7fc55c102ae0 2024-03-26T18:58:45.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc561200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c10d6d0 con 0x7fc55c1018f0 2024-03-26T18:58:45.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc559e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc55c1018f0 0x7fc55c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc553e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc55c102ae0 0x7fc55c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc559e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc55c1018f0 0x7fc55c197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51340/0 (socket says 172.21.15.97:51340) 2024-03-26T18:58:45.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc559e00640 1 -- 172.21.15.97:0/993761135 learned_addr learned my addr 172.21.15.97:0/993761135 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc553e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc55c102ae0 0x7fc55c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44126/0 (socket says 172.21.15.97:44126) 2024-03-26T18:58:45.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.623+0000 7fc55a800640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc55c1034a0 0x7fc55c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc559e00640 1 -- 172.21.15.97:0/993761135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc55c102ae0 msgr2=0x7fc55c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc559e00640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc55c102ae0 0x7fc55c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc559e00640 1 -- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc55c1034a0 msgr2=0x7fc55c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc559e00640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc55c1034a0 0x7fc55c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc559e00640 1 -- 172.21.15.97:0/993761135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55c19f860 con 0x7fc55c1018f0 2024-03-26T18:58:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc55a800640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc55c1034a0 0x7fc55c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc553e00640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc55c102ae0 0x7fc55c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc559e00640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc55c1018f0 0x7fc55c197c50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00e9c0 tx=0x7fc54c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc552a00640 1 -- 172.21.15.97:0/993761135 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00cd80 con 0x7fc55c1018f0 2024-03-26T18:58:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc552a00640 1 -- 172.21.15.97:0/993761135 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc54c0035f0 con 0x7fc55c1018f0 2024-03-26T18:58:45.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc552a00640 1 -- 172.21.15.97:0/993761135 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c010690 con 0x7fc55c1018f0 2024-03-26T18:58:45.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc561200640 1 -- 172.21.15.97:0/993761135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc55c19fb50 con 0x7fc55c1018f0 2024-03-26T18:58:45.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.624+0000 7fc561200640 1 -- 172.21.15.97:0/993761135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc55c1a00a0 con 0x7fc55c1018f0 2024-03-26T18:58:45.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:45 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3259699859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:58:45.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:45 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3061806692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:58:45.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:45 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2133635182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:58:45.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.627+0000 7fc552a00640 1 -- 172.21.15.97:0/993761135 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fc54c0037a0 con 0x7fc55c1018f0 2024-03-26T18:58:45.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.627+0000 7fc563fd7740 1 Processor -- start 2024-03-26T18:58:45.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc563fd7740 1 -- start start 2024-03-26T18:58:45.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc33d80 0x55ee4cc341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc34770 0x55ee4cc39270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4cc3bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc563fd7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cb941b0 con 0x55ee4cc34770 2024-03-26T18:58:45.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc563fd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cb3cb00 con 0x55ee4cc398e0 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc563fd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cb90160 con 0x55ee4cc33d80 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc559e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc34770 0x55ee4cc39270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc55a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc33d80 0x55ee4cc341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc553e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4cc3bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc55a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc33d80 0x55ee4cc341a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51352/0 (socket says 172.21.15.97:51352) 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.628+0000 7fc55a800640 1 -- 172.21.15.97:0/1149390077 learned_addr learned my addr 172.21.15.97:0/1149390077 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.629+0000 7fc553e00640 1 -- 172.21.15.97:0/1149390077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc33d80 msgr2=0x55ee4cc341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.629+0000 7fc553e00640 1 --2- 172.21.15.97:0/1149390077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc33d80 0x55ee4cc341a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.629+0000 7fc553e00640 1 -- 172.21.15.97:0/1149390077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc34770 msgr2=0x55ee4cc39270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.629+0000 7fc553e00640 1 --2- 172.21.15.97:0/1149390077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc34770 0x55ee4cc39270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.629+0000 7fc553e00640 1 -- 172.21.15.97:0/1149390077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4c8a82d0 con 0x55ee4cc398e0 2024-03-26T18:58:45.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.629+0000 7fc55a800640 1 --2- 172.21.15.97:0/1149390077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc33d80 0x55ee4cc341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.629+0000 7fc553e00640 1 --2- 172.21.15.97:0/1149390077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4cc3bd70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc54402c0b0 tx=0x7fc544034ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d9a3cb3bbae1c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.629+0000 7fc54b400640 1 -- 172.21.15.97:0/1149390077 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544044020 con 0x55ee4cc398e0 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.629+0000 7fc54b400640 1 -- 172.21.15.97:0/1149390077 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc544034640 con 0x55ee4cc398e0 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.629+0000 7fc54b400640 1 -- 172.21.15.97:0/1149390077 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544039040 con 0x55ee4cc398e0 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc552a00640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc5400a2f20 0x7fc5400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc552a00640 1 -- 172.21.15.97:0/993761135 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc54c0cf930 con 0x7fc55c1018f0 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc55a800640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc5400a2f20 0x7fc5400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc563fd7740 1 -- 172.21.15.97:0/1149390077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 msgr2=0x55ee4cc3bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc563fd7740 1 --2- 172.21.15.97:0/1149390077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4cc3bd70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc54402c0b0 tx=0x7fc544034ab0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc563fd7740 1 -- 172.21.15.97:0/1149390077 shutdown_connections 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc563fd7740 1 --2- 172.21.15.97:0/1149390077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4cc3bd70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc563fd7740 1 --2- 172.21.15.97:0/1149390077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc34770 0x55ee4cc39270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc563fd7740 1 --2- 172.21.15.97:0/1149390077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc33d80 0x55ee4cc341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc563fd7740 1 -- 172.21.15.97:0/1149390077 >> 172.21.15.97:0/1149390077 conn(0x55ee4cc2aee0 msgr2=0x55ee4cc2b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc563fd7740 1 -- 172.21.15.97:0/1149390077 shutdown_connections 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.630+0000 7fc55a800640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc5400a2f20 0x7fc5400a53e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55ee4cc34fd0 tx=0x7fc554011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.631+0000 7fc563fd7740 1 -- 172.21.15.97:0/1149390077 wait complete. 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.631+0000 7fc563fd7740 1 -- start start 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.631+0000 7fc563fd7740 1 Processor -- start 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.631+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc36860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.631+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc34770 0x55ee4cc36da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.631+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc398e0 0x55ee4cc37db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.631+0000 7fc563fd7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cc3c830 con 0x55ee4cc33d80 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc563fd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cb4fed0 con 0x55ee4cc34770 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc563fd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cc3c9b0 con 0x55ee4cc398e0 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc563fd7740 10 client.0 _ll_register_callbacks cb 0x55ee4cb8b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc559e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc36860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc55a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc398e0 0x55ee4cc37db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc553e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc34770 0x55ee4cc36da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc55a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc398e0 0x55ee4cc37db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51358/0 (socket says 172.21.15.97:51358) 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc553e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc34770 0x55ee4cc36da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44144/0 (socket says 172.21.15.97:44144) 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc55a800640 1 -- 172.21.15.97:0/2512817422 learned_addr learned my addr 172.21.15.97:0/2512817422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc559e00640 1 -- 172.21.15.97:0/2512817422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc398e0 msgr2=0x55ee4cc37db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc559e00640 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc398e0 0x55ee4cc37db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc559e00640 1 -- 172.21.15.97:0/2512817422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc34770 msgr2=0x55ee4cc36da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc559e00640 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc34770 0x55ee4cc36da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc559e00640 1 -- 172.21.15.97:0/2512817422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4c9726b0 con 0x55ee4cc33d80 2024-03-26T18:58:45.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc55a800640 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc398e0 0x55ee4cc37db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc553e00640 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc34770 0x55ee4cc36da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc559e00640 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc36860 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55ee4cc351e0 tx=0x7fc54c09a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62295abc89645cec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.632+0000 7fc53fe00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.633+0000 7fc53fe00640 1 -- 172.21.15.97:0/2512817422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c09bb70 con 0x55ee4cc33d80 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.633+0000 7fc53fe00640 1 -- 172.21.15.97:0/2512817422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc54c09b480 con 0x55ee4cc33d80 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.633+0000 7fc563fd7740 1 -- 172.21.15.97:0/2512817422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc544034640 con 0x55ee4cc33d80 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.633+0000 7fc53fe00640 1 -- 172.21.15.97:0/2512817422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc54c09a170 con 0x55ee4cc33d80 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.633+0000 7fc563fd7740 10 client.15069 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.633+0000 7fc563fd7740 1 -- 172.21.15.97:0/2512817422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee4cc4cc60 con 0x55ee4cc33d80 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.633+0000 7fc53fe00640 1 -- 172.21.15.97:0/2512817422 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc544034640 con 0x55ee4cc33d80 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.633+0000 7fc53fe00640 1 -- 172.21.15.97:0/2512817422 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc54c09aef0 con 0x55ee4cc33d80 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.634+0000 7fc563fd7740 10 client.15069 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.634+0000 7fc563fd7740 10 client.15069 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T18:58:45.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.634+0000 7fc563fd7740 20 client.15069 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.634+0000 7fc563fd7740 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x55ee4cc4f220 0x55ee4cc50690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.634+0000 7fc563fd7740 4 client.15069 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.634+0000 7fc563fd7740 1 -- 172.21.15.97:0/2512817422 --> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee4cba16f0 con 0x55ee4cc4f220 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.634+0000 7fc553e00640 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x55ee4cc4f220 0x55ee4cc50690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.634+0000 7fc553e00640 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x55ee4cc4f220 0x55ee4cc50690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=862536aad1bd61ed server_cookie=327ae78ffc0656b4 in_seq=0 out_seq=0 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.634+0000 7fc53fe00640 10 client.15069 ms_handle_connect on v2:172.21.15.105:6834/1538692215 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc53fe00640 1 -- 172.21.15.97:0/2512817422 <== mds.3 v2:172.21.15.105:6834/1538692215 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ee4cba16f0 con 0x55ee4cc4f220 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc53fe00640 10 client.15069 handle_command_reply: tid=0 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc563fd7740 1 client.15069 shutdown 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc563fd7740 1 -- 172.21.15.97:0/2512817422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 msgr2=0x55ee4cc36860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc563fd7740 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc36860 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55ee4cc351e0 tx=0x7fc54c09a040 comp rx=0 tx=0).stop 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc563fd7740 1 -- 172.21.15.97:0/2512817422 shutdown_connections 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc563fd7740 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x55ee4cc4f220 0x55ee4cc50690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc563fd7740 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc398e0 0x55ee4cc37db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc563fd7740 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc34770 0x55ee4cc36da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc563fd7740 1 --2- 172.21.15.97:0/2512817422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc36860 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc53fe00640 0 client.15069 ms_handle_reset on v2:172.21.15.105:6834/1538692215 2024-03-26T18:58:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.635+0000 7fc563fd7740 1 -- 172.21.15.97:0/2512817422 >> 172.21.15.97:0/2512817422 conn(0x55ee4cc2aee0 msgr2=0x55ee4cc2a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.636+0000 7fc563fd7740 1 -- 172.21.15.97:0/2512817422 shutdown_connections 2024-03-26T18:58:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.636+0000 7fc563fd7740 1 -- 172.21.15.97:0/2512817422 wait complete. 2024-03-26T18:58:45.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.636+0000 7fc563fd7740 20 client.15069 trim_cache size 0 max 16384 2024-03-26T18:58:45.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.655+0000 7fc563fd7740 1 Processor -- start 2024-03-26T18:58:45.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.655+0000 7fc563fd7740 1 -- start start 2024-03-26T18:58:45.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc37d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc3bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4cc4e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc563fd7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cc4e9d0 con 0x55ee4cc33d80 2024-03-26T18:58:45.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc563fd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cc4f0a0 con 0x55ee4cc398e0 2024-03-26T18:58:45.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc563fd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cc4f770 con 0x55ee4cc34770 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc559e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc37d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc553e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc3bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc55a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4cc4e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc553e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc3bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51368/0 (socket says 172.21.15.97:51368) 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc553e00640 1 -- 172.21.15.97:0/1075431484 learned_addr learned my addr 172.21.15.97:0/1075431484 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc559e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc37d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36910/0 (socket says 172.21.15.97:36910) 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc553e00640 1 -- 172.21.15.97:0/1075431484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 msgr2=0x55ee4cc4e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc553e00640 1 --2- 172.21.15.97:0/1075431484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4cc4e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc553e00640 1 -- 172.21.15.97:0/1075431484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 msgr2=0x55ee4cc37d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc553e00640 1 --2- 172.21.15.97:0/1075431484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc37d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.656+0000 7fc553e00640 1 -- 172.21.15.97:0/1075431484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4cc29520 con 0x55ee4cc34770 2024-03-26T18:58:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.657+0000 7fc553e00640 1 --2- 172.21.15.97:0/1075431484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc3bcd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc544005b60 tx=0x7fc544005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1636e6d2879567 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.657+0000 7fc54aa00640 1 -- 172.21.15.97:0/1075431484 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544058070 con 0x55ee4cc34770 2024-03-26T18:58:45.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.657+0000 7fc54aa00640 1 -- 172.21.15.97:0/1075431484 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc54404e070 con 0x55ee4cc34770 2024-03-26T18:58:45.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.657+0000 7fc54aa00640 1 -- 172.21.15.97:0/1075431484 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544044040 con 0x55ee4cc34770 2024-03-26T18:58:45.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.658+0000 7fc563fd7740 1 -- 172.21.15.97:0/1075431484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 msgr2=0x55ee4cc3bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.658+0000 7fc563fd7740 1 --2- 172.21.15.97:0/1075431484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc3bcd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc544005b60 tx=0x7fc544005d30 comp rx=0 tx=0).stop 2024-03-26T18:58:45.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.658+0000 7fc563fd7740 1 -- 172.21.15.97:0/1075431484 shutdown_connections 2024-03-26T18:58:45.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.658+0000 7fc563fd7740 1 --2- 172.21.15.97:0/1075431484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4cc4e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.658+0000 7fc563fd7740 1 --2- 172.21.15.97:0/1075431484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc3bcd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.658+0000 7fc563fd7740 1 --2- 172.21.15.97:0/1075431484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc37d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.658+0000 7fc563fd7740 1 -- 172.21.15.97:0/1075431484 >> 172.21.15.97:0/1075431484 conn(0x55ee4cc2aee0 msgr2=0x55ee4cc2b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.658+0000 7fc563fd7740 1 -- 172.21.15.97:0/1075431484 shutdown_connections 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.658+0000 7fc563fd7740 1 -- 172.21.15.97:0/1075431484 wait complete. 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.658+0000 7fc563fd7740 1 -- start start 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.659+0000 7fc563fd7740 1 Processor -- start 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.659+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.659+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc32a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.659+0000 7fc563fd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4ccb2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.659+0000 7fc563fd7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cc50510 con 0x55ee4cc33d80 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.659+0000 7fc563fd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cc4fe40 con 0x55ee4cc398e0 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.659+0000 7fc563fd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee4cc50be0 con 0x55ee4cc34770 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.659+0000 7fc563fd7740 10 client.0 _ll_register_callbacks cb 0x7fc54c0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc559e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc553e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc32a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc553e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc32a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51376/0 (socket says 172.21.15.97:51376) 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc553e00640 1 -- 172.21.15.97:0/977732291 learned_addr learned my addr 172.21.15.97:0/977732291 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc55a800640 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4ccb2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc559e00640 1 -- 172.21.15.97:0/977732291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 msgr2=0x55ee4cc32a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc559e00640 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc32a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc559e00640 1 -- 172.21.15.97:0/977732291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 msgr2=0x55ee4ccb2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc559e00640 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4ccb2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc559e00640 1 -- 172.21.15.97:0/977732291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4ccb3790 con 0x55ee4cc33d80 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc553e00640 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc32a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc559e00640 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc324d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc54c09a0b0 tx=0x7fc54c09b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77e723fff073408 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc53fe00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc53fe00640 1 -- 172.21.15.97:0/977732291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c09b990 con 0x55ee4cc33d80 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc53fe00640 1 -- 172.21.15.97:0/977732291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc54c097ef0 con 0x55ee4cc33d80 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.660+0000 7fc563fd7740 1 -- 172.21.15.97:0/977732291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc54404e070 con 0x55ee4cc33d80 2024-03-26T18:58:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc53fe00640 1 -- 172.21.15.97:0/977732291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc54c09bb70 con 0x55ee4cc33d80 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc563fd7740 10 client.15075 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc563fd7740 1 -- 172.21.15.97:0/977732291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee4ccb4b00 con 0x55ee4cc33d80 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc53fe00640 1 -- 172.21.15.97:0/977732291 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc54404e070 con 0x55ee4cc33d80 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc53fe00640 1 -- 172.21.15.97:0/977732291 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc54c004000 con 0x55ee4cc33d80 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc563fd7740 10 client.15075 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc563fd7740 10 client.15075 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc563fd7740 20 client.15075 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc563fd7740 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x55ee4ccb9700 0x55ee4ccb8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc563fd7740 4 client.15075 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.661+0000 7fc563fd7740 1 -- 172.21.15.97:0/977732291 --> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] -- command(tid 0: {"prefix": "status"}) -- 0x55ee4cb9ba40 con 0x55ee4ccb9700 2024-03-26T18:58:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc553e00640 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x55ee4ccb9700 0x55ee4ccb8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc553e00640 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x55ee4ccb9700 0x55ee4ccb8410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e1215338a13bb0e5 server_cookie=2c8f5376b92474c2 in_seq=0 out_seq=0 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc53fe00640 10 client.15075 ms_handle_connect on v2:172.21.15.105:6834/1538692215 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc53fe00640 1 -- 172.21.15.97:0/977732291 <== mds.3 v2:172.21.15.105:6834/1538692215 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ee4cb9ba40 con 0x55ee4ccb9700 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc53fe00640 10 client.15075 handle_command_reply: tid=0 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc563fd7740 1 client.15075 shutdown 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc563fd7740 1 -- 172.21.15.97:0/977732291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 msgr2=0x55ee4cc324d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc563fd7740 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc324d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc54c09a0b0 tx=0x7fc54c09b3a0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc563fd7740 1 -- 172.21.15.97:0/977732291 shutdown_connections 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc563fd7740 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.105:6834/1538692215,v1:172.21.15.105:6835/1538692215] conn(0x55ee4ccb9700 0x55ee4ccb8410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc563fd7740 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee4cc398e0 0x55ee4ccb2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc563fd7740 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee4cc34770 0x55ee4cc32a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc563fd7740 1 --2- 172.21.15.97:0/977732291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee4cc33d80 0x55ee4cc324d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.662+0000 7fc563fd7740 1 -- 172.21.15.97:0/977732291 >> 172.21.15.97:0/977732291 conn(0x55ee4cc2aee0 msgr2=0x55ee4cc4e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.663+0000 7fc53fe00640 0 client.15075 ms_handle_reset on v2:172.21.15.105:6834/1538692215 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.663+0000 7fc563fd7740 1 -- 172.21.15.97:0/977732291 shutdown_connections 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.663+0000 7fc563fd7740 1 -- 172.21.15.97:0/977732291 wait complete. 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.663+0000 7fc563fd7740 20 client.15075 trim_cache size 0 max 16384 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 3, 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24494, 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-26T18:58:45.661 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 135.846946441, 2024-03-26T18:58:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-03-26T18:58:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2024-03-26T18:58:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T18:58:45.662 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 170.66326509800001 2024-03-26T18:58:45.662 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.663+0000 7fc561200640 1 -- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc5400a2f20 msgr2=0x7fc5400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.663+0000 7fc561200640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc5400a2f20 0x7fc5400a53e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55ee4cc34fd0 tx=0x7fc554011040 comp rx=0 tx=0).stop 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.663+0000 7fc561200640 1 -- 172.21.15.97:0/993761135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc55c1018f0 msgr2=0x7fc55c197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.663+0000 7fc561200640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc55c1018f0 0x7fc55c197c50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00e9c0 tx=0x7fc54c00ee80 comp rx=0 tx=0).stop 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.664+0000 7fc561200640 1 -- 172.21.15.97:0/993761135 shutdown_connections 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.664+0000 7fc561200640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc5400a2f20 0x7fc5400a53e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.664+0000 7fc561200640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc55c1034a0 0x7fc55c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.664+0000 7fc561200640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc55c102ae0 0x7fc55c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.664+0000 7fc561200640 1 --2- 172.21.15.97:0/993761135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc55c1018f0 0x7fc55c197c50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.664+0000 7fc561200640 1 -- 172.21.15.97:0/993761135 >> 172.21.15.97:0/993761135 conn(0x7fc55c0fce60 msgr2=0x7fc55c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.664+0000 7fc561200640 1 -- 172.21.15.97:0/993761135 shutdown_connections 2024-03-26T18:58:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.664+0000 7fc561200640 1 -- 172.21.15.97:0/993761135 wait complete. 2024-03-26T18:58:45.675 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-26T18:58:45.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.896+0000 7f1161200640 1 -- 172.21.15.97:0/2156587518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f115c102ac0 msgr2=0x7f115c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.896+0000 7f1161200640 1 --2- 172.21.15.97:0/2156587518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f115c102ac0 0x7f115c102f40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1144009860 tx=0x7f114402bb70 comp rx=0 tx=0).stop 2024-03-26T18:58:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.896+0000 7f1161200640 1 -- 172.21.15.97:0/2156587518 shutdown_connections 2024-03-26T18:58:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.896+0000 7f1161200640 1 --2- 172.21.15.97:0/2156587518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f115c103480 0x7f115c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.897+0000 7f1161200640 1 --2- 172.21.15.97:0/2156587518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f115c102ac0 0x7f115c102f40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.897+0000 7f1161200640 1 --2- 172.21.15.97:0/2156587518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f115c1018d0 0x7f115c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.897+0000 7f1161200640 1 -- 172.21.15.97:0/2156587518 >> 172.21.15.97:0/2156587518 conn(0x7f115c0fce40 msgr2=0x7f115c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.897+0000 7f1161200640 1 -- 172.21.15.97:0/2156587518 shutdown_connections 2024-03-26T18:58:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.897+0000 7f1161200640 1 -- 172.21.15.97:0/2156587518 wait complete. 2024-03-26T18:58:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.898+0000 7f1161200640 1 Processor -- start 2024-03-26T18:58:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.898+0000 7f1161200640 1 -- start start 2024-03-26T18:58:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1161200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f115c1018d0 0x7f115c199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1161200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f115c102ac0 0x7f115c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1161200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f115c103480 0x7f115c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1161200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f115c10d550 con 0x7f115c1018d0 2024-03-26T18:58:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1161200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f115c10d3d0 con 0x7f115c103480 2024-03-26T18:58:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1161200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f115c10d6d0 con 0x7f115c102ac0 2024-03-26T18:58:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1159e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f115c1018d0 0x7f115c199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1153e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f115c102ac0 0x7f115c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1153e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f115c102ac0 0x7f115c19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51386/0 (socket says 172.21.15.97:51386) 2024-03-26T18:58:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1159e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f115c1018d0 0x7f115c199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36930/0 (socket says 172.21.15.97:36930) 2024-03-26T18:58:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1153e00640 1 -- 172.21.15.97:0/3869126682 learned_addr learned my addr 172.21.15.97:0/3869126682 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f115a800640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f115c103480 0x7f115c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1153e00640 1 -- 172.21.15.97:0/3869126682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f115c103480 msgr2=0x7f115c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1153e00640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f115c103480 0x7f115c1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1153e00640 1 -- 172.21.15.97:0/3869126682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f115c1018d0 msgr2=0x7f115c199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1153e00640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f115c1018d0 0x7f115c199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.899+0000 7f1153e00640 1 -- 172.21.15.97:0/3869126682 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115c1a1a30 con 0x7f115c102ac0 2024-03-26T18:58:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.900+0000 7f1153e00640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f115c102ac0 0x7f115c19a350 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f11440349e0 tx=0x7f1144032450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.900+0000 7f1152a00640 1 -- 172.21.15.97:0/3869126682 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144041030 con 0x7f115c102ac0 2024-03-26T18:58:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.900+0000 7f1152a00640 1 -- 172.21.15.97:0/3869126682 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114402c510 con 0x7f115c102ac0 2024-03-26T18:58:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.900+0000 7f1161200640 1 -- 172.21.15.97:0/3869126682 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115c1a1cc0 con 0x7f115c102ac0 2024-03-26T18:58:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.900+0000 7f1152a00640 1 -- 172.21.15.97:0/3869126682 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144039040 con 0x7f115c102ac0 2024-03-26T18:58:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.900+0000 7f1161200640 1 -- 172.21.15.97:0/3869126682 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115c1a21b0 con 0x7f115c102ac0 2024-03-26T18:58:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.904+0000 7f1163f69740 1 Processor -- start 2024-03-26T18:58:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.904+0000 7f1152a00640 1 -- 172.21.15.97:0/3869126682 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f1144044070 con 0x7f115c102ac0 2024-03-26T18:58:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f1163f69740 1 -- start start 2024-03-26T18:58:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f1163f69740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb84970 0x55f4abb84d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f1163f69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb89e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f1163f69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb8a4d0 0x55f4abb8c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f1163f69740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f4abae91b0 con 0x55f4abb84970 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f1163f69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f4aba91b00 con 0x55f4abb8a4d0 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f1163f69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4abae5160 con 0x55f4abb85360 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f115a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb84970 0x55f4abb84d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f1159e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb89e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f1153e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb8a4d0 0x55f4abb8c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f115a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb84970 0x55f4abb84d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36932/0 (socket says 172.21.15.97:36932) 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f1159e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb89e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51396/0 (socket says 172.21.15.97:51396) 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.905+0000 7f115a800640 1 -- 172.21.15.97:0/4001704274 learned_addr learned my addr 172.21.15.97:0/4001704274 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f1159e00640 1 -- 172.21.15.97:0/4001704274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb8a4d0 msgr2=0x55f4abb8c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f1159e00640 1 --2- 172.21.15.97:0/4001704274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb8a4d0 0x55f4abb8c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f1159e00640 1 -- 172.21.15.97:0/4001704274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb84970 msgr2=0x55f4abb84d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f1159e00640 1 --2- 172.21.15.97:0/4001704274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb84970 0x55f4abb84d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f1159e00640 1 -- 172.21.15.97:0/4001704274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ab7fd2d0 con 0x55f4abb85360 2024-03-26T18:58:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f1159e00640 1 --2- 172.21.15.97:0/4001704274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb89e60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f115c102920 tx=0x7f114c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcddf5372ca81e0f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f1152a00640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11400a2fa0 0x7f11400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f114b400640 1 -- 172.21.15.97:0/4001704274 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c01e070 con 0x55f4abb85360 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f114b400640 1 -- 172.21.15.97:0/4001704274 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c019910 con 0x55f4abb85360 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f114b400640 1 -- 172.21.15.97:0/4001704274 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c00ea10 con 0x55f4abb85360 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f1152a00640 1 -- 172.21.15.97:0/3869126682 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f11440f0850 con 0x7f115c102ac0 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.906+0000 7f115a800640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11400a2fa0 0x7f11400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.907+0000 7f1163f69740 1 -- 172.21.15.97:0/4001704274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 msgr2=0x55f4abb89e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.907+0000 7f1163f69740 1 --2- 172.21.15.97:0/4001704274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb89e60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f115c102920 tx=0x7f114c011810 comp rx=0 tx=0).stop 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.907+0000 7f1163f69740 1 -- 172.21.15.97:0/4001704274 shutdown_connections 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.907+0000 7f1163f69740 1 --2- 172.21.15.97:0/4001704274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb8a4d0 0x55f4abb8c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.907+0000 7f1163f69740 1 --2- 172.21.15.97:0/4001704274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb89e60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.907+0000 7f1163f69740 1 --2- 172.21.15.97:0/4001704274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb84970 0x55f4abb84d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.907+0000 7f1163f69740 1 -- 172.21.15.97:0/4001704274 >> 172.21.15.97:0/4001704274 conn(0x55f4abb7fee0 msgr2=0x55f4abb802e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.907+0000 7f115a800640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11400a2fa0 0x7f11400a5460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55f4abb85bc0 tx=0x7f1154009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.907+0000 7f1163f69740 1 -- 172.21.15.97:0/4001704274 shutdown_connections 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.907+0000 7f1163f69740 1 -- 172.21.15.97:0/4001704274 wait complete. 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.908+0000 7f1163f69740 1 -- start start 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.908+0000 7f1163f69740 1 Processor -- start 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.908+0000 7f1163f69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb84970 0x55f4abb8eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.908+0000 7f1163f69740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb85360 0x55f4abb8f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.908+0000 7f1163f69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb8a4d0 0x55f4abb96420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1163f69740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f4abb8d480 con 0x55f4abb85360 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1163f69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f4abaa4ed0 con 0x55f4abb8a4d0 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1163f69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4abb8d600 con 0x55f4abb84970 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1163f69740 10 client.0 _ll_register_callbacks cb 0x55f4abae0710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1159e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb84970 0x55f4abb8eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1159e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb84970 0x55f4abb8eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51406/0 (socket says 172.21.15.97:51406) 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1159e00640 1 -- 172.21.15.97:0/4189593917 learned_addr learned my addr 172.21.15.97:0/4189593917 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f115a800640 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb8a4d0 0x55f4abb96420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1153e00640 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb85360 0x55f4abb8f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1159e00640 1 -- 172.21.15.97:0/4189593917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb8a4d0 msgr2=0x55f4abb96420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1159e00640 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb8a4d0 0x55f4abb96420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1159e00640 1 -- 172.21.15.97:0/4189593917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb85360 msgr2=0x55f4abb8f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1159e00640 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb85360 0x55f4abb8f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1159e00640 1 -- 172.21.15.97:0/4189593917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4ab8c76b0 con 0x55f4abb84970 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1153e00640 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb85360 0x55f4abb8f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.909+0000 7f1159e00640 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb84970 0x55f4abb8eea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f114c000c00 tx=0x7f114c019f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a906163579228688 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.910+0000 7f113fe00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.910+0000 7f113fe00640 1 -- 172.21.15.97:0/4189593917 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c029020 con 0x55f4abb84970 2024-03-26T18:58:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.910+0000 7f113fe00640 1 -- 172.21.15.97:0/4189593917 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f114c019630 con 0x55f4abb84970 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.910+0000 7f1163f69740 1 -- 172.21.15.97:0/4189593917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114c019910 con 0x55f4abb84970 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.910+0000 7f113fe00640 1 -- 172.21.15.97:0/4189593917 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f114c01e040 con 0x55f4abb84970 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.910+0000 7f1163f69740 10 client.25192 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.910+0000 7f1163f69740 1 -- 172.21.15.97:0/4189593917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4abb99400 con 0x55f4abb84970 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.910+0000 7f113fe00640 1 -- 172.21.15.97:0/4189593917 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c019910 con 0x55f4abb84970 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.911+0000 7f113fe00640 1 -- 172.21.15.97:0/4189593917 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f114c00ec80 con 0x55f4abb84970 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.911+0000 7f1163f69740 10 client.25192 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.911+0000 7f1163f69740 10 client.25192 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.911+0000 7f1163f69740 20 client.25192 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.911+0000 7f1163f69740 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x55f4abb9b7c0 0x55f4abb9cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.911+0000 7f1163f69740 4 client.25192 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.911+0000 7f1163f69740 1 -- 172.21.15.97:0/4189593917 --> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4abaf66f0 con 0x55f4abb9b7c0 2024-03-26T18:58:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.911+0000 7f1153e00640 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x55f4abb9b7c0 0x55f4abb9cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.912+0000 7f1153e00640 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x55f4abb9b7c0 0x55f4abb9cc30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d992594cc817244 server_cookie=246b1bb9d62698c6 in_seq=0 out_seq=0 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.912+0000 7f113fe00640 10 client.25192 ms_handle_connect on v2:172.21.15.144:6838/994192292 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.912+0000 7f113fe00640 1 -- 172.21.15.97:0/4189593917 <== mds.2 v2:172.21.15.144:6838/994192292 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f4abaf66f0 con 0x55f4abb9b7c0 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.912+0000 7f113fe00640 10 client.25192 handle_command_reply: tid=0 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.912+0000 7f1163f69740 1 client.25192 shutdown 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.912+0000 7f1163f69740 1 -- 172.21.15.97:0/4189593917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb84970 msgr2=0x55f4abb8eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.912+0000 7f1163f69740 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb84970 0x55f4abb8eea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f114c000c00 tx=0x7f114c019f10 comp rx=0 tx=0).stop 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.913+0000 7f1163f69740 1 -- 172.21.15.97:0/4189593917 shutdown_connections 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.913+0000 7f1163f69740 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x55f4abb9b7c0 0x55f4abb9cc30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.913+0000 7f1163f69740 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb8a4d0 0x55f4abb96420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.913+0000 7f1163f69740 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb85360 0x55f4abb8f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.913+0000 7f1163f69740 1 --2- 172.21.15.97:0/4189593917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb84970 0x55f4abb8eea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.913+0000 7f1163f69740 1 -- 172.21.15.97:0/4189593917 >> 172.21.15.97:0/4189593917 conn(0x55f4abb7fee0 msgr2=0x55f4abb7f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.913+0000 7f113fe00640 0 client.25192 ms_handle_reset on v2:172.21.15.144:6838/994192292 2024-03-26T18:58:45.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.913+0000 7f1163f69740 1 -- 172.21.15.97:0/4189593917 shutdown_connections 2024-03-26T18:58:45.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.913+0000 7f1163f69740 1 -- 172.21.15.97:0/4189593917 wait complete. 2024-03-26T18:58:45.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.913+0000 7f1163f69740 20 client.25192 trim_cache size 0 max 16384 2024-03-26T18:58:45.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.932+0000 7f1163f69740 1 Processor -- start 2024-03-26T18:58:45.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.932+0000 7f1163f69740 1 -- start start 2024-03-26T18:58:45.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.932+0000 7f1163f69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb89e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.932+0000 7f1163f69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1163f69740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb8a4d0 0x55f4abb85dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1163f69740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f4abb9af70 con 0x55f4abb8a4d0 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1163f69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f4abb9b640 con 0x55f4abb84970 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1163f69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4abb9bd10 con 0x55f4abb85360 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1159e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb89e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1153e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1159e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb89e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44194/0 (socket says 172.21.15.97:44194) 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1159e00640 1 -- 172.21.15.97:0/3170316629 learned_addr learned my addr 172.21.15.97:0/3170316629 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1153e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51414/0 (socket says 172.21.15.97:51414) 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f115a800640 1 --2- 172.21.15.97:0/3170316629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb8a4d0 0x55f4abb85dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1159e00640 1 -- 172.21.15.97:0/3170316629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 msgr2=0x55f4abb963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1159e00640 1 --2- 172.21.15.97:0/3170316629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1159e00640 1 -- 172.21.15.97:0/3170316629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb8a4d0 msgr2=0x55f4abb85dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1159e00640 1 --2- 172.21.15.97:0/3170316629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb8a4d0 0x55f4abb85dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1159e00640 1 -- 172.21.15.97:0/3170316629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4abb89b10 con 0x55f4abb84970 2024-03-26T18:58:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.933+0000 7f1159e00640 1 --2- 172.21.15.97:0/3170316629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb89e90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f114c009b80 tx=0x7f114c004930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7009b5f4daecb2fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f114aa00640 1 -- 172.21.15.97:0/3170316629 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c029040 con 0x55f4abb84970 2024-03-26T18:58:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f114aa00640 1 -- 172.21.15.97:0/3170316629 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c009870 con 0x55f4abb84970 2024-03-26T18:58:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f114aa00640 1 -- 172.21.15.97:0/3170316629 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c025420 con 0x55f4abb84970 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f1163f69740 1 -- 172.21.15.97:0/3170316629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 msgr2=0x55f4abb89e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f1163f69740 1 --2- 172.21.15.97:0/3170316629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb89e90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f114c009b80 tx=0x7f114c004930 comp rx=0 tx=0).stop 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f1163f69740 1 -- 172.21.15.97:0/3170316629 shutdown_connections 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f1163f69740 1 --2- 172.21.15.97:0/3170316629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb8a4d0 0x55f4abb85dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f1163f69740 1 --2- 172.21.15.97:0/3170316629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abb963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f1163f69740 1 --2- 172.21.15.97:0/3170316629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb89e90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f1163f69740 1 -- 172.21.15.97:0/3170316629 >> 172.21.15.97:0/3170316629 conn(0x55f4abb7fee0 msgr2=0x55f4abb7f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f1163f69740 1 -- 172.21.15.97:0/3170316629 shutdown_connections 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.934+0000 7f1163f69740 1 -- 172.21.15.97:0/3170316629 wait complete. 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.935+0000 7f1163f69740 1 -- start start 2024-03-26T18:58:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.935+0000 7f1163f69740 1 Processor -- start 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.935+0000 7f1163f69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb94330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1163f69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abbfd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1163f69740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb8a4d0 0x55f4abbffa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1163f69740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f4abb9cab0 con 0x55f4abb8a4d0 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1163f69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f4abb9c3e0 con 0x55f4abb84970 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1163f69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4abb9d180 con 0x55f4abb85360 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1163f69740 10 client.0 _ll_register_callbacks cb 0x7f114c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1159e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb94330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1153e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abbfd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f115a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb8a4d0 0x55f4abbffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1159e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb94330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44204/0 (socket says 172.21.15.97:44204) 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1153e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abbfd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51426/0 (socket says 172.21.15.97:51426) 2024-03-26T18:58:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1159e00640 1 -- 172.21.15.97:0/2920069671 learned_addr learned my addr 172.21.15.97:0/2920069671 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1159e00640 1 -- 172.21.15.97:0/2920069671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 msgr2=0x55f4abbfd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1159e00640 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abbfd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1159e00640 1 -- 172.21.15.97:0/2920069671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb8a4d0 msgr2=0x55f4abbffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1159e00640 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb8a4d0 0x55f4abbffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.936+0000 7f1159e00640 1 -- 172.21.15.97:0/2920069671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4abc000d0 con 0x55f4abb84970 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.937+0000 7f1159e00640 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb94330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f114c006010 tx=0x7f114c019a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae00c18aaf7171b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.937+0000 7f113fe00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.937+0000 7f113fe00640 1 -- 172.21.15.97:0/2920069671 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c029040 con 0x55f4abb84970 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.937+0000 7f1163f69740 1 -- 172.21.15.97:0/2920069671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114c009870 con 0x55f4abb84970 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.937+0000 7f113fe00640 1 -- 172.21.15.97:0/2920069671 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f114c004f30 con 0x55f4abb84970 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.937+0000 7f113fe00640 1 -- 172.21.15.97:0/2920069671 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f114c048050 con 0x55f4abb84970 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.937+0000 7f1163f69740 10 client.25226 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.937+0000 7f1163f69740 1 -- 172.21.15.97:0/2920069671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4abc013b0 con 0x55f4abb84970 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.937+0000 7f113fe00640 1 -- 172.21.15.97:0/2920069671 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c009870 con 0x55f4abb84970 2024-03-26T18:58:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.937+0000 7f113fe00640 1 -- 172.21.15.97:0/2920069671 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f114c037020 con 0x55f4abb84970 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.938+0000 7f1163f69740 10 client.25226 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.938+0000 7f1163f69740 10 client.25226 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.938+0000 7f1163f69740 20 client.25226 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.938+0000 7f1163f69740 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x55f4abc05b60 0x55f4abc04870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.938+0000 7f1163f69740 4 client.25226 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.938+0000 7f1163f69740 1 -- 172.21.15.97:0/2920069671 --> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] -- command(tid 0: {"prefix": "status"}) -- 0x55f4abaf0a40 con 0x55f4abc05b60 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.938+0000 7f1153e00640 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x55f4abc05b60 0x55f4abc04870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.938+0000 7f1153e00640 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x55f4abc05b60 0x55f4abc04870 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca06e2246bb96bc5 server_cookie=f341e62a89d81371 in_seq=0 out_seq=0 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.938+0000 7f113fe00640 10 client.25226 ms_handle_connect on v2:172.21.15.144:6838/994192292 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f113fe00640 1 -- 172.21.15.97:0/2920069671 <== mds.2 v2:172.21.15.144:6838/994192292 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f4abaf0a40 con 0x55f4abc05b60 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f113fe00640 10 client.25226 handle_command_reply: tid=0 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 client.25226 shutdown 2024-03-26T18:58:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 -- 172.21.15.97:0/2920069671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 msgr2=0x55f4abb94330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb94330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f114c006010 tx=0x7f114c019a50 comp rx=0 tx=0).stop 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 -- 172.21.15.97:0/2920069671 shutdown_connections 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.144:6838/994192292,v1:172.21.15.144:6839/994192292] conn(0x55f4abc05b60 0x55f4abc04870 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4abb8a4d0 0x55f4abbffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4abb85360 0x55f4abbfd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f113fe00640 0 client.25226 ms_handle_reset on v2:172.21.15.144:6838/994192292 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 --2- 172.21.15.97:0/2920069671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4abb84970 0x55f4abb94330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 -- 172.21.15.97:0/2920069671 >> 172.21.15.97:0/2920069671 conn(0x55f4abb7fee0 msgr2=0x55f4abb7f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 -- 172.21.15.97:0/2920069671 shutdown_connections 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 1 -- 172.21.15.97:0/2920069671 wait complete. 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.939+0000 7f1163f69740 20 client.25226 trim_cache size 0 max 16384 2024-03-26T18:58:45.937 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:58:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:58:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2024-03-26T18:58:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24535, 2024-03-26T18:58:45.938 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-26T18:58:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-26T18:58:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-26T18:58:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 137.090259352, 2024-03-26T18:58:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-03-26T18:58:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2024-03-26T18:58:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T18:58:45.939 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 159.43306099099999 2024-03-26T18:58:45.939 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 -- 172.21.15.97:0/3869126682 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11400a2fa0 msgr2=0x7f11400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11400a2fa0 0x7f11400a5460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55f4abb85bc0 tx=0x7f1154009500 comp rx=0 tx=0).stop 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 -- 172.21.15.97:0/3869126682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f115c102ac0 msgr2=0x7f115c19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f115c102ac0 0x7f115c19a350 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f11440349e0 tx=0x7f1144032450 comp rx=0 tx=0).stop 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 -- 172.21.15.97:0/3869126682 shutdown_connections 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f11400a2fa0 0x7f11400a5460 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f115c103480 0x7f115c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f115c102ac0 0x7f115c19a350 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 --2- 172.21.15.97:0/3869126682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f115c1018d0 0x7f115c199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 -- 172.21.15.97:0/3869126682 >> 172.21.15.97:0/3869126682 conn(0x7f115c0fce40 msgr2=0x7f115c073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 -- 172.21.15.97:0/3869126682 shutdown_connections 2024-03-26T18:58:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:45.940+0000 7f1161200640 1 -- 172.21.15.97:0/3869126682 wait complete. 2024-03-26T18:58:45.953 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-26T18:58:46.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.156+0000 7fca75200640 1 -- 172.21.15.97:0/281568045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca701075b0 msgr2=0x7fca70109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.156+0000 7fca75200640 1 --2- 172.21.15.97:0/281568045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca701075b0 0x7fca70109a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca68009be0 tx=0x7fca6802bef0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.156+0000 7fca75200640 1 -- 172.21.15.97:0/281568045 shutdown_connections 2024-03-26T18:58:46.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.156+0000 7fca75200640 1 --2- 172.21.15.97:0/281568045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca701075b0 0x7fca70109a00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.156+0000 7fca75200640 1 --2- 172.21.15.97:0/281568045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca70104c80 0x7fca70107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.156+0000 7fca75200640 1 --2- 172.21.15.97:0/281568045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca70102350 0x7fca70104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.156+0000 7fca75200640 1 -- 172.21.15.97:0/281568045 >> 172.21.15.97:0/281568045 conn(0x7fca700fbd60 msgr2=0x7fca700fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.156+0000 7fca75200640 1 -- 172.21.15.97:0/281568045 shutdown_connections 2024-03-26T18:58:46.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.156+0000 7fca75200640 1 -- 172.21.15.97:0/281568045 wait complete. 2024-03-26T18:58:46.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.157+0000 7fca75200640 1 Processor -- start 2024-03-26T18:58:46.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.157+0000 7fca75200640 1 -- start start 2024-03-26T18:58:46.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca75200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca70102350 0x7fca70197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca75200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70104c80 0x7fca70198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca75200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca701075b0 0x7fca7019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca75200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca7010d5c0 con 0x7fca701075b0 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca75200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fca7010d440 con 0x7fca70102350 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca75200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca7010d740 con 0x7fca70104c80 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca6de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca70102350 0x7fca70197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca6de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca70102350 0x7fca70197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44224/0 (socket says 172.21.15.97:44224) 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca6e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca701075b0 0x7fca7019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca6d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70104c80 0x7fca70198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.158+0000 7fca6de00640 1 -- 172.21.15.97:0/3305366953 learned_addr learned my addr 172.21.15.97:0/3305366953 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca6de00640 1 -- 172.21.15.97:0/3305366953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70104c80 msgr2=0x7fca70198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca6de00640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70104c80 0x7fca70198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca6de00640 1 -- 172.21.15.97:0/3305366953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca701075b0 msgr2=0x7fca7019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca6de00640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca701075b0 0x7fca7019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca6de00640 1 -- 172.21.15.97:0/3305366953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7019f7f0 con 0x7fca70102350 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca6d400640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70104c80 0x7fca70198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca6de00640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca70102350 0x7fca70197bd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fca6000e9c0 tx=0x7fca6000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca66a00640 1 -- 172.21.15.97:0/3305366953 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6000cde0 con 0x7fca70102350 2024-03-26T18:58:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca66a00640 1 -- 172.21.15.97:0/3305366953 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca600035f0 con 0x7fca70102350 2024-03-26T18:58:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca75200640 1 -- 172.21.15.97:0/3305366953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca701001a0 con 0x7fca70102350 2024-03-26T18:58:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca66a00640 1 -- 172.21.15.97:0/3305366953 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60010690 con 0x7fca70102350 2024-03-26T18:58:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.159+0000 7fca75200640 1 -- 172.21.15.97:0/3305366953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca701006f0 con 0x7fca70102350 2024-03-26T18:58:46.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.162+0000 7fca66a00640 1 -- 172.21.15.97:0/3305366953 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fca600037a0 con 0x7fca70102350 2024-03-26T18:58:46.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.163+0000 7fca77fc5740 1 Processor -- start 2024-03-26T18:58:46.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.163+0000 7fca77fc5740 1 -- start start 2024-03-26T18:58:46.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.163+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c2a970 0x556805c2ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.163+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c2fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.163+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 0x556805c32960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.163+0000 7fca77fc5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556805b8f1b0 con 0x556805c2a970 2024-03-26T18:58:46.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.163+0000 7fca77fc5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556805b37b00 con 0x556805c2b360 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.163+0000 7fca77fc5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556805b8b160 con 0x556805c304d0 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c2fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c2fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44228/0 (socket says 172.21.15.97:44228) 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6de00640 1 -- 172.21.15.97:0/2320381269 learned_addr learned my addr 172.21.15.97:0/2320381269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6d400640 1 --2- 172.21.15.97:0/2320381269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 0x556805c32960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6e800640 1 --2- 172.21.15.97:0/2320381269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c2a970 0x556805c2ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6de00640 1 -- 172.21.15.97:0/2320381269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 msgr2=0x556805c32960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6de00640 1 --2- 172.21.15.97:0/2320381269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 0x556805c32960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6de00640 1 -- 172.21.15.97:0/2320381269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c2a970 msgr2=0x556805c2ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6de00640 1 --2- 172.21.15.97:0/2320381269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c2a970 0x556805c2ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6de00640 1 -- 172.21.15.97:0/2320381269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568058a32d0 con 0x556805c2b360 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6d400640 1 --2- 172.21.15.97:0/2320381269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 0x556805c32960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca66a00640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fca540a2f20 0x7fca540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.164+0000 7fca6de00640 1 --2- 172.21.15.97:0/2320381269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c2fe60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fca600d5a90 tx=0x7fca600d5f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=963f7747686476d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.165+0000 7fca6e800640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fca540a2f20 0x7fca540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.165+0000 7fca66a00640 1 -- 172.21.15.97:0/3305366953 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fca600cf720 con 0x7fca70102350 2024-03-26T18:58:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.165+0000 7fca5b400640 1 -- 172.21.15.97:0/2320381269 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca600980c0 con 0x556805c2b360 2024-03-26T18:58:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.165+0000 7fca5b400640 1 -- 172.21.15.97:0/2320381269 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca6009c520 con 0x556805c2b360 2024-03-26T18:58:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.165+0000 7fca5b400640 1 -- 172.21.15.97:0/2320381269 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6009b780 con 0x556805c2b360 2024-03-26T18:58:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.165+0000 7fca77fc5740 1 -- 172.21.15.97:0/2320381269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 msgr2=0x556805c2fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.165+0000 7fca6e800640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fca540a2f20 0x7fca540a53e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca68006010 tx=0x7fca68004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.165+0000 7fca77fc5740 1 --2- 172.21.15.97:0/2320381269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c2fe60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fca600d5a90 tx=0x7fca600d5f50 comp rx=0 tx=0).stop 2024-03-26T18:58:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.166+0000 7fca77fc5740 1 -- 172.21.15.97:0/2320381269 shutdown_connections 2024-03-26T18:58:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.166+0000 7fca77fc5740 1 --2- 172.21.15.97:0/2320381269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 0x556805c32960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.166+0000 7fca77fc5740 1 --2- 172.21.15.97:0/2320381269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c2fe60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.166+0000 7fca77fc5740 1 --2- 172.21.15.97:0/2320381269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c2a970 0x556805c2ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.166+0000 7fca77fc5740 1 -- 172.21.15.97:0/2320381269 >> 172.21.15.97:0/2320381269 conn(0x556805c25ee0 msgr2=0x556805c262e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.166+0000 7fca77fc5740 1 -- 172.21.15.97:0/2320381269 shutdown_connections 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.166+0000 7fca77fc5740 1 -- 172.21.15.97:0/2320381269 wait complete. 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.166+0000 7fca77fc5740 1 -- start start 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.166+0000 7fca77fc5740 1 Processor -- start 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2a970 0x556805c34ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c304d0 0x556805c3c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca77fc5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556805c33480 con 0x556805c304d0 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca77fc5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556805b4aed0 con 0x556805c2b360 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca77fc5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556805c33600 con 0x556805c2a970 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca77fc5740 10 client.0 _ll_register_callbacks cb 0x556805b86710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca6d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca6de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2a970 0x556805c34ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca6d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c353e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44234/0 (socket says 172.21.15.97:44234) 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca6d400640 1 -- 172.21.15.97:0/4009039351 learned_addr learned my addr 172.21.15.97:0/4009039351 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca6e800640 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c304d0 0x556805c3c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca6d400640 1 -- 172.21.15.97:0/4009039351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2a970 msgr2=0x556805c34ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca6d400640 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2a970 0x556805c34ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca6d400640 1 -- 172.21.15.97:0/4009039351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c304d0 msgr2=0x556805c3c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.167+0000 7fca6d400640 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c304d0 0x556805c3c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.168+0000 7fca6d400640 1 -- 172.21.15.97:0/4009039351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55680596d6b0 con 0x556805c2b360 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.168+0000 7fca6de00640 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2a970 0x556805c34ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.168+0000 7fca6d400640 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c353e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009f60 tx=0x7fca5c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a460dda1180c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.168+0000 7fca4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.168+0000 7fca4fe00640 1 -- 172.21.15.97:0/4009039351 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c01f020 con 0x556805c2b360 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.168+0000 7fca77fc5740 1 -- 172.21.15.97:0/4009039351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca6009c520 con 0x556805c2b360 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.168+0000 7fca4fe00640 1 -- 172.21.15.97:0/4009039351 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fca5c004460 con 0x556805c2b360 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.168+0000 7fca4fe00640 1 -- 172.21.15.97:0/4009039351 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca5c0035f0 con 0x556805c2b360 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.168+0000 7fca77fc5740 10 client.25238 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.168+0000 7fca77fc5740 1 -- 172.21.15.97:0/4009039351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556805c3f470 con 0x556805c2b360 2024-03-26T18:58:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.169+0000 7fca4fe00640 1 -- 172.21.15.97:0/4009039351 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca6009c520 con 0x556805c2b360 2024-03-26T18:58:46.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.169+0000 7fca4fe00640 1 -- 172.21.15.97:0/4009039351 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca5c01b9d0 con 0x556805c2b360 2024-03-26T18:58:46.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.169+0000 7fca77fc5740 10 client.25238 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.169+0000 7fca77fc5740 10 client.25238 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.169+0000 7fca77fc5740 20 client.25238 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.169+0000 7fca77fc5740 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x556805c41b70 0x556805c42fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.169+0000 7fca77fc5740 4 client.25238 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.169+0000 7fca77fc5740 1 -- 172.21.15.97:0/4009039351 --> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556805b9c6f0 con 0x556805c41b70 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.169+0000 7fca6de00640 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x556805c41b70 0x556805c42fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.170+0000 7fca6de00640 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x556805c41b70 0x556805c42fe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c407cd42a863827 server_cookie=18341172a3fb3eb9 in_seq=0 out_seq=0 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.170+0000 7fca4fe00640 10 client.25238 ms_handle_connect on v2:172.21.15.97:6840/4165060179 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca4fe00640 1 -- 172.21.15.97:0/4009039351 <== mds.1 v2:172.21.15.97:6840/4165060179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556805b9c6f0 con 0x556805c41b70 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca4fe00640 10 client.25238 handle_command_reply: tid=0 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 client.25238 shutdown 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 -- 172.21.15.97:0/4009039351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 msgr2=0x556805c353e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c353e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009f60 tx=0x7fca5c009f90 comp rx=0 tx=0).stop 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 -- 172.21.15.97:0/4009039351 shutdown_connections 2024-03-26T18:58:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x556805c41b70 0x556805c42fe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c304d0 0x556805c3c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805c353e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca4fe00640 0 client.25238 ms_handle_reset on v2:172.21.15.97:6840/4165060179 2024-03-26T18:58:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 --2- 172.21.15.97:0/4009039351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2a970 0x556805c34ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 -- 172.21.15.97:0/4009039351 >> 172.21.15.97:0/4009039351 conn(0x556805c25ee0 msgr2=0x556805c25020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 -- 172.21.15.97:0/4009039351 shutdown_connections 2024-03-26T18:58:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 1 -- 172.21.15.97:0/4009039351 wait complete. 2024-03-26T18:58:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.171+0000 7fca77fc5740 20 client.25238 trim_cache size 0 max 16384 2024-03-26T18:58:46.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.205+0000 7fca77fc5740 1 Processor -- start 2024-03-26T18:58:46.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.205+0000 7fca77fc5740 1 -- start start 2024-03-26T18:58:46.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.205+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2a970 0x556805c2be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.205+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2b360 0x556805c3c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.205+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c304d0 0x556805c20c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.205+0000 7fca77fc5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556805c41320 con 0x556805c304d0 2024-03-26T18:58:46.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.205+0000 7fca77fc5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556805c419f0 con 0x556805c2a970 2024-03-26T18:58:46.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.205+0000 7fca77fc5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556805c420c0 con 0x556805c2b360 2024-03-26T18:58:46.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2a970 0x556805c2be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2a970 0x556805c2be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44242/0 (socket says 172.21.15.97:44242) 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6de00640 1 -- 172.21.15.97:0/2562513556 learned_addr learned my addr 172.21.15.97:0/2562513556 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6e800640 1 --2- 172.21.15.97:0/2562513556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c304d0 0x556805c20c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6d400640 1 --2- 172.21.15.97:0/2562513556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2b360 0x556805c3c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6de00640 1 -- 172.21.15.97:0/2562513556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2b360 msgr2=0x556805c3c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6de00640 1 --2- 172.21.15.97:0/2562513556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2b360 0x556805c3c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6de00640 1 -- 172.21.15.97:0/2562513556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c304d0 msgr2=0x556805c20c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6de00640 1 --2- 172.21.15.97:0/2562513556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c304d0 0x556805c20c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6de00640 1 -- 172.21.15.97:0/2562513556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556805c24520 con 0x556805c2a970 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca6de00640 1 --2- 172.21.15.97:0/2562513556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2a970 0x556805c2be00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca6009bb20 tx=0x7fca6009be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ea3c955eb6bc44 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca5aa00640 1 -- 172.21.15.97:0/2562513556 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca600b4070 con 0x556805c2a970 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca5aa00640 1 -- 172.21.15.97:0/2562513556 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca6009c520 con 0x556805c2a970 2024-03-26T18:58:46.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.206+0000 7fca5aa00640 1 -- 172.21.15.97:0/2562513556 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6009b960 con 0x556805c2a970 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.207+0000 7fca77fc5740 1 -- 172.21.15.97:0/2562513556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2a970 msgr2=0x556805c2be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.207+0000 7fca77fc5740 1 --2- 172.21.15.97:0/2562513556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2a970 0x556805c2be00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca6009bb20 tx=0x7fca6009be20 comp rx=0 tx=0).stop 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.207+0000 7fca77fc5740 1 -- 172.21.15.97:0/2562513556 shutdown_connections 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.207+0000 7fca77fc5740 1 --2- 172.21.15.97:0/2562513556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c304d0 0x556805c20c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.207+0000 7fca77fc5740 1 --2- 172.21.15.97:0/2562513556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c2b360 0x556805c3c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.207+0000 7fca77fc5740 1 --2- 172.21.15.97:0/2562513556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2a970 0x556805c2be00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.207+0000 7fca77fc5740 1 -- 172.21.15.97:0/2562513556 >> 172.21.15.97:0/2562513556 conn(0x556805c25ee0 msgr2=0x556805c339d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.208+0000 7fca77fc5740 1 -- 172.21.15.97:0/2562513556 shutdown_connections 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.208+0000 7fca77fc5740 1 -- 172.21.15.97:0/2562513556 wait complete. 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.208+0000 7fca77fc5740 1 -- start start 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.208+0000 7fca77fc5740 1 Processor -- start 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c2a970 0x556805c3a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805cac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca77fc5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 0x556805cae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca77fc5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556805c42e60 con 0x556805c2a970 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca77fc5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556805c42790 con 0x556805c2b360 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca77fc5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556805c43530 con 0x556805c304d0 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca77fc5740 10 client.0 _ll_register_callbacks cb 0x7fca5c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca6d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805cac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca6de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c2a970 0x556805c3a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca6d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805cac0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44254/0 (socket says 172.21.15.97:44254) 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca6d400640 1 -- 172.21.15.97:0/1811841309 learned_addr learned my addr 172.21.15.97:0/1811841309 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca6e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 0x556805cae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca6d400640 1 -- 172.21.15.97:0/1811841309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 msgr2=0x556805cae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.209+0000 7fca6d400640 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 0x556805cae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca6d400640 1 -- 172.21.15.97:0/1811841309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c2a970 msgr2=0x556805c3a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca6d400640 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c2a970 0x556805c3a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca6d400640 1 -- 172.21.15.97:0/1811841309 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556805caeb60 con 0x556805c2b360 2024-03-26T18:58:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca6d400640 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805cac0d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fca5c000c00 tx=0x7fca5c00bbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8c6b9865a64e33a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:58:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca4fe00640 1 -- 172.21.15.97:0/1811841309 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c01f070 con 0x556805c2b360 2024-03-26T18:58:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca4fe00640 1 -- 172.21.15.97:0/1811841309 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fca5c003ba0 con 0x556805c2b360 2024-03-26T18:58:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca77fc5740 1 -- 172.21.15.97:0/1811841309 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca6009c520 con 0x556805c2b360 2024-03-26T18:58:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca4fe00640 1 -- 172.21.15.97:0/1811841309 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca5c010530 con 0x556805c2b360 2024-03-26T18:58:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca77fc5740 10 client.25244 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.210+0000 7fca77fc5740 1 -- 172.21.15.97:0/1811841309 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556805cafe70 con 0x556805c2b360 2024-03-26T18:58:46.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.211+0000 7fca4fe00640 1 -- 172.21.15.97:0/1811841309 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca6009c520 con 0x556805c2b360 2024-03-26T18:58:46.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.211+0000 7fca4fe00640 1 -- 172.21.15.97:0/1811841309 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca5c0264d0 con 0x556805c2b360 2024-03-26T18:58:46.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.211+0000 7fca77fc5740 10 client.25244 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:46.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.211+0000 7fca77fc5740 10 client.25244 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T18:58:46.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.211+0000 7fca77fc5740 20 client.25244 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:46.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.211+0000 7fca77fc5740 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x556805cb4ac0 0x556805cb37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.211+0000 7fca77fc5740 4 client.25244 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:58:46.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.211+0000 7fca77fc5740 1 -- 172.21.15.97:0/1811841309 --> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] -- command(tid 0: {"prefix": "status"}) -- 0x556805b96a40 con 0x556805cb4ac0 2024-03-26T18:58:46.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.211+0000 7fca6de00640 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x556805cb4ac0 0x556805cb37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca6de00640 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x556805cb4ac0 0x556805cb37d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cd6dd73896edf97 server_cookie=dbb6ad177d648127 in_seq=0 out_seq=0 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca4fe00640 10 client.25244 ms_handle_connect on v2:172.21.15.97:6840/4165060179 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca4fe00640 1 -- 172.21.15.97:0/1811841309 <== mds.1 v2:172.21.15.97:6840/4165060179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556805b96a40 con 0x556805cb4ac0 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca4fe00640 10 client.25244 handle_command_reply: tid=0 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca77fc5740 1 client.25244 shutdown 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca77fc5740 1 -- 172.21.15.97:0/1811841309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 msgr2=0x556805cac0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca77fc5740 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805cac0d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fca5c000c00 tx=0x7fca5c00bbe0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca77fc5740 1 -- 172.21.15.97:0/1811841309 shutdown_connections 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca77fc5740 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.97:6840/4165060179,v1:172.21.15.97:6841/4165060179] conn(0x556805cb4ac0 0x556805cb37d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca77fc5740 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556805c304d0 0x556805cae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca77fc5740 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556805c2b360 0x556805cac0d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca77fc5740 1 --2- 172.21.15.97:0/1811841309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556805c2a970 0x556805c3a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca77fc5740 1 -- 172.21.15.97:0/1811841309 >> 172.21.15.97:0/1811841309 conn(0x556805c25ee0 msgr2=0x556805c20c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca4fe00640 0 client.25244 ms_handle_reset on v2:172.21.15.97:6840/4165060179 2024-03-26T18:58:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.212+0000 7fca77fc5740 1 -- 172.21.15.97:0/1811841309 shutdown_connections 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca77fc5740 1 -- 172.21.15.97:0/1811841309 wait complete. 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca77fc5740 20 client.25244 trim_cache size 0 max 16384 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24569, 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 138.376830872, 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-03-26T18:58:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2024-03-26T18:58:46.211 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T18:58:46.211 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 151.67943610099999 2024-03-26T18:58:46.211 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca75200640 1 -- 172.21.15.97:0/3305366953 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fca540a2f20 msgr2=0x7fca540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca75200640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fca540a2f20 0x7fca540a53e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca68006010 tx=0x7fca68004a40 comp rx=0 tx=0).stop 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca75200640 1 -- 172.21.15.97:0/3305366953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca70102350 msgr2=0x7fca70197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca75200640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca70102350 0x7fca70197bd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fca6000e9c0 tx=0x7fca6000ee80 comp rx=0 tx=0).stop 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca75200640 1 -- 172.21.15.97:0/3305366953 shutdown_connections 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca75200640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fca540a2f20 0x7fca540a53e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca75200640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca701075b0 0x7fca7019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca75200640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70104c80 0x7fca70198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca75200640 1 --2- 172.21.15.97:0/3305366953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca70102350 0x7fca70197bd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.213+0000 7fca75200640 1 -- 172.21.15.97:0/3305366953 >> 172.21.15.97:0/3305366953 conn(0x7fca700fbd60 msgr2=0x7fca70102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.214+0000 7fca75200640 1 -- 172.21.15.97:0/3305366953 shutdown_connections 2024-03-26T18:58:46.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.214+0000 7fca75200640 1 -- 172.21.15.97:0/3305366953 wait complete. 2024-03-26T18:58:46.223 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:46 smithi097 ceph-mon[31955]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:58:46.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T18:58:46.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.417+0000 7f563cc00640 1 -- 172.21.15.97:0/1869289041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5638101700 msgr2=0x7f5638101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.417+0000 7f563cc00640 1 --2- 172.21.15.97:0/1869289041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5638101700 0x7f5638101b20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5628009900 tx=0x7f562802bc40 comp rx=0 tx=0).stop 2024-03-26T18:58:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.418+0000 7f563cc00640 1 -- 172.21.15.97:0/1869289041 shutdown_connections 2024-03-26T18:58:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.418+0000 7f563cc00640 1 --2- 172.21.15.97:0/1869289041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56381032b0 0x7f5638109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.418+0000 7f563cc00640 1 --2- 172.21.15.97:0/1869289041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56381028f0 0x7f5638102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.418+0000 7f563cc00640 1 --2- 172.21.15.97:0/1869289041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5638101700 0x7f5638101b20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.418+0000 7f563cc00640 1 -- 172.21.15.97:0/1869289041 >> 172.21.15.97:0/1869289041 conn(0x7f56380fcc70 msgr2=0x7f56380ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.418+0000 7f563cc00640 1 -- 172.21.15.97:0/1869289041 shutdown_connections 2024-03-26T18:58:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.418+0000 7f563cc00640 1 -- 172.21.15.97:0/1869289041 wait complete. 2024-03-26T18:58:46.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f563cc00640 1 Processor -- start 2024-03-26T18:58:46.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f563cc00640 1 -- start start 2024-03-26T18:58:46.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f563cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5638101700 0x7f56381979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f563cc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56381028f0 0x7f5638197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f563cc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56381032b0 0x7f563819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f563cc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f563810d3a0 con 0x7f5638101700 2024-03-26T18:58:46.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f563cc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f563810d220 con 0x7f56381032b0 2024-03-26T18:58:46.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f563cc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f563810d520 con 0x7f56381028f0 2024-03-26T18:58:46.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f5636a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56381028f0 0x7f5638197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f5637e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56381032b0 0x7f563819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f5637400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5638101700 0x7f56381979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f5637e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56381032b0 0x7f563819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44272/0 (socket says 172.21.15.97:44272) 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f5636a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56381028f0 0x7f5638197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51494/0 (socket says 172.21.15.97:51494) 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.419+0000 7f5637e00640 1 -- 172.21.15.97:0/3412864843 learned_addr learned my addr 172.21.15.97:0/3412864843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f5637e00640 1 -- 172.21.15.97:0/3412864843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56381028f0 msgr2=0x7f5638197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f5637e00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56381028f0 0x7f5638197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f5637e00640 1 -- 172.21.15.97:0/3412864843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5638101700 msgr2=0x7f56381979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f5637e00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5638101700 0x7f56381979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f5637e00640 1 -- 172.21.15.97:0/3412864843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563819f600 con 0x7f56381032b0 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f5637e00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56381032b0 0x7f563819ef90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f563000efd0 tx=0x7f563000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f562fe00640 1 -- 172.21.15.97:0/3412864843 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630003b70 con 0x7f56381032b0 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f562fe00640 1 -- 172.21.15.97:0/3412864843 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5630003d10 con 0x7f56381032b0 2024-03-26T18:58:46.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f563cc00640 1 -- 172.21.15.97:0/3412864843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563819f8f0 con 0x7f56381032b0 2024-03-26T18:58:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f562fe00640 1 -- 172.21.15.97:0/3412864843 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300109f0 con 0x7f56381032b0 2024-03-26T18:58:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.420+0000 7f563cc00640 1 -- 172.21.15.97:0/3412864843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563819fe40 con 0x7f56381032b0 2024-03-26T18:58:46.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.423+0000 7f562fe00640 1 -- 172.21.15.97:0/3412864843 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f5630004020 con 0x7f56381032b0 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.424+0000 7f5640f77740 1 Processor -- start 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.424+0000 7f5640f77740 1 -- start start 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.424+0000 7f562fe00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f561c0a2fa0 0x7f561c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5640f77740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb146f80 0x5651fb1473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5640f77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 0x5651fb14c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f562fe00640 1 -- 172.21.15.97:0/3412864843 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f56300cf7e0 con 0x7f56381032b0 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5640f77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 0x5651fb14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5640f77740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651fb0a71b0 con 0x5651fb146f80 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5640f77740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5651fb04fb00 con 0x5651fb147970 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5640f77740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651fb0a3160 con 0x5651fb14cae0 2024-03-26T18:58:46.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5636a00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f561c0a2fa0 0x7f561c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5637e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 0x5651fb14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5636a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 0x5651fb14c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5637e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 0x5651fb14ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51506/0 (socket says 172.21.15.97:51506) 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5637e00640 1 -- 172.21.15.97:0/3283515418 learned_addr learned my addr 172.21.15.97:0/3283515418 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.425+0000 7f5637400640 1 --2- 172.21.15.97:0/3283515418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb146f80 0x5651fb1473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5637e00640 1 -- 172.21.15.97:0/3283515418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 msgr2=0x5651fb14c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5637e00640 1 --2- 172.21.15.97:0/3283515418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 0x5651fb14c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5637e00640 1 -- 172.21.15.97:0/3283515418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb146f80 msgr2=0x5651fb1473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5637e00640 1 --2- 172.21.15.97:0/3283515418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb146f80 0x5651fb1473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5637e00640 1 -- 172.21.15.97:0/3283515418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fadbb2d0 con 0x5651fb14cae0 2024-03-26T18:58:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5637400640 1 --2- 172.21.15.97:0/3283515418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb146f80 0x5651fb1473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5636a00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f561c0a2fa0 0x7f561c0a5460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5638198ef0 tx=0x7f5624009b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5637e00640 1 --2- 172.21.15.97:0/3283515418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 0x5651fb14ef70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f563009be70 tx=0x7f56300104c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52bcb743fc56211f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5623e00640 1 -- 172.21.15.97:0/3283515418 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563009bee0 con 0x5651fb14cae0 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5623e00640 1 -- 172.21.15.97:0/3283515418 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56300a0030 con 0x5651fb14cae0 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.426+0000 7f5623e00640 1 -- 172.21.15.97:0/3283515418 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630004310 con 0x5651fb14cae0 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.427+0000 7f5640f77740 1 -- 172.21.15.97:0/3283515418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 msgr2=0x5651fb14ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.427+0000 7f5640f77740 1 --2- 172.21.15.97:0/3283515418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 0x5651fb14ef70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f563009be70 tx=0x7f56300104c0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.427+0000 7f5640f77740 1 -- 172.21.15.97:0/3283515418 shutdown_connections 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.427+0000 7f5640f77740 1 --2- 172.21.15.97:0/3283515418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 0x5651fb14ef70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.427+0000 7f5640f77740 1 --2- 172.21.15.97:0/3283515418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 0x5651fb14c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.427+0000 7f5640f77740 1 --2- 172.21.15.97:0/3283515418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb146f80 0x5651fb1473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.427+0000 7f5640f77740 1 -- 172.21.15.97:0/3283515418 >> 172.21.15.97:0/3283515418 conn(0x5651fb13e0e0 msgr2=0x5651fb13e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.427+0000 7f5640f77740 1 -- 172.21.15.97:0/3283515418 shutdown_connections 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.427+0000 7f5640f77740 1 -- 172.21.15.97:0/3283515418 wait complete. 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.427+0000 7f5640f77740 1 -- start start 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5640f77740 1 Processor -- start 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5640f77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 0x5651fb14a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5640f77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb147970 0x5651fb14a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5640f77740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 0x5651fb14b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5640f77740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651fb142a50 con 0x5651fb14cae0 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5640f77740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5651fb062ed0 con 0x5651fb146f80 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5640f77740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651fb142bd0 con 0x5651fb147970 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5640f77740 10 client.0 _ll_register_callbacks cb 0x5651fb09e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5636a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb147970 0x5651fb14a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5637400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 0x5651fb14a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5636a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb147970 0x5651fb14a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51508/0 (socket says 172.21.15.97:51508) 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.428+0000 7f5636a00640 1 -- 172.21.15.97:0/2326655173 learned_addr learned my addr 172.21.15.97:0/2326655173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.429+0000 7f5637e00640 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 0x5651fb14b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.429+0000 7f5636a00640 1 -- 172.21.15.97:0/2326655173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 msgr2=0x5651fb14a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.429+0000 7f5636a00640 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 0x5651fb14a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.429+0000 7f5636a00640 1 -- 172.21.15.97:0/2326655173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 msgr2=0x5651fb14b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.429+0000 7f5636a00640 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 0x5651fb14b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.429+0000 7f5636a00640 1 -- 172.21.15.97:0/2326655173 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651fae856b0 con 0x5651fb147970 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.429+0000 7f5636a00640 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb147970 0x5651fb14a720 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5624011420 tx=0x7f562401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9146d0273b29542 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.429+0000 7f5620c00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.429+0000 7f5620c00640 1 -- 172.21.15.97:0/2326655173 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624003800 con 0x5651fb147970 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.429+0000 7f5640f77740 1 -- 172.21.15.97:0/2326655173 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56300a0030 con 0x5651fb147970 2024-03-26T18:58:46.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.430+0000 7f5620c00640 1 -- 172.21.15.97:0/2326655173 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f56240039a0 con 0x5651fb147970 2024-03-26T18:58:46.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.430+0000 7f5620c00640 1 -- 172.21.15.97:0/2326655173 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5624003ba0 con 0x5651fb147970 2024-03-26T18:58:46.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.430+0000 7f5640f77740 10 client.25228 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:46.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.430+0000 7f5640f77740 1 -- 172.21.15.97:0/2326655173 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651fb157650 con 0x5651fb147970 2024-03-26T18:58:46.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.430+0000 7f5620c00640 1 -- 172.21.15.97:0/2326655173 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56300a0030 con 0x5651fb147970 2024-03-26T18:58:46.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.430+0000 7f5620c00640 1 -- 172.21.15.97:0/2326655173 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5624010430 con 0x5651fb147970 2024-03-26T18:58:46.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.430+0000 7f5640f77740 10 client.25228 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:46.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.430+0000 7f5640f77740 10 client.25228 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.430+0000 7f5640f77740 20 client.25228 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.430+0000 7f5640f77740 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5651fb159ef0 0x5651fb15b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.431+0000 7f5640f77740 4 client.25228 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.431+0000 7f5640f77740 1 -- 172.21.15.97:0/2326655173 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651fb0b46f0 con 0x5651fb159ef0 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.431+0000 7f5637400640 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5651fb159ef0 0x5651fb15b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.431+0000 7f5637400640 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5651fb159ef0 0x5651fb15b360 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb80580b602a2358 server_cookie=f4c4ab1b646c55a8 in_seq=0 out_seq=0 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.431+0000 7f5620c00640 10 client.25228 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.432+0000 7f5620c00640 1 -- 172.21.15.97:0/2326655173 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5651fb0b46f0 con 0x5651fb159ef0 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.432+0000 7f5620c00640 10 client.25228 handle_command_reply: tid=0 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.432+0000 7f5640f77740 1 client.25228 shutdown 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 1 -- 172.21.15.97:0/2326655173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb147970 msgr2=0x5651fb14a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb147970 0x5651fb14a720 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5624011420 tx=0x7f562401d000 comp rx=0 tx=0).stop 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 1 -- 172.21.15.97:0/2326655173 shutdown_connections 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5651fb159ef0 0x5651fb15b360 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 0x5651fb14b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb147970 0x5651fb14a720 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 1 --2- 172.21.15.97:0/2326655173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 0x5651fb14a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 1 -- 172.21.15.97:0/2326655173 >> 172.21.15.97:0/2326655173 conn(0x5651fb13e0e0 msgr2=0x5651fb13d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5620c00640 0 client.25228 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:58:46.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 1 -- 172.21.15.97:0/2326655173 shutdown_connections 2024-03-26T18:58:46.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 1 -- 172.21.15.97:0/2326655173 wait complete. 2024-03-26T18:58:46.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.433+0000 7f5640f77740 20 client.25228 trim_cache size 0 max 16384 2024-03-26T18:58:46.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.454+0000 7f5640f77740 1 Processor -- start 2024-03-26T18:58:46.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.454+0000 7f5640f77740 1 -- start start 2024-03-26T18:58:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5640f77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb146f80 0x5651fb158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5640f77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 0x5651fb159000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5640f77740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 0x5651fb148860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5640f77740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651fb1596a0 con 0x5651fb14cae0 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5640f77740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5651fb159d70 con 0x5651fb147970 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5640f77740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651fb15a440 con 0x5651fb146f80 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5637400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb146f80 0x5651fb158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5636a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 0x5651fb159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5637400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb146f80 0x5651fb158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51522/0 (socket says 172.21.15.97:51522) 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5637400640 1 -- 172.21.15.97:0/3358969173 learned_addr learned my addr 172.21.15.97:0/3358969173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5637e00640 1 --2- 172.21.15.97:0/3358969173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 0x5651fb148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5637400640 1 -- 172.21.15.97:0/3358969173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 msgr2=0x5651fb159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5637400640 1 --2- 172.21.15.97:0/3358969173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 0x5651fb159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5637400640 1 -- 172.21.15.97:0/3358969173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 msgr2=0x5651fb148860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5637400640 1 --2- 172.21.15.97:0/3358969173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 0x5651fb148860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5637400640 1 -- 172.21.15.97:0/3358969173 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fb142930 con 0x5651fb146f80 2024-03-26T18:58:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5637e00640 1 --2- 172.21.15.97:0/3358969173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 0x5651fb148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.455+0000 7f5636a00640 1 --2- 172.21.15.97:0/3358969173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 0x5651fb159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:58:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5637400640 1 --2- 172.21.15.97:0/3358969173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb146f80 0x5651fb158ac0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5628005ab0 tx=0x7f562802e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72ad1cdbbbd0a978 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5623400640 1 -- 172.21.15.97:0/3358969173 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562802cc30 con 0x5651fb146f80 2024-03-26T18:58:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5623400640 1 -- 172.21.15.97:0/3358969173 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5628004040 con 0x5651fb146f80 2024-03-26T18:58:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5623400640 1 -- 172.21.15.97:0/3358969173 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628048780 con 0x5651fb146f80 2024-03-26T18:58:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5640f77740 1 -- 172.21.15.97:0/3358969173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb146f80 msgr2=0x5651fb158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5640f77740 1 --2- 172.21.15.97:0/3358969173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb146f80 0x5651fb158ac0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5628005ab0 tx=0x7f562802e040 comp rx=0 tx=0).stop 2024-03-26T18:58:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5640f77740 1 -- 172.21.15.97:0/3358969173 shutdown_connections 2024-03-26T18:58:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5640f77740 1 --2- 172.21.15.97:0/3358969173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb14cae0 0x5651fb148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5640f77740 1 --2- 172.21.15.97:0/3358969173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb147970 0x5651fb159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5640f77740 1 --2- 172.21.15.97:0/3358969173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb146f80 0x5651fb158ac0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.456+0000 7f5640f77740 1 -- 172.21.15.97:0/3358969173 >> 172.21.15.97:0/3358969173 conn(0x5651fb13e0e0 msgr2=0x5651fb14a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.457+0000 7f5640f77740 1 -- 172.21.15.97:0/3358969173 shutdown_connections 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.457+0000 7f5640f77740 1 -- 172.21.15.97:0/3358969173 wait complete. 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.457+0000 7f5640f77740 1 -- start start 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.457+0000 7f5640f77740 1 Processor -- start 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5640f77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 0x5651fb154b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5640f77740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb147970 0x5651fb155070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5640f77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 0x5651fb151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5640f77740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651fb15b1e0 con 0x5651fb147970 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5640f77740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5651fb15ab10 con 0x5651fb146f80 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5640f77740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651fb15b8b0 con 0x5651fb14cae0 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5640f77740 10 client.0 _ll_register_callbacks cb 0x7f5624009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5637400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 0x5651fb154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5637400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 0x5651fb154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44310/0 (socket says 172.21.15.97:44310) 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5637e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 0x5651fb151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5637400640 1 -- 172.21.15.97:0/1972704367 learned_addr learned my addr 172.21.15.97:0/1972704367 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5636a00640 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb147970 0x5651fb155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5637400640 1 -- 172.21.15.97:0/1972704367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 msgr2=0x5651fb151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5637400640 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 0x5651fb151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5637400640 1 -- 172.21.15.97:0/1972704367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb147970 msgr2=0x5651fb155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.458+0000 7f5637400640 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb147970 0x5651fb155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.459+0000 7f5637400640 1 -- 172.21.15.97:0/1972704367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651fb1520f0 con 0x5651fb146f80 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.459+0000 7f5636a00640 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb147970 0x5651fb155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.459+0000 7f5637400640 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 0x5651fb154b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5628036040 tx=0x7f562802c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2709413106989c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.459+0000 7f5620c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.459+0000 7f5620c00640 1 -- 172.21.15.97:0/1972704367 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628005040 con 0x5651fb146f80 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.459+0000 7f5620c00640 1 -- 172.21.15.97:0/1972704367 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f56280051e0 con 0x5651fb146f80 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.459+0000 7f5640f77740 1 -- 172.21.15.97:0/1972704367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5628004040 con 0x5651fb146f80 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.459+0000 7f5620c00640 1 -- 172.21.15.97:0/1972704367 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f56280053e0 con 0x5651fb146f80 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.459+0000 7f5640f77740 10 client.25262 fetch_fsmap learned FSMap version 26 2024-03-26T18:58:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.459+0000 7f5640f77740 1 -- 172.21.15.97:0/1972704367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651fb1c07e0 con 0x5651fb146f80 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.460+0000 7f5620c00640 1 -- 172.21.15.97:0/1972704367 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5628004040 con 0x5651fb146f80 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.460+0000 7f5620c00640 1 -- 172.21.15.97:0/1972704367 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f56280036f0 con 0x5651fb146f80 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.460+0000 7f5640f77740 10 client.25262 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.460+0000 7f5640f77740 10 client.25262 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.460+0000 7f5640f77740 20 client.25262 populate_metadata read hostname 'smithi097' 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.460+0000 7f5640f77740 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5651fb1c4f80 0x5651fb1c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.460+0000 7f5640f77740 4 client.25262 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.460+0000 7f5640f77740 1 -- 172.21.15.97:0/1972704367 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "status"}) -- 0x5651fb0aea40 con 0x5651fb1c4f80 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.460+0000 7f5636a00640 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5651fb1c4f80 0x5651fb1c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5636a00640 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5651fb1c4f80 0x5651fb1c3c90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697a1737772dd0fb server_cookie=29ec0ade67cbae1e in_seq=0 out_seq=0 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5620c00640 10 client.25262 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5620c00640 1 -- 172.21.15.97:0/1972704367 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5651fb0aea40 con 0x5651fb1c4f80 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5620c00640 10 client.25262 handle_command_reply: tid=0 2024-03-26T18:58:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 client.25262 shutdown 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 -- 172.21.15.97:0/1972704367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 msgr2=0x5651fb154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 0x5651fb154b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5628036040 tx=0x7f562802c890 comp rx=0 tx=0).stop 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 -- 172.21.15.97:0/1972704367 shutdown_connections 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5651fb1c4f80 0x5651fb1c3c90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651fb14cae0 0x5651fb151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651fb147970 0x5651fb155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 --2- 172.21.15.97:0/1972704367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651fb146f80 0x5651fb154b30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 -- 172.21.15.97:0/1972704367 >> 172.21.15.97:0/1972704367 conn(0x5651fb13e0e0 msgr2=0x5651fb158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5620c00640 0 client.25262 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 -- 172.21.15.97:0/1972704367 shutdown_connections 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 1 -- 172.21.15.97:0/1972704367 wait complete. 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.461+0000 7f5640f77740 20 client.25262 trim_cache size 0 max 16384 2024-03-26T18:58:46.459 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "2172d50a-eba1-11ee-95d0-87774f69a715", 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24581, 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 141.920112972, 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 149.09237423499999 2024-03-26T18:58:46.460 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T18:58:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 -- 172.21.15.97:0/3412864843 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f561c0a2fa0 msgr2=0x7f561c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f561c0a2fa0 0x7f561c0a5460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5638198ef0 tx=0x7f5624009b60 comp rx=0 tx=0).stop 2024-03-26T18:58:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 -- 172.21.15.97:0/3412864843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56381032b0 msgr2=0x7f563819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56381032b0 0x7f563819ef90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f563000efd0 tx=0x7f563000c6a0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 -- 172.21.15.97:0/3412864843 shutdown_connections 2024-03-26T18:58:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f561c0a2fa0 0x7f561c0a5460 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56381032b0 0x7f563819ef90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56381028f0 0x7f5638197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 --2- 172.21.15.97:0/3412864843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5638101700 0x7f56381979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 -- 172.21.15.97:0/3412864843 >> 172.21.15.97:0/3412864843 conn(0x7f56380fcc70 msgr2=0x7f5638105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 -- 172.21.15.97:0/3412864843 shutdown_connections 2024-03-26T18:58:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:46.462+0000 7f563cc00640 1 -- 172.21.15.97:0/3412864843 wait complete. 2024-03-26T18:58:46.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:46 smithi105 ceph-mon[36270]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:58:46.473 INFO:teuthology.run_tasks:Running task sequential... 2024-03-26T18:58:46.484 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-26T18:58:46.484 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-26T18:58:46.485 INFO:teuthology.task.print:Enabling mgr modules 2024-03-26T18:58:46.485 INFO:teuthology.run_tasks:Running task kclient... 2024-03-26T18:58:46.506 INFO:tasks.kclient:Mounting kernel clients... 2024-03-26T18:58:46.506 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-26T18:58:46.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:58:46.507 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-26T18:58:46.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:58:46.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-03-26T18:58:46.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:46 smithi144 ceph-mon[36053]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-26T18:58:46.637 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-03-26T18:58:46.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:58:46.641 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-03-26T18:58:46.641 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-26T18:58:46.641 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-03-26T18:58:46.641 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-03-26T18:58:46.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-26T18:58:47.074 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:58:48.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:48 smithi097 ceph-mon[31955]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:48 smithi105 ceph-mon[36270]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:48.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:48 smithi144 ceph-mon[36053]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:58:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.450+0000 7fa4baa00700 1 -- 172.21.15.97:0/730535217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4b410a650 msgr2=0x7fa4b410aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.450+0000 7fa4baa00700 1 --2- 172.21.15.97:0/730535217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4b410a650 0x7fa4b410aa30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009990 tx=0x7fa4a4009ca0 comp rx=0 tx=0).stop 2024-03-26T18:58:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.450+0000 7fa4baa00700 1 -- 172.21.15.97:0/730535217 shutdown_connections 2024-03-26T18:58:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.450+0000 7fa4baa00700 1 --2- 172.21.15.97:0/730535217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4b4104f80 0x7fa4b4111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.450+0000 7fa4baa00700 1 --2- 172.21.15.97:0/730535217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4b410a650 0x7fa4b410aa30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.450+0000 7fa4baa00700 1 --2- 172.21.15.97:0/730535217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4b41045c0 0x7fa4b4104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.450+0000 7fa4baa00700 1 -- 172.21.15.97:0/730535217 >> 172.21.15.97:0/730535217 conn(0x7fa4b41000b0 msgr2=0x7fa4b41024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.450+0000 7fa4baa00700 1 -- 172.21.15.97:0/730535217 shutdown_connections 2024-03-26T18:58:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.450+0000 7fa4baa00700 1 -- 172.21.15.97:0/730535217 wait complete. 2024-03-26T18:58:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.451+0000 7fa4baa00700 1 Processor -- start 2024-03-26T18:58:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4baa00700 1 -- start start 2024-03-26T18:58:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4baa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4b41045c0 0x7fa4b40768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4baa00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4b4104f80 0x7fa4b4076e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4baa00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4b410a650 0x7fa4b4077340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4baa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4114940 con 0x7fa4b41045c0 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4baa00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41147c0 con 0x7fa4b4104f80 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4baa00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4114ac0 con 0x7fa4b410a650 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4b3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4b41045c0 0x7fa4b40768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4b8a00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4b410a650 0x7fa4b4077340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4b3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4b41045c0 0x7fa4b40768c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37088/0 (socket says 172.21.15.97:37088) 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4b3e00700 1 -- 172.21.15.97:0/2562451210 learned_addr learned my addr 172.21.15.97:0/2562451210 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.452+0000 7fa4b3400700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4b4104f80 0x7fa4b4076e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.453+0000 7fa4b3400700 1 -- 172.21.15.97:0/2562451210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4b410a650 msgr2=0x7fa4b4077340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.453+0000 7fa4b3400700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4b410a650 0x7fa4b4077340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.453+0000 7fa4b3400700 1 -- 172.21.15.97:0/2562451210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4b41045c0 msgr2=0x7fa4b40768c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.453+0000 7fa4b3400700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4b41045c0 0x7fa4b40768c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.453+0000 7fa4b3400700 1 -- 172.21.15.97:0/2562451210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b41ab3c0 con 0x7fa4b4104f80 2024-03-26T18:58:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.453+0000 7fa4b3400700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4b4104f80 0x7fa4b4076e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000ea20 tx=0x7fa4a000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.453+0000 7fa4b0c00700 1 -- 172.21.15.97:0/2562451210 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000cbd0 con 0x7fa4b4104f80 2024-03-26T18:58:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.453+0000 7fa4baa00700 1 -- 172.21.15.97:0/2562451210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b41ab6b0 con 0x7fa4b4104f80 2024-03-26T18:58:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.453+0000 7fa4b0c00700 1 -- 172.21.15.97:0/2562451210 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a000ee60 con 0x7fa4b4104f80 2024-03-26T18:58:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.454+0000 7fa4b0c00700 1 -- 172.21.15.97:0/2562451210 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0018680 con 0x7fa4b4104f80 2024-03-26T18:58:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.454+0000 7fa4baa00700 1 -- 172.21.15.97:0/2562451210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b41abc40 con 0x7fa4b4104f80 2024-03-26T18:58:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.455+0000 7fa4baa00700 1 -- 172.21.15.97:0/2562451210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b410cb50 con 0x7fa4b4104f80 2024-03-26T18:58:49.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.458+0000 7fa4b0c00700 1 -- 172.21.15.97:0/2562451210 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fa4a0018820 con 0x7fa4b4104f80 2024-03-26T18:58:49.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.460+0000 7fa4b0c00700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa4980a2fb0 0x7fa4980a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:49.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.460+0000 7fa4b0c00700 1 -- 172.21.15.97:0/2562451210 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa4a00cf530 con 0x7fa4b4104f80 2024-03-26T18:58:49.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.460+0000 7fa4b3e00700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa4980a2fb0 0x7fa4980a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:49.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.461+0000 7fa4b3e00700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa4980a2fb0 0x7fa4980a5470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009750 tx=0x7fa4a400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:49.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.467+0000 7fa4b0c00700 1 -- 172.21.15.97:0/2562451210 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa4a009c330 con 0x7fa4b4104f80 2024-03-26T18:58:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.803+0000 7fa4baa00700 1 -- 172.21.15.97:0/2562451210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa4b4077df0 con 0x7fa4b4104f80 2024-03-26T18:58:49.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.804+0000 7fa4b0c00700 1 -- 172.21.15.97:0/2562451210 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fa4a009bc30 con 0x7fa4b4104f80 2024-03-26T18:58:49.998 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 -- 172.21.15.97:0/2562451210 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa4980a2fb0 msgr2=0x7fa4980a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa4980a2fb0 0x7fa4980a5470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009750 tx=0x7fa4a400b280 comp rx=0 tx=0).stop 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 -- 172.21.15.97:0/2562451210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4b4104f80 msgr2=0x7fa4b4076e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4b4104f80 0x7fa4b4076e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000ea20 tx=0x7fa4a000ed30 comp rx=0 tx=0).stop 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 -- 172.21.15.97:0/2562451210 shutdown_connections 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa4980a2fb0 0x7fa4980a5470 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4b410a650 0x7fa4b4077340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4b41045c0 0x7fa4b40768c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 --2- 172.21.15.97:0/2562451210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4b4104f80 0x7fa4b4076e00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 -- 172.21.15.97:0/2562451210 >> 172.21.15.97:0/2562451210 conn(0x7fa4b41000b0 msgr2=0x7fa4b4101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 -- 172.21.15.97:0/2562451210 shutdown_connections 2024-03-26T18:58:49.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:49.807+0000 7fa4baa00700 1 -- 172.21.15.97:0/2562451210 wait complete. 2024-03-26T18:58:50.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:50 smithi105 ceph-mon[36270]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:50.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:50 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2562451210' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T18:58:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:50 smithi097 ceph-mon[31955]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:50 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2562451210' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T18:58:50.606 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-26T18:58:50.606 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-26T18:58:50.606 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-03-26T18:58:50.606 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-26T18:58:50.606 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-03-26T18:58:50.606 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-26T18:58:50.606 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-26T18:58:50.606 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-26T18:58:50.606 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-03-26T18:58:50.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:58:50.607 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-03-26T18:58:50.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:50 smithi144 ceph-mon[36053]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:58:50.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:50 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2562451210' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T18:58:50.654 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-26T18:58:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-26T18:58:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-03-26T18:58:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-26T18:58:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-03-26T18:58:50.654 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-26T18:58:50.654 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41911sec preferred_lft 41911sec 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-03-26T18:58:50.655 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-26T18:58:50.656 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-26T18:58:50.657 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T18:58:50.657 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-26T18:58:50.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-03-26T18:58:50.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-03-26T18:58:50.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-03-26T18:58:50.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-26T18:58:50.657 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-26T18:58:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:58:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:58:51.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:58:51.202 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-26T18:58:51.283 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-03-26T18:58:51.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:58:51.289 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-03-26T18:58:51.349 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-03-26T18:58:51.349 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-26T18:58:51.350 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-26T18:58:51.350 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-26T18:58:51.350 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-26T18:58:51.351 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T18:58:51.351 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-26T18:58:51.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-26T18:58:51.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-26T18:58:51.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-26T18:58:51.351 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-26T18:58:51.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:58:52.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:58:52.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:58:52.117 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-26T18:58:52.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:58:52.169 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-03-26T18:58:52.215 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T18:58:52.215 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-26T18:58:52.215 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--tmp-cephfs 2024-03-26T18:58:52.216 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-03-26T18:58:52.216 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-26T18:58:52.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:52 smithi097 ceph-mon[31955]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:52.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:58:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:58:52.453 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-03-26T18:58:52.453 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T18:58:52.453 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-26T18:58:52.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-03-26T18:58:52.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-26T18:58:52.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-03-26T18:58:52.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-03-26T18:58:52.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-03-26T18:58:52.453 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-26T18:58:52.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:52 smithi105 ceph-mon[36270]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:52.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:58:52.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:52 smithi144 ceph-mon[36053]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:58:52.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:58:52.819 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T18:58:52.819 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-26T18:58:52.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-03-26T18:58:52.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-03-26T18:58:52.819 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-26T18:58:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:58:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:58:53.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T18:58:53.041 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /tmp/cephfs 2024-03-26T18:58:53.081 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/tmp/cephfs' 2024-03-26T18:58:53.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T18:58:53.082 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /tmp/cephfs 2024-03-26T18:58:53.170 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-26T18:58:53.170 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-26T18:58:53.170 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/7cc1b6af-7058-492a-908c-827e2e8de910 2024-03-26T18:58:53.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:58:53.170 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/7cc1b6af-7058-492a-908c-827e2e8de910 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-03-26T18:58:53.342 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-03-26T18:58:53.342 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-03-26T18:58:53.343 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-03-26T18:58:53.968 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-03-26T18:58:53.968 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-03-26T18:58:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-03-26T18:58:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.105:6789,172.21.15.144:6789" 2024-03-26T18:58:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.105:6789,172.21.15.144:6789:/volumes/qa/sv_1/7cc1b6af-7058-492a-908c-827e2e8de910 2024-03-26T18:58:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=2172d50a-eba1-11ee-95d0-87774f69a715" will pass to kernel 2024-03-26T18:58:53.978 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-26T18:58:53.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:58:53.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /tmp/cephfs 2024-03-26T18:58:54.062 DEBUG:teuthology.orchestra.run.smithi097:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T18:58:54.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-26T18:58:54.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:58:54.189 INFO:teuthology.orchestra.run.smithi097.stdout:client25243 2024-03-26T18:58:54.190 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T18:58:54.190 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T18:58:54.233 DEBUG:teuthology.orchestra.run.smithi097:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T18:58:54.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:54 smithi097 ceph-mon[31955]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:54.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-26T18:58:54.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:58:54.406 INFO:teuthology.orchestra.run.smithi097.stdout:client25243 2024-03-26T18:58:54.406 DEBUG:teuthology.orchestra.run.smithi097:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T18:58:54.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:54 smithi105 ceph-mon[36270]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:54.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-26T18:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:58:54.532 INFO:teuthology.orchestra.run.smithi097.stdout:client25243 2024-03-26T18:58:54.533 DEBUG:teuthology.orchestra.run.smithi097:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2172d50a-eba1-11ee-95d0-87774f69a715.client25243/status') 2024-03-26T18:58:54.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:54 smithi144 ceph-mon[36053]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T18:58:54.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:58:54.672 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.25243 (3)172.21.15.97:0/791004037 2024-03-26T18:58:54.672 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2024-03-26T18:58:54.672 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2024-03-26T18:58:54.673 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2024-03-26T18:58:54.673 INFO:teuthology.orchestra.run.smithi097.stderr:70 bytes copied, 6.5198e-05 s, 1.1 MB/s 2024-03-26T18:58:54.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:58:54.686 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T18:58:54.697 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T18:58:54.697 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-03-26T18:58:54.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-26T18:58:54.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.912+0000 7f9ed9a00640 1 -- 172.21.15.97:0/3286394809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed41034b0 msgr2=0x7f9ed4109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:54.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.912+0000 7f9ed9a00640 1 --2- 172.21.15.97:0/3286394809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed41034b0 0x7f9ed4109d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009ca0 tx=0x7f9ecc02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:58:54.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.912+0000 7f9ed9a00640 1 -- 172.21.15.97:0/3286394809 shutdown_connections 2024-03-26T18:58:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.912+0000 7f9ed9a00640 1 --2- 172.21.15.97:0/3286394809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed41034b0 0x7f9ed4109d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.912+0000 7f9ed9a00640 1 --2- 172.21.15.97:0/3286394809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed4102af0 0x7f9ed4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.912+0000 7f9ed9a00640 1 --2- 172.21.15.97:0/3286394809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ed4101900 0x7f9ed4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.912+0000 7f9ed9a00640 1 -- 172.21.15.97:0/3286394809 >> 172.21.15.97:0/3286394809 conn(0x7f9ed40fce50 msgr2=0x7f9ed40ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.912+0000 7f9ed9a00640 1 -- 172.21.15.97:0/3286394809 shutdown_connections 2024-03-26T18:58:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.912+0000 7f9ed9a00640 1 -- 172.21.15.97:0/3286394809 wait complete. 2024-03-26T18:58:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.913+0000 7f9ed9a00640 1 Processor -- start 2024-03-26T18:58:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed9a00640 1 -- start start 2024-03-26T18:58:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed9a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ed4101900 0x7f9ed4199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed9a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed4102af0 0x7f9ed419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed41034b0 0x7f9ed41a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed9a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed410d560 con 0x7f9ed41034b0 2024-03-26T18:58:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed9a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed410d3e0 con 0x7f9ed4102af0 2024-03-26T18:58:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed9a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed410d6e0 con 0x7f9ed4101900 2024-03-26T18:58:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ed4101900 0x7f9ed4199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9eca000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed4102af0 0x7f9ed419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ed4101900 0x7f9ed4199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:46054/0 (socket says 172.21.15.97:46054) 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed2a00640 1 -- 172.21.15.97:0/960496038 learned_addr learned my addr 172.21.15.97:0/960496038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9eca000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed4102af0 0x7f9ed419a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60798/0 (socket says 172.21.15.97:60798) 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.914+0000 7f9ed3400640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed41034b0 0x7f9ed41a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ed2a00640 1 -- 172.21.15.97:0/960496038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed4102af0 msgr2=0x7f9ed419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ed2a00640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed4102af0 0x7f9ed419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ed2a00640 1 -- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed41034b0 msgr2=0x7f9ed41a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ed2a00640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed41034b0 0x7f9ed41a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ed2a00640 1 -- 172.21.15.97:0/960496038 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed41a1950 con 0x7f9ed4101900 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ed3400640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed41034b0 0x7f9ed41a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ed2a00640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ed4101900 0x7f9ed4199d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400e9c0 tx=0x7f9ec400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ecbe00640 1 -- 172.21.15.97:0/960496038 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400cde0 con 0x7f9ed4101900 2024-03-26T18:58:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ecbe00640 1 -- 172.21.15.97:0/960496038 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec4010910 con 0x7f9ed4101900 2024-03-26T18:58:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ed9a00640 1 -- 172.21.15.97:0/960496038 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed41a1c40 con 0x7f9ed4101900 2024-03-26T18:58:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ecbe00640 1 -- 172.21.15.97:0/960496038 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4018a00 con 0x7f9ed4101900 2024-03-26T18:58:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.915+0000 7f9ed9a00640 1 -- 172.21.15.97:0/960496038 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed41a2190 con 0x7f9ed4101900 2024-03-26T18:58:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.916+0000 7f9ed9a00640 1 -- 172.21.15.97:0/960496038 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed41a2360 con 0x7f9ed4101900 2024-03-26T18:58:54.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.920+0000 7f9ecbe00640 1 -- 172.21.15.97:0/960496038 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f9ec401f020 con 0x7f9ed4101900 2024-03-26T18:58:54.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.921+0000 7f9ecbe00640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9eb80a2fc0 0x7f9eb80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:54.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.921+0000 7f9ecbe00640 1 -- 172.21.15.97:0/960496038 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9ec40cf730 con 0x7f9ed4101900 2024-03-26T18:58:54.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.922+0000 7f9eca000640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9eb80a2fc0 0x7f9eb80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:54.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.922+0000 7f9eca000640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9eb80a2fc0 0x7f9eb80a5480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009bd0 tx=0x7f9ebc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:54.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:54.929+0000 7f9ecbe00640 1 -- 172.21.15.97:0/960496038 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9ec409c530 con 0x7f9ed4101900 2024-03-26T18:58:55.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:55.202+0000 7f9ed9a00640 1 -- 172.21.15.97:0/960496038 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f9ed41a2530 con 0x7f9ed4101900 2024-03-26T18:58:56.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.180+0000 7f9ecbe00640 1 -- 172.21.15.97:0/960496038 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f9ec409be30 con 0x7f9ed4101900 2024-03-26T18:58:56.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 -- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9eb80a2fc0 msgr2=0x7f9eb80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:56.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9eb80a2fc0 0x7f9eb80a5480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009bd0 tx=0x7f9ebc005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:56.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 -- 172.21.15.97:0/960496038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ed4101900 msgr2=0x7f9ed4199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:56.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ed4101900 0x7f9ed4199d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400e9c0 tx=0x7f9ec400ee80 comp rx=0 tx=0).stop 2024-03-26T18:58:56.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 -- 172.21.15.97:0/960496038 shutdown_connections 2024-03-26T18:58:56.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9eb80a2fc0 0x7f9eb80a5480 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed41034b0 0x7f9ed41a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed4102af0 0x7f9ed419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 --2- 172.21.15.97:0/960496038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ed4101900 0x7f9ed4199d40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 -- 172.21.15.97:0/960496038 >> 172.21.15.97:0/960496038 conn(0x7f9ed40fce50 msgr2=0x7f9ed4073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:56.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 -- 172.21.15.97:0/960496038 shutdown_connections 2024-03-26T18:58:56.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.184+0000 7f9ed9a00640 1 -- 172.21.15.97:0/960496038 wait complete. 2024-03-26T18:58:56.199 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-26T18:58:56.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.391+0000 7fb8dc200640 1 -- 172.21.15.97:0/787788166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4103b20 msgr2=0x7fb8d4103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:56.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.391+0000 7fb8dc200640 1 --2- 172.21.15.97:0/787788166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4103b20 0x7fb8d4103f40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009900 tx=0x7fb8c802bc30 comp rx=0 tx=0).stop 2024-03-26T18:58:56.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.391+0000 7fb8dc200640 1 -- 172.21.15.97:0/787788166 shutdown_connections 2024-03-26T18:58:56.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.391+0000 7fb8dc200640 1 --2- 172.21.15.97:0/787788166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d41056d0 0x7fb8d410bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.391+0000 7fb8dc200640 1 --2- 172.21.15.97:0/787788166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8d4104d10 0x7fb8d4105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.391+0000 7fb8dc200640 1 --2- 172.21.15.97:0/787788166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4103b20 0x7fb8d4103f40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.391+0000 7fb8dc200640 1 -- 172.21.15.97:0/787788166 >> 172.21.15.97:0/787788166 conn(0x7fb8d40ff0b0 msgr2=0x7fb8d41014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:56.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.392+0000 7fb8dc200640 1 -- 172.21.15.97:0/787788166 shutdown_connections 2024-03-26T18:58:56.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.392+0000 7fb8dc200640 1 -- 172.21.15.97:0/787788166 wait complete. 2024-03-26T18:58:56.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.393+0000 7fb8dc200640 1 Processor -- start 2024-03-26T18:58:56.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.393+0000 7fb8dc200640 1 -- start start 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8dc200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4103b20 0x7fb8d4199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8dc200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8d4104d10 0x7fb8d419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8dc200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d41056d0 0x7fb8d41a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8dc200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d410f7e0 con 0x7fb8d41056d0 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8dc200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d410f660 con 0x7fb8d4103b20 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8dc200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d410f960 con 0x7fb8d4104d10 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8d9800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4103b20 0x7fb8d4199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8d9800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4103b20 0x7fb8d4199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60820/0 (socket says 172.21.15.97:60820) 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8d9800640 1 -- 172.21.15.97:0/1032195911 learned_addr learned my addr 172.21.15.97:0/1032195911 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8d8e00640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8d4104d10 0x7fb8d419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8da200640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d41056d0 0x7fb8d41a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8d9800640 1 -- 172.21.15.97:0/1032195911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8d4104d10 msgr2=0x7fb8d419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8d9800640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8d4104d10 0x7fb8d419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8d9800640 1 -- 172.21.15.97:0/1032195911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d41056d0 msgr2=0x7fb8d41a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8d9800640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d41056d0 0x7fb8d41a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.394+0000 7fb8d9800640 1 -- 172.21.15.97:0/1032195911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d41a1990 con 0x7fb8d4103b20 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.395+0000 7fb8d9800640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4103b20 0x7fb8d4199d70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8000c00 tx=0x7fb8c802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.395+0000 7fb8d2000640 1 -- 172.21.15.97:0/1032195911 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8043020 con 0x7fb8d4103b20 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.395+0000 7fb8d2000640 1 -- 172.21.15.97:0/1032195911 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c80054e0 con 0x7fb8d4103b20 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.395+0000 7fb8dc200640 1 -- 172.21.15.97:0/1032195911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d41a1c20 con 0x7fb8d4103b20 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.395+0000 7fb8d2000640 1 -- 172.21.15.97:0/1032195911 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8003c40 con 0x7fb8d4103b20 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.395+0000 7fb8dc200640 1 -- 172.21.15.97:0/1032195911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d41a21c0 con 0x7fb8d4103b20 2024-03-26T18:58:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.396+0000 7fb8dc200640 1 -- 172.21.15.97:0/1032195911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d410ade0 con 0x7fb8d4103b20 2024-03-26T18:58:56.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.400+0000 7fb8d2000640 1 -- 172.21.15.97:0/1032195911 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fb8c8046020 con 0x7fb8d4103b20 2024-03-26T18:58:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.401+0000 7fb8d2000640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb8bc0a3040 0x7fb8bc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:58:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.401+0000 7fb8d2000640 1 -- 172.21.15.97:0/1032195911 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb8c80f1440 con 0x7fb8d4103b20 2024-03-26T18:58:56.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.401+0000 7fb8d8e00640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb8bc0a3040 0x7fb8bc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:58:56.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.402+0000 7fb8d8e00640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb8bc0a3040 0x7fb8bc0a5500 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb8d419b280 tx=0x7fb8c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:58:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.409+0000 7fb8d2000640 1 -- 172.21.15.97:0/1032195911 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb8c804d020 con 0x7fb8d4103b20 2024-03-26T18:58:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:56 smithi105 ceph-mon[36270]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-26T18:58:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:56 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/960496038' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T18:58:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:56 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T18:58:56.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:56 smithi097 ceph-mon[31955]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-26T18:58:56.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:56 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/960496038' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T18:58:56.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:56 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T18:58:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:56 smithi144 ceph-mon[36053]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-26T18:58:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:56 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/960496038' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T18:58:56.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:56 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T18:58:56.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.668+0000 7fb8dc200640 1 -- 172.21.15.97:0/1032195911 --> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fb8d4061e60 con 0x7fb8bc0a3040 2024-03-26T18:58:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.697+0000 7fb8d2000640 1 -- 172.21.15.97:0/1032195911 <== mgr.14152 v2:172.21.15.97:6800/3680188982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fb8d4061e60 con 0x7fb8bc0a3040 2024-03-26T18:58:56.690 INFO:teuthology.orchestra.run.smithi097.stdout:{} 2024-03-26T18:58:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 -- 172.21.15.97:0/1032195911 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb8bc0a3040 msgr2=0x7fb8bc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb8bc0a3040 0x7fb8bc0a5500 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb8d419b280 tx=0x7fb8c0005f90 comp rx=0 tx=0).stop 2024-03-26T18:58:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 -- 172.21.15.97:0/1032195911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4103b20 msgr2=0x7fb8d4199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:58:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4103b20 0x7fb8d4199d70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8000c00 tx=0x7fb8c802c650 comp rx=0 tx=0).stop 2024-03-26T18:58:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 -- 172.21.15.97:0/1032195911 shutdown_connections 2024-03-26T18:58:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb8bc0a3040 0x7fb8bc0a5500 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d41056d0 0x7fb8d41a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8d4104d10 0x7fb8d419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 --2- 172.21.15.97:0/1032195911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4103b20 0x7fb8d4199d70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:58:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 -- 172.21.15.97:0/1032195911 >> 172.21.15.97:0/1032195911 conn(0x7fb8d40ff0b0 msgr2=0x7fb8d4107fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:58:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.700+0000 7fb8dc200640 1 -- 172.21.15.97:0/1032195911 shutdown_connections 2024-03-26T18:58:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T18:58:56.701+0000 7fb8dc200640 1 -- 172.21.15.97:0/1032195911 wait complete. 2024-03-26T18:58:56.712 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-26T18:58:56.724 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-26T18:58:56.733 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-03-26T18:58:56.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-26T18:58:56.738 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T18:58:56.747 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T18:58:56.748 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi097.front.sepia.ceph.com 2024-03-26T18:58:56.748 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-03-26T18:58:56.852 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:58:56.894 INFO:teuthology.orchestra.run.smithi097.stdout:Filesystem Size Used Avail Use% Mounted on 2024-03-26T18:58:56.894 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6789,172.21.15.105:6789,172.21.15.144:6789:/volumes/qa/sv_1/7cc1b6af-7058-492a-908c-827e2e8de910 340G 0 340G 0% /tmp/cephfs 2024-03-26T18:58:56.902 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-03-26T18:58:56.969 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:58:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:58:56.677+0000 7f46e3800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:58:56.969 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:58:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:58:56.677+0000 7f46e3800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:58:56.969 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:58:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:58:56.677+0000 7f46e3800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:58:56.970 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:58:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:58:56.677+0000 7f46e3800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:58:56.970 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 18:58:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x[32196]: 2024-03-26T18:58:56.677+0000 7f46e3800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:58:57.066 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-03-26T18:58:57.199 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-03-26T18:58:57.264 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:57 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T18:58:57.264 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:57 smithi097 ceph-mon[31955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:58:57.264 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:57 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:58:57.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:57 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T18:58:57.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:57 smithi105 ceph-mon[36270]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:58:57.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:57 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:58:57.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:57 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T18:58:57.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:57 smithi144 ceph-mon[36053]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:58:57.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:57 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:58:58.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:58 smithi105 ceph-mon[36270]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:58 smithi105 ceph-mon[36270]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-26T18:58:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:58 smithi105 ceph-mon[36270]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-26T18:58:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:58 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:58 smithi097 ceph-mon[31955]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:58 smithi097 ceph-mon[31955]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-26T18:58:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:58 smithi097 ceph-mon[31955]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-26T18:58:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:58 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:58 smithi144 ceph-mon[36053]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:58:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:58 smithi144 ceph-mon[36053]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-26T18:58:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:58 smithi144 ceph-mon[36053]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-26T18:58:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:58.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:58 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:58:59.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-03-26T18:58:59.879 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-03-26T18:58:59.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-03-26T18:59:00.001 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:59 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.002 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:59 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.002 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:59 smithi097 ceph-mon[31955]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-26T18:59:00.002 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:59 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.002 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:58:59 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.028 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-26T18:59:00.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:59 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:59 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:59 smithi144 ceph-mon[36053]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-26T18:59:00.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:59 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:58:59 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.145 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/: 2024-03-26T18:59:00.145 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:00.146 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:58 . 2024-03-26T18:59:00.146 INFO:teuthology.orchestra.run.smithi097.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:58 .. 2024-03-26T18:59:00.146 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:58 data 2024-03-26T18:59:00.146 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:00.146 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data: 2024-03-26T18:59:00.146 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:00.146 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:58 . 2024-03-26T18:59:00.146 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:58 .. 2024-03-26T18:59:00.151 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-03-26T18:59:00.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:59 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:59 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:59 smithi105 ceph-mon[36270]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-26T18:59:00.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:59 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.211 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:58:59 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:00.305 INFO:teuthology.orchestra.run.smithi097.stdout:[Service] 2024-03-26T18:59:00.305 INFO:teuthology.orchestra.run.smithi097.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-03-26T18:59:00.306 INFO:teuthology.orchestra.run.smithi097.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-03-26T18:59:00.312 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-03-26T18:59:00.521 INFO:teuthology.orchestra.run.smithi097.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-03-26T18:59:02.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:02.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:01 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:02.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:01 smithi097 ceph-mon[31955]: pgmap v393: 97 pgs: 97 active+clean; 659 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-03-26T18:59:02.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:02.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:01 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:02.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:01 smithi144 ceph-mon[36053]: pgmap v393: 97 pgs: 97 active+clean; 659 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-03-26T18:59:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:01 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:02.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:01 smithi105 ceph-mon[36270]: pgmap v393: 97 pgs: 97 active+clean; 659 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-03-26T18:59:03.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:59:03.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:59:03.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:02 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:03.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:59:03.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:59:03.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:02 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:03.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:59:03.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:59:03.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:02 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T18:59:04.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:03 smithi144 ceph-mon[36053]: pgmap v394: 97 pgs: 97 active+clean; 1.6 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 88 KiB/s wr, 14 op/s 2024-03-26T18:59:04.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:03 smithi105 ceph-mon[36270]: pgmap v394: 97 pgs: 97 active+clean; 1.6 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 88 KiB/s wr, 14 op/s 2024-03-26T18:59:04.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:03 smithi097 ceph-mon[31955]: pgmap v394: 97 pgs: 97 active+clean; 1.6 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 88 KiB/s wr, 14 op/s 2024-03-26T18:59:06.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:06 smithi097 ceph-mon[31955]: pgmap v395: 97 pgs: 97 active+clean; 18 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 MiB/s wr, 29 op/s 2024-03-26T18:59:06.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:06 smithi144 ceph-mon[36053]: pgmap v395: 97 pgs: 97 active+clean; 18 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 MiB/s wr, 29 op/s 2024-03-26T18:59:06.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:06 smithi105 ceph-mon[36270]: pgmap v395: 97 pgs: 97 active+clean; 18 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 MiB/s wr, 29 op/s 2024-03-26T18:59:08.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:08 smithi097 ceph-mon[31955]: pgmap v396: 97 pgs: 97 active+clean; 20 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.6 MiB/s wr, 32 op/s 2024-03-26T18:59:08.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:08 smithi144 ceph-mon[36053]: pgmap v396: 97 pgs: 97 active+clean; 20 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.6 MiB/s wr, 32 op/s 2024-03-26T18:59:08.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:08 smithi105 ceph-mon[36270]: pgmap v396: 97 pgs: 97 active+clean; 20 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.6 MiB/s wr, 32 op/s 2024-03-26T18:59:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.007+0000 7f18a9600640 1 -- 172.21.15.97:0/1283954980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a4107430 msgr2=0x7f18a4109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.007+0000 7f18a9600640 1 --2- 172.21.15.97:0/1283954980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a4107430 0x7f18a4109880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f189c009ca0 tx=0x7f189c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.007+0000 7f18a9600640 1 -- 172.21.15.97:0/1283954980 shutdown_connections 2024-03-26T18:59:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.007+0000 7f18a9600640 1 --2- 172.21.15.97:0/1283954980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a4107430 0x7f18a4109880 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.007+0000 7f18a9600640 1 --2- 172.21.15.97:0/1283954980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a4104b00 0x7f18a4106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.007+0000 7f18a9600640 1 --2- 172.21.15.97:0/1283954980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a41021d0 0x7f18a41045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.007+0000 7f18a9600640 1 -- 172.21.15.97:0/1283954980 >> 172.21.15.97:0/1283954980 conn(0x7f18a40fbbc0 msgr2=0x7f18a40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.007+0000 7f18a9600640 1 -- 172.21.15.97:0/1283954980 shutdown_connections 2024-03-26T18:59:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.007+0000 7f18a9600640 1 -- 172.21.15.97:0/1283954980 wait complete. 2024-03-26T18:59:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.008+0000 7f18a9600640 1 Processor -- start 2024-03-26T18:59:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.008+0000 7f18a9600640 1 -- start start 2024-03-26T18:59:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a41021d0 0x7f18a4199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a9600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a4104b00 0x7f18a419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a9600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a4107430 0x7f18a41a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a9600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18a410d3d0 con 0x7f18a41021d0 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a9600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18a410d250 con 0x7f18a4107430 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a9600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18a410d550 con 0x7f18a4104b00 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a4104b00 0x7f18a419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a41021d0 0x7f18a4199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a8c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a4107430 0x7f18a41a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a4104b00 0x7f18a419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:39942/0 (socket says 172.21.15.97:39942) 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.009+0000 7f18a3400640 1 -- 172.21.15.97:0/986969658 learned_addr learned my addr 172.21.15.97:0/986969658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a3400640 1 -- 172.21.15.97:0/986969658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a4107430 msgr2=0x7f18a41a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a3400640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a4107430 0x7f18a41a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a3400640 1 -- 172.21.15.97:0/986969658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a41021d0 msgr2=0x7f18a4199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a3400640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a41021d0 0x7f18a4199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a3400640 1 -- 172.21.15.97:0/986969658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a41a1830 con 0x7f18a4104b00 2024-03-26T18:59:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a8c00640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a4107430 0x7f18a41a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:59:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a3400640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a4104b00 0x7f18a419a160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f189000e9c0 tx=0x7f189000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a0c00640 1 -- 172.21.15.97:0/986969658 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000cd80 con 0x7f18a4104b00 2024-03-26T18:59:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a0c00640 1 -- 172.21.15.97:0/986969658 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1890005500 con 0x7f18a4104b00 2024-03-26T18:59:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a0c00640 1 -- 172.21.15.97:0/986969658 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890010690 con 0x7f18a4104b00 2024-03-26T18:59:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.010+0000 7f18a9600640 1 -- 172.21.15.97:0/986969658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a41a1b20 con 0x7f18a4104b00 2024-03-26T18:59:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.011+0000 7f18a9600640 1 -- 172.21.15.97:0/986969658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a41a2020 con 0x7f18a4104b00 2024-03-26T18:59:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.014+0000 7f18a0c00640 1 -- 172.21.15.97:0/986969658 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137905+0+0 (secure 0 0 0) 0x7f18900046c0 con 0x7f18a4104b00 2024-03-26T18:59:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.014+0000 7f18ad852740 1 Processor -- start 2024-03-26T18:59:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.014+0000 7f18ad852740 1 -- start start 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.014+0000 7f18ad852740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 0x55dad638d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18ad852740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad6392450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18ad852740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 0x55dad6394f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18ad852740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dad62f1e80 con 0x55dad638cf60 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18ad852740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dad629a660 con 0x55dad6392ac0 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18ad852740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dad62ede30 con 0x55dad638d950 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad6392450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18a8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 0x55dad638d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad6392450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:39958/0 (socket says 172.21.15.97:39958) 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18a3e00640 1 -- 172.21.15.97:0/1161706244 learned_addr learned my addr 172.21.15.97:0/1161706244 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18a3400640 1 --2- 172.21.15.97:0/1161706244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 0x55dad6394f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18a3e00640 1 -- 172.21.15.97:0/1161706244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 msgr2=0x55dad6394f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18a3e00640 1 --2- 172.21.15.97:0/1161706244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 0x55dad6394f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18a3e00640 1 -- 172.21.15.97:0/1161706244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 msgr2=0x55dad638d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18a3e00640 1 --2- 172.21.15.97:0/1161706244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 0x55dad638d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.015+0000 7f18a3e00640 1 -- 172.21.15.97:0/1161706244 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dad6006210 con 0x55dad638d950 2024-03-26T18:59:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.016+0000 7f18a3e00640 1 --2- 172.21.15.97:0/1161706244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad6392450 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1894009c90 tx=0x7f1894011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c49cd9311895cd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.016+0000 7f1899200640 1 -- 172.21.15.97:0/1161706244 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894005220 con 0x55dad638d950 2024-03-26T18:59:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.016+0000 7f1899200640 1 -- 172.21.15.97:0/1161706244 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1894004c80 con 0x55dad638d950 2024-03-26T18:59:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.016+0000 7f1899200640 1 -- 172.21.15.97:0/1161706244 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18940198d0 con 0x55dad638d950 2024-03-26T18:59:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.016+0000 7f18a0c00640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18880a2fa0 0x7f18880a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.016+0000 7f18ad852740 1 -- 172.21.15.97:0/1161706244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 msgr2=0x55dad6392450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.016+0000 7f18ad852740 1 --2- 172.21.15.97:0/1161706244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad6392450 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1894009c90 tx=0x7f1894011810 comp rx=0 tx=0).stop 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.017+0000 7f18a0c00640 1 -- 172.21.15.97:0/986969658 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f18900cf930 con 0x7f18a4104b00 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.017+0000 7f18ad852740 1 -- 172.21.15.97:0/1161706244 shutdown_connections 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.017+0000 7f18ad852740 1 --2- 172.21.15.97:0/1161706244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 0x55dad6394f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.017+0000 7f18ad852740 1 --2- 172.21.15.97:0/1161706244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad6392450 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.017+0000 7f18ad852740 1 --2- 172.21.15.97:0/1161706244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 0x55dad638d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.017+0000 7f18ad852740 1 -- 172.21.15.97:0/1161706244 >> 172.21.15.97:0/1161706244 conn(0x55dad63884d0 msgr2=0x55dad63888d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.017+0000 7f18ad852740 1 -- 172.21.15.97:0/1161706244 shutdown_connections 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.017+0000 7f18ad852740 1 -- 172.21.15.97:0/1161706244 wait complete. 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.017+0000 7f18a8c00640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18880a2fa0 0x7f18880a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.017+0000 7f18ad852740 1 -- start start 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.018+0000 7f18ad852740 1 Processor -- start 2024-03-26T18:59:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.018+0000 7f18a8c00640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18880a2fa0 0x7f18880a5460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f189c02c630 tx=0x7f189c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.018+0000 7f18ad852740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 0x55dad6396f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.018+0000 7f18ad852740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad63974d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.018+0000 7f18ad852740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 0x55dad639e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.018+0000 7f18ad852740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dad6395a70 con 0x55dad638cf60 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.018+0000 7f18ad852740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dad62adac0 con 0x55dad6392ac0 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.018+0000 7f18ad852740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dad6395bf0 con 0x55dad638d950 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18ad852740 10 client.0 _ll_register_callbacks cb 0x55dad62ad490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 0x55dad6396f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad63974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a8c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 0x55dad639e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad63974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:39962/0 (socket says 172.21.15.97:39962) 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 0x55dad6396f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33294/0 (socket says 172.21.15.97:33294) 2024-03-26T18:59:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3400640 1 -- 172.21.15.97:0/2132781947 learned_addr learned my addr 172.21.15.97:0/2132781947 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3400640 1 -- 172.21.15.97:0/2132781947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 msgr2=0x55dad639e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3400640 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 0x55dad639e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3400640 1 -- 172.21.15.97:0/2132781947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 msgr2=0x55dad6396f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3400640 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 0x55dad6396f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3400640 1 -- 172.21.15.97:0/2132781947 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad60d2620 con 0x55dad638d950 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a8c00640 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 0x55dad639e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f18a3400640 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad63974d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f18900a4410 tx=0x7f18900cd790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76d488d19b748331 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.019+0000 7f188d600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f188d600640 1 -- 172.21.15.97:0/2132781947 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189009a070 con 0x55dad638d950 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f188d600640 1 -- 172.21.15.97:0/2132781947 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f18900a46b0 con 0x55dad638d950 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f18ad852740 1 -- 172.21.15.97:0/2132781947 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1894004c80 con 0x55dad638d950 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f188d600640 1 -- 172.21.15.97:0/2132781947 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f189009c630 con 0x55dad638d950 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f18ad852740 10 client.25273 fetch_fsmap learned FSMap version 27 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f18ad852740 1 -- 172.21.15.97:0/2132781947 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dad63a1950 con 0x55dad638d950 2024-03-26T18:59:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f188d600640 1 -- 172.21.15.97:0/2132781947 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1894004c80 con 0x55dad638d950 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f188d600640 1 -- 172.21.15.97:0/2132781947 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f189009b510 con 0x55dad638d950 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f18ad852740 10 client.25273 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f18ad852740 10 client.25273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f18ad852740 20 client.25273 populate_metadata read hostname 'smithi097' 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.020+0000 7f18ad852740 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dad63a3090 0x55dad63a5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.021+0000 7f18ad852740 4 client.25273 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.021+0000 7f18ad852740 1 -- 172.21.15.97:0/2132781947 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dad62ff3c0 con 0x55dad63a3090 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.021+0000 7f18a3e00640 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dad63a3090 0x55dad63a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.021+0000 7f18a3e00640 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dad63a3090 0x55dad63a5480 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f0dd4fd8e0a3c1 server_cookie=d1aa9ee3eaa40176 in_seq=0 out_seq=0 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.021+0000 7f188d600640 10 client.25273 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.022+0000 7f188d600640 1 -- 172.21.15.97:0/2132781947 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dad62ff3c0 con 0x55dad63a3090 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.022+0000 7f188d600640 10 client.25273 handle_command_reply: tid=0 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.022+0000 7f18ad852740 1 client.25273 shutdown 2024-03-26T18:59:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.022+0000 7f18ad852740 1 -- 172.21.15.97:0/2132781947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 msgr2=0x55dad63974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.022+0000 7f18ad852740 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad63974d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f18900a4410 tx=0x7f18900cd790 comp rx=0 tx=0).stop 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.022+0000 7f18ad852740 1 -- 172.21.15.97:0/2132781947 shutdown_connections 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.022+0000 7f18ad852740 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dad63a3090 0x55dad63a5480 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.023+0000 7f18ad852740 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad6392ac0 0x55dad639e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.023+0000 7f18ad852740 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad63974d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.023+0000 7f18ad852740 1 --2- 172.21.15.97:0/2132781947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad638cf60 0x55dad6396f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.023+0000 7f18ad852740 1 -- 172.21.15.97:0/2132781947 >> 172.21.15.97:0/2132781947 conn(0x55dad63884d0 msgr2=0x55dad63876f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.023+0000 7f188d600640 0 client.25273 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.023+0000 7f18ad852740 1 -- 172.21.15.97:0/2132781947 shutdown_connections 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.023+0000 7f18ad852740 1 -- 172.21.15.97:0/2132781947 wait complete. 2024-03-26T18:59:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.023+0000 7f18ad852740 20 client.25273 trim_cache size 0 max 16384 2024-03-26T18:59:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.049+0000 7f18ad852740 1 Processor -- start 2024-03-26T18:59:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18ad852740 1 -- start start 2024-03-26T18:59:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18ad852740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638cf60 0x55dad6391fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18ad852740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad639e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18ad852740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 0x55dad6386e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18ad852740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41aade0 con 0x55dad6392ac0 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18ad852740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41ac920 con 0x55dad638cf60 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18ad852740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41ab4b0 con 0x55dad638d950 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638cf60 0x55dad6391fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18a3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad639e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638cf60 0x55dad6391fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:35186/0 (socket says 172.21.15.97:35186) 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18a3e00640 1 -- 172.21.15.97:0/4225967182 learned_addr learned my addr 172.21.15.97:0/4225967182 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.050+0000 7f18a8c00640 1 --2- 172.21.15.97:0/4225967182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 0x55dad6386e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.051+0000 7f18a3e00640 1 -- 172.21.15.97:0/4225967182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 msgr2=0x55dad639e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.051+0000 7f18a3e00640 1 --2- 172.21.15.97:0/4225967182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad639e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.051+0000 7f18a3e00640 1 -- 172.21.15.97:0/4225967182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 msgr2=0x55dad6386e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.051+0000 7f18a3e00640 1 --2- 172.21.15.97:0/4225967182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 0x55dad6386e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.051+0000 7f18a3e00640 1 -- 172.21.15.97:0/4225967182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dad6383ae0 con 0x55dad638cf60 2024-03-26T18:59:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.051+0000 7f18a3e00640 1 --2- 172.21.15.97:0/4225967182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638cf60 0x55dad6391fb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1894009250 tx=0x7f1894009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5859752bd0d2c3f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.051+0000 7f188fe00640 1 -- 172.21.15.97:0/4225967182 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894005000 con 0x55dad638cf60 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.051+0000 7f188fe00640 1 -- 172.21.15.97:0/4225967182 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1894031070 con 0x55dad638cf60 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.052+0000 7f188fe00640 1 -- 172.21.15.97:0/4225967182 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189402b790 con 0x55dad638cf60 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.053+0000 7f18ad852740 1 -- 172.21.15.97:0/4225967182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638cf60 msgr2=0x55dad6391fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.053+0000 7f18ad852740 1 --2- 172.21.15.97:0/4225967182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638cf60 0x55dad6391fb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1894009250 tx=0x7f1894009640 comp rx=0 tx=0).stop 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.053+0000 7f18ad852740 1 -- 172.21.15.97:0/4225967182 shutdown_connections 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.053+0000 7f18ad852740 1 --2- 172.21.15.97:0/4225967182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 0x55dad6386e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.053+0000 7f18ad852740 1 --2- 172.21.15.97:0/4225967182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638d950 0x55dad639e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.053+0000 7f18ad852740 1 --2- 172.21.15.97:0/4225967182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638cf60 0x55dad6391fb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.053+0000 7f18ad852740 1 -- 172.21.15.97:0/4225967182 >> 172.21.15.97:0/4225967182 conn(0x55dad63884d0 msgr2=0x55dad638bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.053+0000 7f18ad852740 1 -- 172.21.15.97:0/4225967182 shutdown_connections 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.053+0000 7f18ad852740 1 -- 172.21.15.97:0/4225967182 wait complete. 2024-03-26T18:59:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.053+0000 7f18ad852740 1 -- start start 2024-03-26T18:59:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.054+0000 7f18ad852740 1 Processor -- start 2024-03-26T18:59:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.054+0000 7f18ad852740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638cf60 0x55dad639bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.054+0000 7f18ad852740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638d950 0x55dad639c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.054+0000 7f18ad852740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 0x55dad639d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.054+0000 7f18ad852740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41ac250 con 0x55dad6392ac0 2024-03-26T18:59:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.054+0000 7f18ad852740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41abb80 con 0x55dad638d950 2024-03-26T18:59:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.054+0000 7f18ad852740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41a8dc0 con 0x55dad638cf60 2024-03-26T18:59:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.054+0000 7f18ad852740 10 client.0 _ll_register_callbacks cb 0x7f1890004a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:59:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638cf60 0x55dad639bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638cf60 0x55dad639bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:39976/0 (socket says 172.21.15.97:39976) 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638d950 0x55dad639c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a3e00640 1 -- 172.21.15.97:0/2305286664 learned_addr learned my addr 172.21.15.97:0/2305286664 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a8c00640 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 0x55dad639d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a3e00640 1 -- 172.21.15.97:0/2305286664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638d950 msgr2=0x55dad639c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a3e00640 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638d950 0x55dad639c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a3e00640 1 -- 172.21.15.97:0/2305286664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 msgr2=0x55dad639d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a3e00640 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 0x55dad639d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a3e00640 1 -- 172.21.15.97:0/2305286664 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad639da50 con 0x55dad638cf60 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a8c00640 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 0x55dad639d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f18a3e00640 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638cf60 0x55dad639bf10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f189401b040 tx=0x7f1894009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1904c160acc1bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.055+0000 7f188d600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.056+0000 7f188d600640 1 -- 172.21.15.97:0/2305286664 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894005380 con 0x55dad638cf60 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.056+0000 7f188d600640 1 -- 172.21.15.97:0/2305286664 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f18940035f0 con 0x55dad638cf60 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.056+0000 7f18ad852740 1 -- 172.21.15.97:0/2305286664 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1894031070 con 0x55dad638cf60 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.056+0000 7f188d600640 1 -- 172.21.15.97:0/2305286664 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18940037f0 con 0x55dad638cf60 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.056+0000 7f18ad852740 10 client.25279 fetch_fsmap learned FSMap version 27 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.056+0000 7f18ad852740 1 -- 172.21.15.97:0/2305286664 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dad64096c0 con 0x55dad638cf60 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.056+0000 7f188d600640 1 -- 172.21.15.97:0/2305286664 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1894031070 con 0x55dad638cf60 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.057+0000 7f188d600640 1 -- 172.21.15.97:0/2305286664 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f189402bbe0 con 0x55dad638cf60 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.057+0000 7f18ad852740 10 client.25279 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.057+0000 7f18ad852740 10 client.25279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.057+0000 7f18ad852740 20 client.25279 populate_metadata read hostname 'smithi097' 2024-03-26T18:59:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.057+0000 7f18ad852740 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dad6409ec0 0x55dad640c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.057+0000 7f18ad852740 4 client.25279 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:59:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.057+0000 7f18ad852740 1 -- 172.21.15.97:0/2305286664 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dad62f9710 con 0x55dad6409ec0 2024-03-26T18:59:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.057+0000 7f18a3400640 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dad6409ec0 0x55dad640c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.057+0000 7f188d600640 10 client.25279 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.057+0000 7f18a3400640 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dad6409ec0 0x55dad640c2e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dfea3c7d96c066a server_cookie=336113bf872b6bdb in_seq=0 out_seq=0 2024-03-26T18:59:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.337+0000 7f188d600640 1 -- 172.21.15.97:0/2305286664 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dad62f9710 con 0x55dad6409ec0 2024-03-26T18:59:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f188d600640 10 client.25279 handle_command_reply: tid=0 2024-03-26T18:59:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 client.25279 shutdown 2024-03-26T18:59:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 -- 172.21.15.97:0/2305286664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638cf60 msgr2=0x55dad639bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638cf60 0x55dad639bf10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f189401b040 tx=0x7f1894009e40 comp rx=0 tx=0).stop 2024-03-26T18:59:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 -- 172.21.15.97:0/2305286664 shutdown_connections 2024-03-26T18:59:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dad6409ec0 0x55dad640c2e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dad6392ac0 0x55dad639d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dad638d950 0x55dad639c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 --2- 172.21.15.97:0/2305286664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dad638cf60 0x55dad639bf10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f188d600640 0 client.25279 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 -- 172.21.15.97:0/2305286664 >> 172.21.15.97:0/2305286664 conn(0x55dad63884d0 msgr2=0x55dad638bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 -- 172.21.15.97:0/2305286664 shutdown_connections 2024-03-26T18:59:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 1 -- 172.21.15.97:0/2305286664 wait complete. 2024-03-26T18:59:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.338+0000 7f18ad852740 20 client.25279 trim_cache size 0 max 16384 2024-03-26T18:59:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T18:59:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T18:59:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T18:59:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T18:59:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 -- 172.21.15.97:0/986969658 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18880a2fa0 msgr2=0x7f18880a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18880a2fa0 0x7f18880a5460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f189c02c630 tx=0x7f189c02e040 comp rx=0 tx=0).stop 2024-03-26T18:59:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 -- 172.21.15.97:0/986969658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a4104b00 msgr2=0x7f18a419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a4104b00 0x7f18a419a160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f189000e9c0 tx=0x7f189000ee80 comp rx=0 tx=0).stop 2024-03-26T18:59:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 -- 172.21.15.97:0/986969658 shutdown_connections 2024-03-26T18:59:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18880a2fa0 0x7f18880a5460 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a4107430 0x7f18a41a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a4104b00 0x7f18a419a160 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 --2- 172.21.15.97:0/986969658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18a41021d0 0x7f18a4199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 -- 172.21.15.97:0/986969658 >> 172.21.15.97:0/986969658 conn(0x7f18a40fbbc0 msgr2=0x7f18a41018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 -- 172.21.15.97:0/986969658 shutdown_connections 2024-03-26T18:59:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:10.339+0000 7f18a9600640 1 -- 172.21.15.97:0/986969658 wait complete. 2024-03-26T18:59:10.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:10 smithi097 ceph-mon[31955]: pgmap v397: 97 pgs: 97 active+clean; 24 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.0 MiB/s wr, 35 op/s 2024-03-26T18:59:10.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:10 smithi144 ceph-mon[36053]: pgmap v397: 97 pgs: 97 active+clean; 24 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.0 MiB/s wr, 35 op/s 2024-03-26T18:59:10.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:10 smithi105 ceph-mon[36270]: pgmap v397: 97 pgs: 97 active+clean; 24 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.0 MiB/s wr, 35 op/s 2024-03-26T18:59:11.558 INFO:teuthology.orchestra.run.smithi097.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-03-26T18:59:11.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/: 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:58 . 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:58 .. 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 data 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data: 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:total 35 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 . 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:58 .. 2024-03-26T18:59:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 base 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Mar 26 18:59 global 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 log 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_commit_ts 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_dynshmem 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Mar 26 18:59 pg_hba.conf 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Mar 26 18:59 pg_ident.conf 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 pg_logical 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 pg_multixact 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_notify 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_replslot 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_serial 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_snapshots 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_stat 2024-03-26T18:59:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_stat_tmp 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 pg_subtrans 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_tblspc 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_twophase 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 PG_VERSION 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 pg_wal 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 pg_xact 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Mar 26 18:59 postgresql.auto.conf 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Mar 26 18:59 postgresql.conf 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/base: 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 . 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:59 1 2024-03-26T18:59:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:59 13493 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:59 13494 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/base/1: 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:total 7777 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:59 . 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 .. 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 112 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 113 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 1247 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1247_fsm 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1247_vm 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:59 1249 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1249_fsm 2024-03-26T18:59:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1249_vm 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:59 1255 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1255_fsm 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1255_vm 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:59 1259 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1259_fsm 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1259_vm 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:59 13344 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13344_fsm 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13344_vm 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13346 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13348 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13349_fsm 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349_vm 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13351 2024-03-26T18:59:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13353 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13354_fsm 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354_vm 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13356 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13358 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13359_fsm 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359_vm 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13361 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13363 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1417 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1418 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 174 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 175 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2187 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2224 2024-03-26T18:59:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2328 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2336 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2337 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2579 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2600 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2600_fsm 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2600_vm 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2601_fsm 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601_vm 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2602 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2602_fsm 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2602_vm 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2603 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2603_fsm 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2603_vm 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2604 2024-03-26T18:59:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2605 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2605_fsm 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2605_vm 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2606_fsm 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606_vm 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2607 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2607_fsm 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2607_vm 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:59 2608 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2608_fsm 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2608_vm 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:59 2609 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2609_fsm 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2609_vm 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2610 2024-03-26T18:59:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2610_fsm 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2610_vm 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2611 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2612_fsm 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612_vm 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2613 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2615_fsm 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615_vm 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2616 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2616_fsm 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2616_vm 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:59 2617 2024-03-26T18:59:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2617_fsm 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2617_vm 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:59 2618 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2618_fsm 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2618_vm 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 18:59 2619 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2619_fsm 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2619_vm 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2620 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2650 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2651 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2652 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2653 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2654 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2655 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2656 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2657 2024-03-26T18:59:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:59 2658 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 2659 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2660 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2661 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2662 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2663 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2664 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2665 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2666 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2667 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2668 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2669 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2670 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:59 2673 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:59 2674 2024-03-26T18:59:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:59 2675 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2678 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2679 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2680 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2681 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2682 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2683 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2684 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2685 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2686 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2687 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2688 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2689 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:59 2690 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:59 2691 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2692 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2693 2024-03-26T18:59:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2696 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2699 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2701 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2702 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2703 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2704 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2753 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2753_fsm 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2753_vm 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2754 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2755 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2756 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2757 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2830 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2831 2024-03-26T18:59:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2832 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2833 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2834 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2835 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2836 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2837 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:59 2838 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2838_fsm 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2838_vm 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2839 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840_fsm 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2840_vm 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2841 2024-03-26T18:59:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2995 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2996 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3079_fsm 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079_vm 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3080 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3081 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3085 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3118 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3119 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:59 3164 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3256 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3257 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3258 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3350 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3351 2024-03-26T18:59:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3379 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3380 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3381 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3394 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3394_fsm 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3394_vm 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3395 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3429 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3430 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3431 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3433 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3439 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3440 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3455 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:59 3456 2024-03-26T18:59:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3456_fsm 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3456_vm 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3466 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3467 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3468 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3501 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3502 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3503 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3534 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3541_fsm 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541_vm 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3542 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3574 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3575 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3576 2024-03-26T18:59:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3596 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3597 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3598 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3599 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3600_fsm 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600_vm 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3601_fsm 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601_vm 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3602_fsm 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602_vm 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603_fsm 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3603_vm 2024-03-26T18:59:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3604 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3605 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3606 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3607 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3608 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 3609 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3712 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3764_fsm 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764_vm 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3766 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3767 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3997 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4143 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4144 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4145 2024-03-26T18:59:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4146 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4147 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4148 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4149 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4150 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4151 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4152 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4153 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4154 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4155 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4156 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4157 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4158 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4159 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4160 2024-03-26T18:59:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4161 2024-03-26T18:59:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4162 2024-03-26T18:59:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4163 2024-03-26T18:59:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4164 2024-03-26T18:59:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4165 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4166 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4167 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4168 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4169 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4170 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4171 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4172 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4173 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4174 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 5002 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 548 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 549 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6102 2024-03-26T18:59:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6104 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6106 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6110 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6111 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6112 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6113 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6117 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 826 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 827 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 828 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:59 pg_filenode.map 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 PG_VERSION 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:total 7777 2024-03-26T18:59:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:59 . 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 .. 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 112 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 113 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 1247 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1247_fsm 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1247_vm 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:59 1249 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1249_fsm 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1249_vm 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:59 1255 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1255_fsm 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1255_vm 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:59 1259 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1259_fsm 2024-03-26T18:59:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1259_vm 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:59 13344 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13344_fsm 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13344_vm 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13346 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13348 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13349_fsm 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349_vm 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13351 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13353 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13354_fsm 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354_vm 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13356 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13358 2024-03-26T18:59:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13359_fsm 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359_vm 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13361 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13363 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1417 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1418 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 174 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 175 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2187 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2224 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2328 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2336 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2337 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2579 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2600 2024-03-26T18:59:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2600_fsm 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2600_vm 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2601_fsm 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601_vm 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2602 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2602_fsm 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2602_vm 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2603 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2603_fsm 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2603_vm 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2604 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2605 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2605_fsm 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2605_vm 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606 2024-03-26T18:59:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2606_fsm 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606_vm 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2607 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2607_fsm 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2607_vm 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:59 2608 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2608_fsm 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2608_vm 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:59 2609 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2609_fsm 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2609_vm 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2610 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2610_fsm 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2610_vm 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2611 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612 2024-03-26T18:59:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2612_fsm 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612_vm 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2613 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2615_fsm 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615_vm 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2616 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2616_fsm 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2616_vm 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:59 2617 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2617_fsm 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2617_vm 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:59 2618 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2618_fsm 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2618_vm 2024-03-26T18:59:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 18:59 2619 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2619_fsm 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2619_vm 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2620 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2650 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2651 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2652 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2653 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2654 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2655 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2656 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2657 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:59 2658 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 2659 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2660 2024-03-26T18:59:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2661 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2662 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2663 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2664 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2665 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2666 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2667 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2668 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2669 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2670 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:59 2673 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:59 2674 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:59 2675 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2678 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2679 2024-03-26T18:59:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2680 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2681 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2682 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2683 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2684 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2685 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2686 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2687 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2688 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2689 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:59 2690 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:59 2691 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2692 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2693 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2696 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2699 2024-03-26T18:59:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2701 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2702 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2703 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2704 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2753 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2753_fsm 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2753_vm 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2754 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2755 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2756 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2757 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2830 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2831 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2832 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2833 2024-03-26T18:59:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2834 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2835 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2836 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2837 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:59 2838 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2838_fsm 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2838_vm 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2839 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840_fsm 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2840_vm 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2841 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2995 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2996 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079 2024-03-26T18:59:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3079_fsm 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079_vm 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3080 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3081 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3085 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3118 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3119 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:59 3164 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3256 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3257 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3258 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3350 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3351 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3379 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3380 2024-03-26T18:59:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3381 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3394 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3394_fsm 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3394_vm 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3395 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3429 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3430 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3431 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3433 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3439 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3440 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3455 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:59 3456 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3456_fsm 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3456_vm 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3466 2024-03-26T18:59:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3467 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3468 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3501 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3502 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3503 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3534 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3541_fsm 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541_vm 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3542 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3574 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3575 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3576 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3596 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3597 2024-03-26T18:59:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3598 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3599 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3600_fsm 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600_vm 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3601_fsm 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601_vm 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3602_fsm 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602_vm 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603_fsm 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3603_vm 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3604 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3605 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3606 2024-03-26T18:59:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3607 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3608 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 3609 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3712 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3764_fsm 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764_vm 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3766 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3767 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3997 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4143 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4144 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4145 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4146 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4147 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4148 2024-03-26T18:59:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4149 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4150 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4151 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4152 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4153 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4154 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4155 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4156 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4157 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4158 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4159 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4160 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4161 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4162 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4163 2024-03-26T18:59:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4164 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4165 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4166 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4167 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4168 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4169 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4170 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4171 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4172 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4173 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4174 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 5002 2024-03-26T18:59:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 548 2024-03-26T18:59:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 549 2024-03-26T18:59:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6102 2024-03-26T18:59:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6104 2024-03-26T18:59:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6106 2024-03-26T18:59:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6110 2024-03-26T18:59:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6111 2024-03-26T18:59:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6112 2024-03-26T18:59:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6113 2024-03-26T18:59:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6117 2024-03-26T18:59:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 826 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 827 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 828 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:59 pg_filenode.map 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 PG_VERSION 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:total 7777 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:59 . 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 .. 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 112 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 113 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 1247 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1247_fsm 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1247_vm 2024-03-26T18:59:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:59 1249 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1249_fsm 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1249_vm 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:59 1255 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1255_fsm 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1255_vm 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:59 1259 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1259_fsm 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1259_vm 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:59 13344 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13344_fsm 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13344_vm 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13346 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13348 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349 2024-03-26T18:59:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13349_fsm 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349_vm 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13351 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13353 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13354_fsm 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354_vm 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13356 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13358 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13359_fsm 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359_vm 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13361 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13363 2024-03-26T18:59:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1417 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1418 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 174 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 175 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2187 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2224 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2328 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2336 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2337 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2579 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2600 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2600_fsm 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2600_vm 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2601_fsm 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601_vm 2024-03-26T18:59:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2602 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2602_fsm 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2602_vm 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2603 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2603_fsm 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2603_vm 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2604 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2605 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2605_fsm 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2605_vm 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2606_fsm 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606_vm 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2607 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2607_fsm 2024-03-26T18:59:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2607_vm 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:59 2608 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2608_fsm 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2608_vm 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:59 2609 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2609_fsm 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2609_vm 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2610 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2610_fsm 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2610_vm 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2611 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2612_fsm 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612_vm 2024-03-26T18:59:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2613 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2615_fsm 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615_vm 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2616 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2616_fsm 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2616_vm 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:59 2617 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2617_fsm 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2617_vm 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:59 2618 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2618_fsm 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2618_vm 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 18:59 2619 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2619_fsm 2024-03-26T18:59:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2619_vm 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2620 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2650 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2651 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2652 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2653 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2654 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2655 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2656 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2657 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:59 2658 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 2659 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2660 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2661 2024-03-26T18:59:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2662 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2663 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2664 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2665 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2666 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2667 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2668 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2669 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2670 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:59 2673 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:59 2674 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:59 2675 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2678 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2679 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2680 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2681 2024-03-26T18:59:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2682 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2683 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2684 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2685 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2686 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2687 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2688 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2689 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:59 2690 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:59 2691 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2692 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2693 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2696 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2699 2024-03-26T18:59:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2701 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2702 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2703 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2704 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2753 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2753_fsm 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2753_vm 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2754 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2755 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2756 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2757 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2830 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2831 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2832 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2833 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2834 2024-03-26T18:59:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2835 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2836 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2837 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:59 2838 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2838_fsm 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2838_vm 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2839 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840_fsm 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2840_vm 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2841 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2995 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2996 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3079_fsm 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079_vm 2024-03-26T18:59:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3080 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3081 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3085 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3118 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3119 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:59 3164 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3256 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3257 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3258 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3350 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3351 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3379 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3380 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3381 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3394 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3394_fsm 2024-03-26T18:59:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3394_vm 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3395 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3429 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3430 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3431 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3433 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3439 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3440 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3455 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:59 3456 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3456_fsm 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3456_vm 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3466 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3467 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3468 2024-03-26T18:59:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3501 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3502 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3503 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3534 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3541_fsm 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541_vm 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3542 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3574 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3575 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3576 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3596 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3597 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3598 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3599 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3600_fsm 2024-03-26T18:59:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600_vm 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3601_fsm 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601_vm 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3602_fsm 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602_vm 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603_fsm 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3603_vm 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3604 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3605 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3606 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3607 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3608 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 3609 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3712 2024-03-26T18:59:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3764_fsm 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764_vm 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3766 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3767 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3997 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4143 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4144 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4145 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4146 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4147 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4148 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4149 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4150 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4151 2024-03-26T18:59:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4152 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4153 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4154 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4155 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4156 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4157 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4158 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4159 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4160 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4161 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4162 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4163 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4164 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4165 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4166 2024-03-26T18:59:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4167 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4168 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4169 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4170 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4171 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4172 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4173 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4174 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 5002 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 548 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 549 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6102 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6104 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6106 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6110 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6111 2024-03-26T18:59:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6112 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6113 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6117 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 826 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 827 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 828 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:59 pg_filenode.map 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 PG_VERSION 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/global: 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:total 537 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Mar 26 18:59 . 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1213 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1213_fsm 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1213_vm 2024-03-26T18:59:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1214 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1214_fsm 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1214_vm 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 1232 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 1233 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1260 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1260_fsm 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1260_vm 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1261 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1261_fsm 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1261_vm 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1262 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1262_fsm 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1262_vm 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2396 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2396_fsm 2024-03-26T18:59:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2396_vm 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2397 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2671 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2672 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2676 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2677 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2694 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2695 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2697 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2698 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2846 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2847 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2964 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2965 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2966 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2967 2024-03-26T18:59:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3592 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3593 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4060 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4061 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4175 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4176 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4177 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4178 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4181 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4182 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4183 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4184 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4185 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4186 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6000 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6001 2024-03-26T18:59:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6002 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6100 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6114 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6115 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 pg_control 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:59 pg_filenode.map 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/log: 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:total 1 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 . 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 mappings 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Mar 26 18:59 replorigin_checkpoint 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 snapshots 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 .. 2024-03-26T18:59:11.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 .. 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 . 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 members 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 offsets 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-03-26T18:59:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:total 8 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 . 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 .. 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 0000 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:total 8 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 . 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 .. 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 0000 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:total 8 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 . 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 0000 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:total 16384 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 . 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:59 000000010000000000000001 2024-03-26T18:59:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 archive_status 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 .. 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:total 8 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 . 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:59 .. 2024-03-26T18:59:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 0000 2024-03-26T18:59:11.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-03-26T18:59:12.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-03-26T18:59:12.140 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:12 smithi097 ceph-mon[31955]: pgmap v398: 97 pgs: 97 active+clean; 25 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.1 MiB/s wr, 50 op/s 2024-03-26T18:59:12.199 INFO:teuthology.orchestra.run.smithi097.stderr:dropping old tables... 2024-03-26T18:59:12.200 INFO:teuthology.orchestra.run.smithi097.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-03-26T18:59:12.200 INFO:teuthology.orchestra.run.smithi097.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-03-26T18:59:12.200 INFO:teuthology.orchestra.run.smithi097.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-03-26T18:59:12.200 INFO:teuthology.orchestra.run.smithi097.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-03-26T18:59:12.200 INFO:teuthology.orchestra.run.smithi097.stderr:creating tables... 2024-03-26T18:59:12.216 INFO:teuthology.orchestra.run.smithi097.stderr:generating data (client-side)... 2024-03-26T18:59:12.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:12 smithi144 ceph-mon[36053]: pgmap v398: 97 pgs: 97 active+clean; 25 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.1 MiB/s wr, 50 op/s 2024-03-26T18:59:12.382 INFO:teuthology.orchestra.run.smithi097.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.14 s, remaining 4.28 s) 2024-03-26T18:59:12.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:12 smithi105 ceph-mon[36270]: pgmap v398: 97 pgs: 97 active+clean; 25 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.1 MiB/s wr, 50 op/s 2024-03-26T18:59:12.736 INFO:teuthology.orchestra.run.smithi097.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.49 s, remaining 7.37 s) 2024-03-26T18:59:13.029 INFO:teuthology.orchestra.run.smithi097.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.78 s, remaining 7.58 s) 2024-03-26T18:59:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.95 s, remaining 6.63 s) 2024-03-26T18:59:13.566 INFO:teuthology.orchestra.run.smithi097.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.32 s, remaining 7.13 s) 2024-03-26T18:59:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.65 s, remaining 7.13 s) 2024-03-26T18:59:13.998 INFO:teuthology.orchestra.run.smithi097.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.75 s, remaining 6.26 s) 2024-03-26T18:59:14.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:14 smithi097 ceph-mon[31955]: pgmap v399: 97 pgs: 97 active+clean; 30 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 2.6 MiB/s wr, 112 op/s 2024-03-26T18:59:14.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:14 smithi144 ceph-mon[36053]: pgmap v399: 97 pgs: 97 active+clean; 30 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 2.6 MiB/s wr, 112 op/s 2024-03-26T18:59:14.399 INFO:teuthology.orchestra.run.smithi097.stderr:800000 of 3200000 tuples (25%) done (elapsed 2.15 s, remaining 6.46 s) 2024-03-26T18:59:14.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:14 smithi105 ceph-mon[36270]: pgmap v399: 97 pgs: 97 active+clean; 30 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 2.6 MiB/s wr, 112 op/s 2024-03-26T18:59:14.536 INFO:teuthology.orchestra.run.smithi097.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.29 s, remaining 5.86 s) 2024-03-26T18:59:14.847 INFO:teuthology.orchestra.run.smithi097.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.60 s, remaining 5.72 s) 2024-03-26T18:59:15.263 INFO:teuthology.orchestra.run.smithi097.stderr:1100000 of 3200000 tuples (34%) done (elapsed 3.02 s, remaining 5.76 s) 2024-03-26T18:59:15.361 INFO:teuthology.orchestra.run.smithi097.stderr:1200000 of 3200000 tuples (37%) done (elapsed 3.12 s, remaining 5.19 s) 2024-03-26T18:59:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.54 s, remaining 5.18 s) 2024-03-26T18:59:16.207 INFO:teuthology.orchestra.run.smithi097.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.96 s, remaining 5.09 s) 2024-03-26T18:59:16.328 INFO:teuthology.orchestra.run.smithi097.stderr:1500000 of 3200000 tuples (46%) done (elapsed 4.08 s, remaining 4.63 s) 2024-03-26T18:59:16.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:16 smithi097 ceph-mon[31955]: pgmap v400: 97 pgs: 97 active+clean; 60 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 7.3 MiB/s wr, 227 op/s 2024-03-26T18:59:16.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:16 smithi144 ceph-mon[36053]: pgmap v400: 97 pgs: 97 active+clean; 60 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 7.3 MiB/s wr, 227 op/s 2024-03-26T18:59:16.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:16 smithi105 ceph-mon[36270]: pgmap v400: 97 pgs: 97 active+clean; 60 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 7.3 MiB/s wr, 227 op/s 2024-03-26T18:59:16.652 INFO:teuthology.orchestra.run.smithi097.stderr:1600000 of 3200000 tuples (50%) done (elapsed 4.41 s, remaining 4.41 s) 2024-03-26T18:59:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:1700000 of 3200000 tuples (53%) done (elapsed 4.63 s, remaining 4.08 s) 2024-03-26T18:59:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.89 s, remaining 3.80 s) 2024-03-26T18:59:17.447 INFO:teuthology.orchestra.run.smithi097.stderr:1900000 of 3200000 tuples (59%) done (elapsed 5.20 s, remaining 3.56 s) 2024-03-26T18:59:17.623 INFO:teuthology.orchestra.run.smithi097.stderr:2000000 of 3200000 tuples (62%) done (elapsed 5.38 s, remaining 3.23 s) 2024-03-26T18:59:17.942 INFO:teuthology.orchestra.run.smithi097.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.70 s, remaining 2.98 s) 2024-03-26T18:59:18.326 INFO:teuthology.orchestra.run.smithi097.stderr:2200000 of 3200000 tuples (68%) done (elapsed 6.08 s, remaining 2.76 s) 2024-03-26T18:59:18.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:18 smithi097 ceph-mon[31955]: pgmap v401: 97 pgs: 97 active+clean; 72 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 7.4 MiB/s wr, 288 op/s 2024-03-26T18:59:18.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:18 smithi144 ceph-mon[36053]: pgmap v401: 97 pgs: 97 active+clean; 72 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 7.4 MiB/s wr, 288 op/s 2024-03-26T18:59:18.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:18 smithi105 ceph-mon[36270]: pgmap v401: 97 pgs: 97 active+clean; 72 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 7.4 MiB/s wr, 288 op/s 2024-03-26T18:59:18.488 INFO:teuthology.orchestra.run.smithi097.stderr:2300000 of 3200000 tuples (71%) done (elapsed 6.24 s, remaining 2.44 s) 2024-03-26T18:59:18.788 INFO:teuthology.orchestra.run.smithi097.stderr:2400000 of 3200000 tuples (75%) done (elapsed 6.54 s, remaining 2.18 s) 2024-03-26T18:59:19.114 INFO:teuthology.orchestra.run.smithi097.stderr:2500000 of 3200000 tuples (78%) done (elapsed 6.87 s, remaining 1.92 s) 2024-03-26T18:59:19.285 INFO:teuthology.orchestra.run.smithi097.stderr:2600000 of 3200000 tuples (81%) done (elapsed 7.04 s, remaining 1.62 s) 2024-03-26T18:59:19.620 INFO:teuthology.orchestra.run.smithi097.stderr:2700000 of 3200000 tuples (84%) done (elapsed 7.38 s, remaining 1.37 s) 2024-03-26T18:59:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2800000 of 3200000 tuples (87%) done (elapsed 7.69 s, remaining 1.10 s) 2024-03-26T18:59:20.043 INFO:teuthology.orchestra.run.smithi097.stderr:2900000 of 3200000 tuples (90%) done (elapsed 7.80 s, remaining 0.81 s) 2024-03-26T18:59:20.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:20 smithi097 ceph-mon[31955]: pgmap v402: 97 pgs: 97 active+clean; 196 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 28 MiB/s wr, 298 op/s 2024-03-26T18:59:20.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:20 smithi144 ceph-mon[36053]: pgmap v402: 97 pgs: 97 active+clean; 196 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 28 MiB/s wr, 298 op/s 2024-03-26T18:59:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:3000000 of 3200000 tuples (93%) done (elapsed 8.14 s, remaining 0.54 s) 2024-03-26T18:59:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:20 smithi105 ceph-mon[36270]: pgmap v402: 97 pgs: 97 active+clean; 196 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 28 MiB/s wr, 298 op/s 2024-03-26T18:59:20.689 INFO:teuthology.orchestra.run.smithi097.stderr:3100000 of 3200000 tuples (96%) done (elapsed 8.44 s, remaining 0.27 s) 2024-03-26T18:59:20.812 INFO:teuthology.orchestra.run.smithi097.stderr:3200000 of 3200000 tuples (100%) done (elapsed 8.57 s, remaining 0.00 s) 2024-03-26T18:59:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:vacuuming... 2024-03-26T18:59:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:creating primary keys... 2024-03-26T18:59:22.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:22 smithi097 ceph-mon[31955]: pgmap v403: 97 pgs: 97 active+clean; 264 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 39 MiB/s wr, 301 op/s 2024-03-26T18:59:22.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:22 smithi144 ceph-mon[36053]: pgmap v403: 97 pgs: 97 active+clean; 264 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 39 MiB/s wr, 301 op/s 2024-03-26T18:59:22.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:22 smithi105 ceph-mon[36270]: pgmap v403: 97 pgs: 97 active+clean; 264 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 39 MiB/s wr, 301 op/s 2024-03-26T18:59:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.495+0000 7f2346600640 1 -- 172.21.15.97:0/2634509396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23401019f0 msgr2=0x7f2340101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.495+0000 7f2346600640 1 --2- 172.21.15.97:0/2634509396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23401019f0 0x7f2340101e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2328009860 tx=0x7f232802bb70 comp rx=0 tx=0).stop 2024-03-26T18:59:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.495+0000 7f2346600640 1 -- 172.21.15.97:0/2634509396 shutdown_connections 2024-03-26T18:59:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.495+0000 7f2346600640 1 --2- 172.21.15.97:0/2634509396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23401023b0 0x7f2340108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.495+0000 7f2346600640 1 --2- 172.21.15.97:0/2634509396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23401019f0 0x7f2340101e70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.495+0000 7f2346600640 1 --2- 172.21.15.97:0/2634509396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2340100800 0x7f2340100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.495+0000 7f2346600640 1 -- 172.21.15.97:0/2634509396 >> 172.21.15.97:0/2634509396 conn(0x7f23400fbd70 msgr2=0x7f23400fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.495+0000 7f2346600640 1 -- 172.21.15.97:0/2634509396 shutdown_connections 2024-03-26T18:59:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.496+0000 7f2346600640 1 -- 172.21.15.97:0/2634509396 wait complete. 2024-03-26T18:59:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.497+0000 7f2346600640 1 Processor -- start 2024-03-26T18:59:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.497+0000 7f2346600640 1 -- start start 2024-03-26T18:59:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f2346600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2340100800 0x7f2340104740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f2346600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23401019f0 0x7f2340102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f2346600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23401023b0 0x7f2340103310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f2346600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f234010c480 con 0x7f23401019f0 2024-03-26T18:59:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f2346600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f234010c300 con 0x7f2340100800 2024-03-26T18:59:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f2346600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f234010c600 con 0x7f23401023b0 2024-03-26T18:59:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f233f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2340100800 0x7f2340104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f233fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23401023b0 0x7f2340103310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f233f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2340100800 0x7f2340104740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:33776/0 (socket says 172.21.15.97:33776) 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f233f400640 1 -- 172.21.15.97:0/1505783593 learned_addr learned my addr 172.21.15.97:0/1505783593 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f2337e00640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23401019f0 0x7f2340102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f233f400640 1 -- 172.21.15.97:0/1505783593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23401023b0 msgr2=0x7f2340103310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f233f400640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23401023b0 0x7f2340103310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f233f400640 1 -- 172.21.15.97:0/1505783593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23401019f0 msgr2=0x7f2340102dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f233f400640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23401019f0 0x7f2340102dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f233f400640 1 -- 172.21.15.97:0/1505783593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2340103bd0 con 0x7f2340100800 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.498+0000 7f233fe00640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23401023b0 0x7f2340103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.499+0000 7f233f400640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2340100800 0x7f2340104740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f233000e9f0 tx=0x7f233000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.499+0000 7f233cc00640 1 -- 172.21.15.97:0/1505783593 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233000ccc0 con 0x7f2340100800 2024-03-26T18:59:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.499+0000 7f233cc00640 1 -- 172.21.15.97:0/1505783593 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23300035f0 con 0x7f2340100800 2024-03-26T18:59:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.499+0000 7f233cc00640 1 -- 172.21.15.97:0/1505783593 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233000ce60 con 0x7f2340100800 2024-03-26T18:59:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.499+0000 7f2346600640 1 -- 172.21.15.97:0/1505783593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23401a1d20 con 0x7f2340100800 2024-03-26T18:59:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.499+0000 7f2346600640 1 -- 172.21.15.97:0/1505783593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23401a2190 con 0x7f2340100800 2024-03-26T18:59:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.504+0000 7f2349418740 1 Processor -- start 2024-03-26T18:59:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.504+0000 7f2349418740 1 -- start start 2024-03-26T18:59:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.504+0000 7f2349418740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f74733cda0 0x55f74733d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.504+0000 7f2349418740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733d790 0x55f747342290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.504+0000 7f233cc00640 1 -- 172.21.15.97:0/1505783593 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137905+0+0 (secure 0 0 0) 0x7f23300037a0 con 0x7f2340100800 2024-03-26T18:59:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.504+0000 7f2349418740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747342900 0x55f747344d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.504+0000 7f2349418740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f7472a1e80 con 0x55f74733d790 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.504+0000 7f2349418740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f74724a660 con 0x55f747342900 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.504+0000 7f2349418740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f74729de30 con 0x55f74733cda0 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.505+0000 7f2337e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747342900 0x55f747344d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.505+0000 7f2337e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747342900 0x55f747344d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:33778/0 (socket says 172.21.15.97:33778) 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.505+0000 7f2337e00640 1 -- 172.21.15.97:0/2831470884 learned_addr learned my addr 172.21.15.97:0/2831470884 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.505+0000 7f233f400640 1 --2- 172.21.15.97:0/2831470884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733d790 0x55f747342290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.505+0000 7f2337e00640 1 -- 172.21.15.97:0/2831470884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f74733cda0 msgr2=0x55f74733d1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.505+0000 7f2337e00640 1 --2- 172.21.15.97:0/2831470884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f74733cda0 0x55f74733d1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.505+0000 7f2337e00640 1 -- 172.21.15.97:0/2831470884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733d790 msgr2=0x55f747342290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.505+0000 7f2337e00640 1 --2- 172.21.15.97:0/2831470884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733d790 0x55f747342290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.505+0000 7f2337e00640 1 -- 172.21.15.97:0/2831470884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f746fb6210 con 0x55f747342900 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.506+0000 7f2337e00640 1 --2- 172.21.15.97:0/2831470884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747342900 0x55f747344d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2328037040 tx=0x7f2328005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d38d6caa8596e50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.506+0000 7f2334c00640 1 -- 172.21.15.97:0/2831470884 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2328039040 con 0x55f747342900 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.506+0000 7f2334c00640 1 -- 172.21.15.97:0/2831470884 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2328034db0 con 0x55f747342900 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.506+0000 7f2334c00640 1 -- 172.21.15.97:0/2831470884 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232802c800 con 0x55f747342900 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.506+0000 7f233cc00640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f23240a30e0 0x7f23240a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.507+0000 7f2349418740 1 -- 172.21.15.97:0/2831470884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747342900 msgr2=0x55f747344d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.507+0000 7f2349418740 1 --2- 172.21.15.97:0/2831470884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747342900 0x55f747344d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2328037040 tx=0x7f2328005b20 comp rx=0 tx=0).stop 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.507+0000 7f233cc00640 1 -- 172.21.15.97:0/1505783593 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f23300cf7b0 con 0x7f2340100800 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.508+0000 7f2349418740 1 -- 172.21.15.97:0/2831470884 shutdown_connections 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.508+0000 7f2349418740 1 --2- 172.21.15.97:0/2831470884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747342900 0x55f747344d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.508+0000 7f2349418740 1 --2- 172.21.15.97:0/2831470884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733d790 0x55f747342290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.508+0000 7f2349418740 1 --2- 172.21.15.97:0/2831470884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f74733cda0 0x55f74733d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.508+0000 7f2349418740 1 -- 172.21.15.97:0/2831470884 >> 172.21.15.97:0/2831470884 conn(0x55f747338310 msgr2=0x55f747338710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.509+0000 7f233fe00640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f23240a30e0 0x7f23240a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.509+0000 7f2349418740 1 -- 172.21.15.97:0/2831470884 shutdown_connections 2024-03-26T18:59:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.509+0000 7f2349418740 1 -- 172.21.15.97:0/2831470884 wait complete. 2024-03-26T18:59:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.509+0000 7f233fe00640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f23240a30e0 0x7f23240a55a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55f74733dff0 tx=0x7f2338009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.511+0000 7f2349418740 1 -- start start 2024-03-26T18:59:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.511+0000 7f2349418740 1 Processor -- start 2024-03-26T18:59:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f2349418740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733cda0 0x55f747346d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f2349418740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 0x55f747347280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f2349418740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f747342900 0x55f74734e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f2349418740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f747345850 con 0x55f74733cda0 2024-03-26T18:59:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f2349418740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f74725dac0 con 0x55f74733d790 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f2349418740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f7473459d0 con 0x55f747342900 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f2349418740 10 client.0 _ll_register_callbacks cb 0x55f74725d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f233f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733cda0 0x55f747346d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f233f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733cda0 0x55f747346d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55742/0 (socket says 172.21.15.97:55742) 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f233f400640 1 -- 172.21.15.97:0/3271501689 learned_addr learned my addr 172.21.15.97:0/3271501689 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f2337e00640 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 0x55f747347280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f233fe00640 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f747342900 0x55f74734e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f233f400640 1 -- 172.21.15.97:0/3271501689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f747342900 msgr2=0x55f74734e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f233f400640 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f747342900 0x55f74734e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f233f400640 1 -- 172.21.15.97:0/3271501689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 msgr2=0x55f747347280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.512+0000 7f233f400640 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 0x55f747347280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.513+0000 7f233f400640 1 -- 172.21.15.97:0/3271501689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f747082620 con 0x55f74733cda0 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.513+0000 7f233f400640 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733cda0 0x55f747346d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f233000c6a0 tx=0x7f233009bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bfc82678df4a61 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.513+0000 7f232d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T18:59:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.513+0000 7f232d600640 1 -- 172.21.15.97:0/3271501689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233009bd50 con 0x55f74733cda0 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.513+0000 7f2349418740 1 -- 172.21.15.97:0/3271501689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2328034db0 con 0x55f74733cda0 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.513+0000 7f232d600640 1 -- 172.21.15.97:0/3271501689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f23300cd380 con 0x55f74733cda0 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f232d600640 1 -- 172.21.15.97:0/3271501689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f233009fea0 con 0x55f74733cda0 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f2349418740 10 client.15180 fetch_fsmap learned FSMap version 27 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f2349418740 1 -- 172.21.15.97:0/3271501689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f7473516f0 con 0x55f74733cda0 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f232d600640 1 -- 172.21.15.97:0/3271501689 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2328034db0 con 0x55f74733cda0 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f232d600640 1 -- 172.21.15.97:0/3271501689 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f233007abc0 con 0x55f74733cda0 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f2349418740 10 client.15180 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f2349418740 10 client.15180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f2349418740 20 client.15180 populate_metadata read hostname 'smithi097' 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f2349418740 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f747352e10 0x55f7473552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f2349418740 4 client.15180 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.514+0000 7f2349418740 1 -- 172.21.15.97:0/3271501689 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7472af3c0 con 0x55f747352e10 2024-03-26T18:59:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.515+0000 7f2337e00640 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f747352e10 0x55f7473552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.515+0000 7f2337e00640 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f747352e10 0x55f7473552a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880aaa7a48540c5a server_cookie=fbfd1d215d1664cc in_seq=0 out_seq=0 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.515+0000 7f232d600640 10 client.15180 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.516+0000 7f232d600640 1 -- 172.21.15.97:0/3271501689 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f7472af3c0 con 0x55f747352e10 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.516+0000 7f232d600640 10 client.15180 handle_command_reply: tid=0 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.516+0000 7f2349418740 1 client.15180 shutdown 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 1 -- 172.21.15.97:0/3271501689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733cda0 msgr2=0x55f747346d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733cda0 0x55f747346d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f233000c6a0 tx=0x7f233009bbe0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 1 -- 172.21.15.97:0/3271501689 shutdown_connections 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f747352e10 0x55f7473552a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f747342900 0x55f74734e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f232d600640 0 client.15180 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 0x55f747347280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 1 --2- 172.21.15.97:0/3271501689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733cda0 0x55f747346d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 1 -- 172.21.15.97:0/3271501689 >> 172.21.15.97:0/3271501689 conn(0x55f747338310 msgr2=0x55f747337540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 1 -- 172.21.15.97:0/3271501689 shutdown_connections 2024-03-26T18:59:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 1 -- 172.21.15.97:0/3271501689 wait complete. 2024-03-26T18:59:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.517+0000 7f2349418740 20 client.15180 trim_cache size 0 max 16384 2024-03-26T18:59:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.539+0000 7f2349418740 1 Processor -- start 2024-03-26T18:59:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.539+0000 7f2349418740 1 -- start start 2024-03-26T18:59:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.539+0000 7f2349418740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f74733cda0 0x55f747347060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.539+0000 7f2349418740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 0x55f74733e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.539+0000 7f2349418740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f747342900 0x55f74733e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.539+0000 7f2349418740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23401a6ae0 con 0x55f747342900 2024-03-26T18:59:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.539+0000 7f2349418740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23401a8620 con 0x55f74733d790 2024-03-26T18:59:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.539+0000 7f2349418740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23401a71b0 con 0x55f74733cda0 2024-03-26T18:59:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.540+0000 7f2337e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 0x55f74733e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.540+0000 7f2337e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 0x55f74733e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:33794/0 (socket says 172.21.15.97:33794) 2024-03-26T18:59:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.540+0000 7f2337e00640 1 -- 172.21.15.97:0/1806403484 learned_addr learned my addr 172.21.15.97:0/1806403484 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.540+0000 7f2337e00640 1 -- 172.21.15.97:0/1806403484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f74733cda0 msgr2=0x55f747347060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T18:59:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.540+0000 7f2337e00640 1 --2- 172.21.15.97:0/1806403484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f74733cda0 0x55f747347060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.540+0000 7f2337e00640 1 -- 172.21.15.97:0/1806403484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f747342900 msgr2=0x55f74733e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.540+0000 7f2337e00640 1 --2- 172.21.15.97:0/1806403484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f747342900 0x55f74733e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.540+0000 7f2337e00640 1 -- 172.21.15.97:0/1806403484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f747345790 con 0x55f74733d790 2024-03-26T18:59:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.540+0000 7f2337e00640 1 --2- 172.21.15.97:0/1806403484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 0x55f74733e020 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2328009780 tx=0x7f2328034f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ed7a76e853c5af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.541+0000 7f232fe00640 1 -- 172.21.15.97:0/1806403484 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2328063020 con 0x55f74733d790 2024-03-26T18:59:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.541+0000 7f232fe00640 1 -- 172.21.15.97:0/1806403484 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2328058040 con 0x55f74733d790 2024-03-26T18:59:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.541+0000 7f232fe00640 1 -- 172.21.15.97:0/1806403484 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2328057040 con 0x55f74733d790 2024-03-26T18:59:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.541+0000 7f2349418740 1 -- 172.21.15.97:0/1806403484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 msgr2=0x55f74733e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.541+0000 7f2349418740 1 --2- 172.21.15.97:0/1806403484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 0x55f74733e020 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2328009780 tx=0x7f2328034f60 comp rx=0 tx=0).stop 2024-03-26T18:59:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.542+0000 7f2349418740 1 -- 172.21.15.97:0/1806403484 shutdown_connections 2024-03-26T18:59:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.542+0000 7f2349418740 1 --2- 172.21.15.97:0/1806403484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f747342900 0x55f74733e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.542+0000 7f2349418740 1 --2- 172.21.15.97:0/1806403484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f74733d790 0x55f74733e020 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.542+0000 7f2349418740 1 --2- 172.21.15.97:0/1806403484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f74733cda0 0x55f747347060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.542+0000 7f2349418740 1 -- 172.21.15.97:0/1806403484 >> 172.21.15.97:0/1806403484 conn(0x55f747338310 msgr2=0x55f747346950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.544+0000 7f2349418740 1 -- 172.21.15.97:0/1806403484 shutdown_connections 2024-03-26T18:59:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.544+0000 7f2349418740 1 -- 172.21.15.97:0/1806403484 wait complete. 2024-03-26T18:59:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.544+0000 7f2349418740 1 -- start start 2024-03-26T18:59:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.544+0000 7f2349418740 1 Processor -- start 2024-03-26T18:59:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f2349418740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733d790 0x55f74734b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f2349418740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f747342900 0x55f74734be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f2349418740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747352e10 0x55f74734ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f2349418740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23401a7f50 con 0x55f74733d790 2024-03-26T18:59:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f2349418740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23401a7880 con 0x55f747352e10 2024-03-26T18:59:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f2349418740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23401a4b00 con 0x55f747342900 2024-03-26T18:59:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f2349418740 10 client.0 _ll_register_callbacks cb 0x7f2330097940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:59:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f233f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733d790 0x55f74734b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f233fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747352e10 0x55f74734ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f233fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747352e10 0x55f74734ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:33798/0 (socket says 172.21.15.97:33798) 2024-03-26T18:59:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f233fe00640 1 -- 172.21.15.97:0/1640728924 learned_addr learned my addr 172.21.15.97:0/1640728924 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.545+0000 7f2337e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f747342900 0x55f74734be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.546+0000 7f233fe00640 1 -- 172.21.15.97:0/1640728924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f747342900 msgr2=0x55f74734be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.546+0000 7f233fe00640 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f747342900 0x55f74734be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.546+0000 7f233fe00640 1 -- 172.21.15.97:0/1640728924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733d790 msgr2=0x55f74734b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.546+0000 7f233fe00640 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733d790 0x55f74734b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.546+0000 7f233fe00640 1 -- 172.21.15.97:0/1640728924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74734d450 con 0x55f747352e10 2024-03-26T18:59:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.546+0000 7f233fe00640 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747352e10 0x55f74734ce60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2338009810 tx=0x7f2338004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69041a58f4acdcb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.546+0000 7f232d600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:59:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.547+0000 7f232d600640 1 -- 172.21.15.97:0/1640728924 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338004d20 con 0x55f747352e10 2024-03-26T18:59:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.547+0000 7f2349418740 1 -- 172.21.15.97:0/1640728924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2328058040 con 0x55f747352e10 2024-03-26T18:59:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.547+0000 7f232d600640 1 -- 172.21.15.97:0/1640728924 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f233800f8d0 con 0x55f747352e10 2024-03-26T18:59:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.547+0000 7f232d600640 1 -- 172.21.15.97:0/1640728924 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f233800fad0 con 0x55f747352e10 2024-03-26T18:59:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.547+0000 7f2349418740 10 client.25322 fetch_fsmap learned FSMap version 27 2024-03-26T18:59:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.547+0000 7f2349418740 1 -- 172.21.15.97:0/1640728924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f7473b9510 con 0x55f747352e10 2024-03-26T18:59:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.547+0000 7f232d600640 1 -- 172.21.15.97:0/1640728924 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2328058040 con 0x55f747352e10 2024-03-26T18:59:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.547+0000 7f232d600640 1 -- 172.21.15.97:0/1640728924 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2338025070 con 0x55f747352e10 2024-03-26T18:59:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.548+0000 7f2349418740 10 client.25322 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T18:59:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.548+0000 7f2349418740 10 client.25322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:59:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.548+0000 7f2349418740 20 client.25322 populate_metadata read hostname 'smithi097' 2024-03-26T18:59:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.548+0000 7f2349418740 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f7473b9f10 0x55f7473bc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.548+0000 7f2349418740 4 client.25322 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:59:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.548+0000 7f2349418740 1 -- 172.21.15.97:0/1640728924 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7472a9710 con 0x55f7473b9f10 2024-03-26T18:59:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.549+0000 7f233f400640 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f7473b9f10 0x55f7473bc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.549+0000 7f232d600640 10 client.25322 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.549+0000 7f233f400640 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f7473b9f10 0x55f7473bc330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=284a890949abe43a server_cookie=1e6c5813bc6412a4 in_seq=0 out_seq=0 2024-03-26T18:59:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.641+0000 7f232d600640 1 -- 172.21.15.97:0/1640728924 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7472a9710 con 0x55f7473b9f10 2024-03-26T18:59:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.641+0000 7f232d600640 10 client.25322 handle_command_reply: tid=0 2024-03-26T18:59:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.641+0000 7f2349418740 1 client.25322 shutdown 2024-03-26T18:59:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 1 -- 172.21.15.97:0/1640728924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747352e10 msgr2=0x55f74734ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747352e10 0x55f74734ce60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2338009810 tx=0x7f2338004a40 comp rx=0 tx=0).stop 2024-03-26T18:59:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 1 -- 172.21.15.97:0/1640728924 shutdown_connections 2024-03-26T18:59:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f7473b9f10 0x55f7473bc330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f747352e10 0x55f74734ce60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f747342900 0x55f74734be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 1 --2- 172.21.15.97:0/1640728924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f74733d790 0x55f74734b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f232d600640 0 client.25322 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 1 -- 172.21.15.97:0/1640728924 >> 172.21.15.97:0/1640728924 conn(0x55f747338310 msgr2=0x55f747336cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 1 -- 172.21.15.97:0/1640728924 shutdown_connections 2024-03-26T18:59:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 1 -- 172.21.15.97:0/1640728924 wait complete. 2024-03-26T18:59:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.642+0000 7f2349418740 20 client.25322 trim_cache size 0 max 16384 2024-03-26T18:59:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T18:59:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T18:59:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T18:59:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 -- 172.21.15.97:0/1505783593 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f23240a30e0 msgr2=0x7f23240a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f23240a30e0 0x7f23240a55a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55f74733dff0 tx=0x7f2338009500 comp rx=0 tx=0).stop 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 -- 172.21.15.97:0/1505783593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2340100800 msgr2=0x7f2340104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2340100800 0x7f2340104740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f233000e9f0 tx=0x7f233000eeb0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 -- 172.21.15.97:0/1505783593 shutdown_connections 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f23240a30e0 0x7f23240a55a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23401023b0 0x7f2340103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23401019f0 0x7f2340102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 --2- 172.21.15.97:0/1505783593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2340100800 0x7f2340104740 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 -- 172.21.15.97:0/1505783593 >> 172.21.15.97:0/1505783593 conn(0x7f23400fbd70 msgr2=0x7f23400fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 -- 172.21.15.97:0/1505783593 shutdown_connections 2024-03-26T18:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:23.643+0000 7f2346600640 1 -- 172.21.15.97:0/1505783593 wait complete. 2024-03-26T18:59:24.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:24 smithi097 ceph-mon[31955]: pgmap v404: 97 pgs: 97 active+clean; 332 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 50 MiB/s wr, 287 op/s 2024-03-26T18:59:24.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:24 smithi144 ceph-mon[36053]: pgmap v404: 97 pgs: 97 active+clean; 332 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 50 MiB/s wr, 287 op/s 2024-03-26T18:59:24.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:24 smithi105 ceph-mon[36270]: pgmap v404: 97 pgs: 97 active+clean; 332 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 50 MiB/s wr, 287 op/s 2024-03-26T18:59:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:done in 12.66 s (drop tables 0.00 s, create tables 0.02 s, client-side generate 8.68 s, vacuum 1.19 s, primary keys 2.78 s). 2024-03-26T18:59:24.869 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-03-26T18:59:25.023 INFO:teuthology.orchestra.run.smithi097.stderr:starting vacuum...end. 2024-03-26T18:59:26.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:25 smithi144 ceph-mon[36053]: pgmap v405: 97 pgs: 97 active+clean; 385 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 58 MiB/s wr, 247 op/s 2024-03-26T18:59:26.210 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:25 smithi105 ceph-mon[36270]: pgmap v405: 97 pgs: 97 active+clean; 385 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 58 MiB/s wr, 247 op/s 2024-03-26T18:59:26.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:25 smithi097 ceph-mon[31955]: pgmap v405: 97 pgs: 97 active+clean; 385 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 58 MiB/s wr, 247 op/s 2024-03-26T18:59:28.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:28 smithi097 ceph-mon[31955]: pgmap v406: 97 pgs: 97 active+clean; 401 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 56 MiB/s wr, 137 op/s 2024-03-26T18:59:28.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:28 smithi144 ceph-mon[36053]: pgmap v406: 97 pgs: 97 active+clean; 401 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 56 MiB/s wr, 137 op/s 2024-03-26T18:59:28.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:28 smithi105 ceph-mon[36270]: pgmap v406: 97 pgs: 97 active+clean; 401 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 56 MiB/s wr, 137 op/s 2024-03-26T18:59:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 5.0 s, 2690.3 tps, lat 35.416 ms stddev 41.203 2024-03-26T18:59:30.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:30 smithi097 ceph-mon[31955]: pgmap v407: 97 pgs: 97 active+clean; 477 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 64 MiB/s wr, 106 op/s 2024-03-26T18:59:30.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:30 smithi144 ceph-mon[36053]: pgmap v407: 97 pgs: 97 active+clean; 477 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 64 MiB/s wr, 106 op/s 2024-03-26T18:59:30.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:30 smithi105 ceph-mon[36270]: pgmap v407: 97 pgs: 97 active+clean; 477 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 64 MiB/s wr, 106 op/s 2024-03-26T18:59:32.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:32 smithi097 ceph-mon[31955]: pgmap v408: 97 pgs: 97 active+clean; 578 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 MiB/s wr, 168 op/s 2024-03-26T18:59:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:32 smithi144 ceph-mon[36053]: pgmap v408: 97 pgs: 97 active+clean; 578 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 MiB/s wr, 168 op/s 2024-03-26T18:59:32.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:32 smithi105 ceph-mon[36270]: pgmap v408: 97 pgs: 97 active+clean; 578 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 MiB/s wr, 168 op/s 2024-03-26T18:59:34.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:34 smithi097 ceph-mon[31955]: pgmap v409: 97 pgs: 97 active+clean; 677 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 37 MiB/s wr, 195 op/s 2024-03-26T18:59:34.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:34 smithi144 ceph-mon[36053]: pgmap v409: 97 pgs: 97 active+clean; 677 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 37 MiB/s wr, 195 op/s 2024-03-26T18:59:34.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:34 smithi105 ceph-mon[36270]: pgmap v409: 97 pgs: 97 active+clean; 677 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 37 MiB/s wr, 195 op/s 2024-03-26T18:59:35.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 10.0 s, 2631.1 tps, lat 37.979 ms stddev 40.580 2024-03-26T18:59:36.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:36 smithi097 ceph-mon[31955]: pgmap v410: 97 pgs: 97 active+clean; 886 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 MiB/s wr, 396 op/s 2024-03-26T18:59:36.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:36 smithi144 ceph-mon[36053]: pgmap v410: 97 pgs: 97 active+clean; 886 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 MiB/s wr, 396 op/s 2024-03-26T18:59:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:36 smithi105 ceph-mon[36270]: pgmap v410: 97 pgs: 97 active+clean; 886 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 MiB/s wr, 396 op/s 2024-03-26T18:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.825+0000 7ff880600640 1 -- 172.21.15.97:0/37589535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff878109230 msgr2=0x7ff8781096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.825+0000 7ff880600640 1 --2- 172.21.15.97:0/37589535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff878109230 0x7ff8781096c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff870009ca0 tx=0x7ff87002c100 comp rx=0 tx=0).stop 2024-03-26T18:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.826+0000 7ff880600640 1 -- 172.21.15.97:0/37589535 shutdown_connections 2024-03-26T18:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.826+0000 7ff880600640 1 --2- 172.21.15.97:0/37589535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff878109230 0x7ff8781096c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.826+0000 7ff880600640 1 --2- 172.21.15.97:0/37589535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff87810c4e0 0x7ff87810e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.826+0000 7ff880600640 1 --2- 172.21.15.97:0/37589535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff87806a6c0 0x7ff878108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.826+0000 7ff880600640 1 -- 172.21.15.97:0/37589535 >> 172.21.15.97:0/37589535 conn(0x7ff87806eac0 msgr2=0x7ff878070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.826+0000 7ff880600640 1 -- 172.21.15.97:0/37589535 shutdown_connections 2024-03-26T18:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.826+0000 7ff880600640 1 -- 172.21.15.97:0/37589535 wait complete. 2024-03-26T18:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.827+0000 7ff880600640 1 Processor -- start 2024-03-26T18:59:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.828+0000 7ff880600640 1 -- start start 2024-03-26T18:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.828+0000 7ff880600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff87806a6c0 0x7ff8781a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.828+0000 7ff880600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff878109230 0x7ff8781a6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.828+0000 7ff880600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff87810c4e0 0x7ff8781adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.828+0000 7ff880600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff878113bd0 con 0x7ff87810c4e0 2024-03-26T18:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.828+0000 7ff880600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff878113a50 con 0x7ff878109230 2024-03-26T18:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.828+0000 7ff880600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff878113d50 con 0x7ff87806a6c0 2024-03-26T18:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.829+0000 7ff87d200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff878109230 0x7ff8781a6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.829+0000 7ff87d200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff878109230 0x7ff8781a6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57408/0 (socket says 172.21.15.97:57408) 2024-03-26T18:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.829+0000 7ff87d200640 1 -- 172.21.15.97:0/4185942043 learned_addr learned my addr 172.21.15.97:0/4185942043 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.829+0000 7ff87e600640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff87810c4e0 0x7ff8781adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.829+0000 7ff87dc00640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff87806a6c0 0x7ff8781a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.830+0000 7ff87d200640 1 -- 172.21.15.97:0/4185942043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff87806a6c0 msgr2=0x7ff8781a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.830+0000 7ff87d200640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff87806a6c0 0x7ff8781a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.830+0000 7ff87d200640 1 -- 172.21.15.97:0/4185942043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff87810c4e0 msgr2=0x7ff8781adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.830+0000 7ff87d200640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff87810c4e0 0x7ff8781adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.830+0000 7ff87d200640 1 -- 172.21.15.97:0/4185942043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8781ae590 con 0x7ff878109230 2024-03-26T18:59:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.831+0000 7ff87d200640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff878109230 0x7ff8781a6eb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff86400b450 tx=0x7ff86400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.832+0000 7ff876000640 1 -- 172.21.15.97:0/4185942043 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864015020 con 0x7ff878109230 2024-03-26T18:59:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.832+0000 7ff876000640 1 -- 172.21.15.97:0/4185942043 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff864011d70 con 0x7ff878109230 2024-03-26T18:59:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.832+0000 7ff880600640 1 -- 172.21.15.97:0/4185942043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8781ae880 con 0x7ff878109230 2024-03-26T18:59:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.832+0000 7ff876000640 1 -- 172.21.15.97:0/4185942043 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86400f340 con 0x7ff878109230 2024-03-26T18:59:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.833+0000 7ff880600640 1 -- 172.21.15.97:0/4185942043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8781aee50 con 0x7ff878109230 2024-03-26T18:59:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.838+0000 7ff8833b8740 1 Processor -- start 2024-03-26T18:59:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.839+0000 7ff8833b8740 1 -- start start 2024-03-26T18:59:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.839+0000 7ff876000640 1 -- 172.21.15.97:0/4185942043 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137905+0+0 (secure 0 0 0) 0x7ff864018020 con 0x7ff878109230 2024-03-26T18:59:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.840+0000 7ff8833b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a55111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.840+0000 7ff87e600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a55111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.840+0000 7ff87e600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a55111c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56032/0 (socket says 172.21.15.97:56032) 2024-03-26T18:59:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.840+0000 7ff87e600640 1 -- 172.21.15.97:0/2809474531 learned_addr learned my addr 172.21.15.97:0/2809474531 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.840+0000 7ff8833b8740 1 --2- 172.21.15.97:0/2809474531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5511790 0x5605a5516290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.840+0000 7ff8833b8740 1 --2- 172.21.15.97:0/2809474531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5516900 0x5605a5518d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.841+0000 7ff8833b8740 1 -- 172.21.15.97:0/2809474531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5605a5475e80 con 0x5605a5516900 2024-03-26T18:59:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.841+0000 7ff8833b8740 1 -- 172.21.15.97:0/2809474531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5605a541e660 con 0x5605a5511790 2024-03-26T18:59:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.841+0000 7ff8833b8740 1 -- 172.21.15.97:0/2809474531 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5605a5471e30 con 0x5605a5510da0 2024-03-26T18:59:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.841+0000 7ff87e600640 1 -- 172.21.15.97:0/2809474531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5511790 msgr2=0x5605a5516290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:59:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.841+0000 7ff87e600640 1 --2- 172.21.15.97:0/2809474531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5511790 0x5605a5516290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.841+0000 7ff87e600640 1 -- 172.21.15.97:0/2809474531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5516900 msgr2=0x5605a5518d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:59:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.841+0000 7ff87e600640 1 --2- 172.21.15.97:0/2809474531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5516900 0x5605a5518d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.841+0000 7ff87e600640 1 -- 172.21.15.97:0/2809474531 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a518a210 con 0x5605a5510da0 2024-03-26T18:59:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.842+0000 7ff876000640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8600a3220 0x7ff8600a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.842+0000 7ff87e600640 1 --2- 172.21.15.97:0/2809474531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a55111c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff87002c710 tx=0x7ff8700323e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=461110a87c7658ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.842+0000 7ff86aa00640 1 -- 172.21.15.97:0/2809474531 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870043070 con 0x5605a5510da0 2024-03-26T18:59:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.842+0000 7ff86aa00640 1 -- 172.21.15.97:0/2809474531 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff870003970 con 0x5605a5510da0 2024-03-26T18:59:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.842+0000 7ff87dc00640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8600a3220 0x7ff8600a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.843+0000 7ff876000640 1 -- 172.21.15.97:0/4185942043 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff8640ced50 con 0x7ff878109230 2024-03-26T18:59:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.843+0000 7ff8833b8740 1 -- 172.21.15.97:0/2809474531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 msgr2=0x5605a55111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.843+0000 7ff87dc00640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8600a3220 0x7ff8600a56e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5605a5511b90 tx=0x7ff86c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.843+0000 7ff8833b8740 1 --2- 172.21.15.97:0/2809474531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a55111c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff87002c710 tx=0x7ff8700323e0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.844+0000 7ff8833b8740 1 -- 172.21.15.97:0/2809474531 shutdown_connections 2024-03-26T18:59:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.844+0000 7ff8833b8740 1 --2- 172.21.15.97:0/2809474531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5516900 0x5605a5518d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.844+0000 7ff8833b8740 1 --2- 172.21.15.97:0/2809474531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5511790 0x5605a5516290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.844+0000 7ff8833b8740 1 --2- 172.21.15.97:0/2809474531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a55111c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.844+0000 7ff8833b8740 1 -- 172.21.15.97:0/2809474531 >> 172.21.15.97:0/2809474531 conn(0x5605a550c310 msgr2=0x5605a550c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.844+0000 7ff8833b8740 1 -- 172.21.15.97:0/2809474531 shutdown_connections 2024-03-26T18:59:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.844+0000 7ff8833b8740 1 -- 172.21.15.97:0/2809474531 wait complete. 2024-03-26T18:59:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.845+0000 7ff8833b8740 1 -- start start 2024-03-26T18:59:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.845+0000 7ff8833b8740 1 Processor -- start 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.846+0000 7ff8833b8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5510da0 0x5605a551adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.846+0000 7ff8833b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5511790 0x5605a551b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.846+0000 7ff8833b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5516900 0x5605a5522330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.846+0000 7ff87dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5510da0 0x5605a551adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.846+0000 7ff87dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5510da0 0x5605a551adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46138/0 (socket says 172.21.15.97:46138) 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.846+0000 7ff87e600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5516900 0x5605a5522330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff87e600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5516900 0x5605a5522330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56034/0 (socket says 172.21.15.97:56034) 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff8833b8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5605a55198f0 con 0x5605a5510da0 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff8833b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5605a5431ac0 con 0x5605a5511790 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff8833b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5605a5519a70 con 0x5605a5516900 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff87dc00640 1 -- 172.21.15.97:0/1945099672 learned_addr learned my addr 172.21.15.97:0/1945099672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff87d200640 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5511790 0x5605a551b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff87e600640 1 -- 172.21.15.97:0/1945099672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5511790 msgr2=0x5605a551b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff87e600640 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5511790 0x5605a551b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff87e600640 1 -- 172.21.15.97:0/1945099672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5510da0 msgr2=0x5605a551adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff8833b8740 10 client.0 _ll_register_callbacks cb 0x5605a5431490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:59:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff87e600640 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5510da0 0x5605a551adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.847+0000 7ff87e600640 1 -- 172.21.15.97:0/1945099672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a5256620 con 0x5605a5516900 2024-03-26T18:59:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.848+0000 7ff87dc00640 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5510da0 0x5605a551adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:59:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.848+0000 7ff8833b8740 1 -- 172.21.15.97:0/1945099672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff870003970 con 0x5605a5516900 2024-03-26T18:59:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.848+0000 7ff87e600640 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5516900 0x5605a5522330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff870009ca0 tx=0x7ff870004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd99ded60b8603c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.848+0000 7ff85f400640 10 client.25303 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.848+0000 7ff85f400640 1 -- 172.21.15.97:0/1945099672 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87004f020 con 0x5605a5516900 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.848+0000 7ff85f400640 1 -- 172.21.15.97:0/1945099672 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff870004530 con 0x5605a5516900 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.849+0000 7ff85f400640 1 -- 172.21.15.97:0/1945099672 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff870039040 con 0x5605a5516900 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.849+0000 7ff8833b8740 10 client.25303 fetch_fsmap learned FSMap version 27 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.849+0000 7ff8833b8740 1 -- 172.21.15.97:0/1945099672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5605a5519920 con 0x5605a5516900 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.849+0000 7ff85f400640 1 -- 172.21.15.97:0/1945099672 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff870048450 con 0x5605a5516900 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.849+0000 7ff8833b8740 10 client.25303 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.849+0000 7ff8833b8740 10 client.25303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.850+0000 7ff8833b8740 20 client.25303 populate_metadata read hostname 'smithi097' 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.850+0000 7ff8833b8740 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5605a55266e0 0x5605a5528ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.850+0000 7ff8833b8740 4 client.25303 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:59:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.850+0000 7ff8833b8740 1 -- 172.21.15.97:0/1945099672 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605a54833c0 con 0x5605a55266e0 2024-03-26T18:59:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.850+0000 7ff87dc00640 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5605a55266e0 0x5605a5528ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.851+0000 7ff87dc00640 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5605a55266e0 0x5605a5528ad0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752aa2ea2fb2d7b7 server_cookie=92a8d9664c10ad1c in_seq=0 out_seq=0 2024-03-26T18:59:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.851+0000 7ff85f400640 10 client.25303 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.852+0000 7ff85f400640 1 -- 172.21.15.97:0/1945099672 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5605a54833c0 con 0x5605a55266e0 2024-03-26T18:59:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.852+0000 7ff85f400640 10 client.25303 handle_command_reply: tid=0 2024-03-26T18:59:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.852+0000 7ff8833b8740 1 client.25303 shutdown 2024-03-26T18:59:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.856+0000 7ff8833b8740 1 -- 172.21.15.97:0/1945099672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5516900 msgr2=0x5605a5522330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.856+0000 7ff8833b8740 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5516900 0x5605a5522330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff870009ca0 tx=0x7ff870004a40 comp rx=0 tx=0).stop 2024-03-26T18:59:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.856+0000 7ff8833b8740 1 -- 172.21.15.97:0/1945099672 shutdown_connections 2024-03-26T18:59:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.856+0000 7ff8833b8740 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5605a55266e0 0x5605a5528ad0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.856+0000 7ff8833b8740 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5516900 0x5605a5522330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.856+0000 7ff85f400640 0 client.25303 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.856+0000 7ff8833b8740 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5511790 0x5605a551b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.856+0000 7ff8833b8740 1 --2- 172.21.15.97:0/1945099672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5510da0 0x5605a551adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.856+0000 7ff8833b8740 1 -- 172.21.15.97:0/1945099672 >> 172.21.15.97:0/1945099672 conn(0x5605a550c310 msgr2=0x5605a550b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.858+0000 7ff8833b8740 1 -- 172.21.15.97:0/1945099672 shutdown_connections 2024-03-26T18:59:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.858+0000 7ff8833b8740 1 -- 172.21.15.97:0/1945099672 wait complete. 2024-03-26T18:59:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.859+0000 7ff8833b8740 20 client.25303 trim_cache size 0 max 16384 2024-03-26T18:59:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.902+0000 7ff8833b8740 1 Processor -- start 2024-03-26T18:59:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff8833b8740 1 -- start start 2024-03-26T18:59:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff8833b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a5512020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff8833b8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 0x5605a55222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff8833b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5512560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff8833b8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781b7990 con 0x5605a5511790 2024-03-26T18:59:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff8833b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781b94d0 con 0x5605a5516900 2024-03-26T18:59:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff8833b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781b8060 con 0x5605a5510da0 2024-03-26T18:59:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff87dc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a5512020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff87e600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5512560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff87e600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5512560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57434/0 (socket says 172.21.15.97:57434) 2024-03-26T18:59:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff87e600640 1 -- 172.21.15.97:0/2757401561 learned_addr learned my addr 172.21.15.97:0/2757401561 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff87dc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a5512020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56048/0 (socket says 172.21.15.97:56048) 2024-03-26T18:59:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.903+0000 7ff87d200640 1 --2- 172.21.15.97:0/2757401561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 0x5605a55222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.904+0000 7ff87e600640 1 -- 172.21.15.97:0/2757401561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 msgr2=0x5605a5512020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.904+0000 7ff87e600640 1 --2- 172.21.15.97:0/2757401561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a5512020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.904+0000 7ff87e600640 1 -- 172.21.15.97:0/2757401561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 msgr2=0x5605a55222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.904+0000 7ff87e600640 1 --2- 172.21.15.97:0/2757401561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 0x5605a55222f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.904+0000 7ff87e600640 1 -- 172.21.15.97:0/2757401561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a550d080 con 0x5605a5516900 2024-03-26T18:59:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.905+0000 7ff87e600640 1 --2- 172.21.15.97:0/2757401561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5512560 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff870005e30 tx=0x7ff87004b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8acb14820a5d105 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.905+0000 7ff86a000640 1 -- 172.21.15.97:0/2757401561 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870061070 con 0x5605a5516900 2024-03-26T18:59:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.905+0000 7ff86a000640 1 -- 172.21.15.97:0/2757401561 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff870003970 con 0x5605a5516900 2024-03-26T18:59:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.905+0000 7ff86a000640 1 -- 172.21.15.97:0/2757401561 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870061070 con 0x5605a5516900 2024-03-26T18:59:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.906+0000 7ff8833b8740 1 -- 172.21.15.97:0/2757401561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 msgr2=0x5605a5512560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.906+0000 7ff8833b8740 1 --2- 172.21.15.97:0/2757401561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5512560 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff870005e30 tx=0x7ff87004b000 comp rx=0 tx=0).stop 2024-03-26T18:59:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.907+0000 7ff8833b8740 1 -- 172.21.15.97:0/2757401561 shutdown_connections 2024-03-26T18:59:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.907+0000 7ff8833b8740 1 --2- 172.21.15.97:0/2757401561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5512560 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.907+0000 7ff8833b8740 1 --2- 172.21.15.97:0/2757401561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 0x5605a55222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.907+0000 7ff8833b8740 1 --2- 172.21.15.97:0/2757401561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a5512020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.907+0000 7ff8833b8740 1 -- 172.21.15.97:0/2757401561 >> 172.21.15.97:0/2757401561 conn(0x5605a550c310 msgr2=0x5605a550fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.907+0000 7ff8833b8740 1 -- 172.21.15.97:0/2757401561 shutdown_connections 2024-03-26T18:59:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.907+0000 7ff8833b8740 1 -- 172.21.15.97:0/2757401561 wait complete. 2024-03-26T18:59:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.907+0000 7ff8833b8740 1 -- start start 2024-03-26T18:59:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.908+0000 7ff8833b8740 1 Processor -- start 2024-03-26T18:59:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.909+0000 7ff8833b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a5521f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.909+0000 7ff8833b8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 0x5605a55205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.909+0000 7ff8833b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5520b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.910+0000 7ff87d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 0x5605a55205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.910+0000 7ff87d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 0x5605a55205d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46154/0 (socket says 172.21.15.97:46154) 2024-03-26T18:59:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.910+0000 7ff8833b8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781b8e00 con 0x5605a5511790 2024-03-26T18:59:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.910+0000 7ff8833b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781b8730 con 0x5605a5516900 2024-03-26T18:59:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.910+0000 7ff8833b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781b59c0 con 0x5605a5510da0 2024-03-26T18:59:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.910+0000 7ff87d200640 1 -- 172.21.15.97:0/3454944762 learned_addr learned my addr 172.21.15.97:0/3454944762 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.910+0000 7ff8833b8740 10 client.0 _ll_register_callbacks cb 0x7ff870032690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:59:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.910+0000 7ff87e600640 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5520b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.910+0000 7ff87dc00640 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a5521f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.911+0000 7ff87e600640 1 -- 172.21.15.97:0/3454944762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 msgr2=0x5605a5521f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.911+0000 7ff87e600640 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a5521f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.911+0000 7ff87e600640 1 -- 172.21.15.97:0/3454944762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 msgr2=0x5605a55205d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.911+0000 7ff87e600640 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 0x5605a55205d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.911+0000 7ff87e600640 1 -- 172.21.15.97:0/3454944762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a5519e90 con 0x5605a5516900 2024-03-26T18:59:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.911+0000 7ff87e600640 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5520b10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff870000f80 tx=0x7ff870032590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81116ea3baa9f5b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.911+0000 7ff85f400640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:59:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.911+0000 7ff85f400640 1 -- 172.21.15.97:0/3454944762 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870061070 con 0x5605a5516900 2024-03-26T18:59:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.912+0000 7ff85f400640 1 -- 172.21.15.97:0/3454944762 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff870034640 con 0x5605a5516900 2024-03-26T18:59:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.912+0000 7ff8833b8740 1 -- 172.21.15.97:0/3454944762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff870003970 con 0x5605a5516900 2024-03-26T18:59:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.912+0000 7ff85f400640 1 -- 172.21.15.97:0/3454944762 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff87006b050 con 0x5605a5516900 2024-03-26T18:59:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.912+0000 7ff8833b8740 10 client.25337 fetch_fsmap learned FSMap version 27 2024-03-26T18:59:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.912+0000 7ff8833b8740 1 -- 172.21.15.97:0/3454944762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605a55109e0 con 0x5605a5516900 2024-03-26T18:59:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.913+0000 7ff85f400640 1 -- 172.21.15.97:0/3454944762 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff870003970 con 0x5605a5516900 2024-03-26T18:59:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.913+0000 7ff85f400640 1 -- 172.21.15.97:0/3454944762 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff8700664c0 con 0x5605a5516900 2024-03-26T18:59:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.914+0000 7ff8833b8740 10 client.25337 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T18:59:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.914+0000 7ff8833b8740 10 client.25337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:59:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.914+0000 7ff8833b8740 20 client.25337 populate_metadata read hostname 'smithi097' 2024-03-26T18:59:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.914+0000 7ff8833b8740 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5605a558db50 0x5605a558ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.914+0000 7ff8833b8740 4 client.25337 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:59:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.914+0000 7ff8833b8740 1 -- 172.21.15.97:0/3454944762 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605a547d710 con 0x5605a558db50 2024-03-26T18:59:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.914+0000 7ff87dc00640 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5605a558db50 0x5605a558ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.915+0000 7ff85f400640 10 client.25337 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.916+0000 7ff87dc00640 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5605a558db50 0x5605a558ff40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af42b509cbef33db server_cookie=1cca995bff0ac7c7 in_seq=0 out_seq=0 2024-03-26T18:59:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.959+0000 7ff85f400640 1 -- 172.21.15.97:0/3454944762 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605a547d710 con 0x5605a558db50 2024-03-26T18:59:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.959+0000 7ff85f400640 10 client.25337 handle_command_reply: tid=0 2024-03-26T18:59:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.960+0000 7ff8833b8740 1 client.25337 shutdown 2024-03-26T18:59:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.961+0000 7ff8833b8740 1 -- 172.21.15.97:0/3454944762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 msgr2=0x5605a5520b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.961+0000 7ff8833b8740 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5520b10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff870000f80 tx=0x7ff870032590 comp rx=0 tx=0).stop 2024-03-26T18:59:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.961+0000 7ff8833b8740 1 -- 172.21.15.97:0/3454944762 shutdown_connections 2024-03-26T18:59:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.961+0000 7ff8833b8740 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5605a558db50 0x5605a558ff40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.961+0000 7ff8833b8740 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5605a5516900 0x5605a5520b10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.961+0000 7ff8833b8740 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5511790 0x5605a55205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.961+0000 7ff8833b8740 1 --2- 172.21.15.97:0/3454944762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5605a5510da0 0x5605a5521f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.961+0000 7ff8833b8740 1 -- 172.21.15.97:0/3454944762 >> 172.21.15.97:0/3454944762 conn(0x5605a550c310 msgr2=0x5605a550acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.961+0000 7ff85f400640 0 client.25337 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.961+0000 7ff8833b8740 1 -- 172.21.15.97:0/3454944762 shutdown_connections 2024-03-26T18:59:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.962+0000 7ff8833b8740 1 -- 172.21.15.97:0/3454944762 wait complete. 2024-03-26T18:59:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.962+0000 7ff8833b8740 20 client.25337 trim_cache size 0 max 16384 2024-03-26T18:59:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T18:59:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T18:59:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T18:59:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T18:59:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 -- 172.21.15.97:0/4185942043 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8600a3220 msgr2=0x7ff8600a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8600a3220 0x7ff8600a56e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5605a5511b90 tx=0x7ff86c00b390 comp rx=0 tx=0).stop 2024-03-26T18:59:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 -- 172.21.15.97:0/4185942043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff878109230 msgr2=0x7ff8781a6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff878109230 0x7ff8781a6eb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff86400b450 tx=0x7ff86400b910 comp rx=0 tx=0).stop 2024-03-26T18:59:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 -- 172.21.15.97:0/4185942043 shutdown_connections 2024-03-26T18:59:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff8600a3220 0x7ff8600a56e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff87810c4e0 0x7ff8781adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff878109230 0x7ff8781a6eb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 --2- 172.21.15.97:0/4185942043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff87806a6c0 0x7ff8781a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 -- 172.21.15.97:0/4185942043 >> 172.21.15.97:0/4185942043 conn(0x7ff87806eac0 msgr2=0x7ff878111300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 -- 172.21.15.97:0/4185942043 shutdown_connections 2024-03-26T18:59:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:36.963+0000 7ff880600640 1 -- 172.21.15.97:0/4185942043 wait complete. 2024-03-26T18:59:38.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:38 smithi097 ceph-mon[31955]: pgmap v411: 97 pgs: 97 active+clean; 894 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 400 op/s 2024-03-26T18:59:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:38 smithi144 ceph-mon[36053]: pgmap v411: 97 pgs: 97 active+clean; 894 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 400 op/s 2024-03-26T18:59:38.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:38 smithi105 ceph-mon[36270]: pgmap v411: 97 pgs: 97 active+clean; 894 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 400 op/s 2024-03-26T18:59:40.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 15.0 s, 2820.0 tps, lat 35.467 ms stddev 38.175 2024-03-26T18:59:40.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:40 smithi097 ceph-mon[31955]: pgmap v412: 97 pgs: 97 active+clean; 919 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 563 op/s 2024-03-26T18:59:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:40 smithi144 ceph-mon[36053]: pgmap v412: 97 pgs: 97 active+clean; 919 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 563 op/s 2024-03-26T18:59:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:40 smithi105 ceph-mon[36270]: pgmap v412: 97 pgs: 97 active+clean; 919 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 563 op/s 2024-03-26T18:59:42.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:42 smithi097 ceph-mon[31955]: pgmap v413: 97 pgs: 97 active+clean; 926 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 608 op/s 2024-03-26T18:59:42.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:42 smithi144 ceph-mon[36053]: pgmap v413: 97 pgs: 97 active+clean; 926 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 608 op/s 2024-03-26T18:59:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:42 smithi105 ceph-mon[36270]: pgmap v413: 97 pgs: 97 active+clean; 926 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 608 op/s 2024-03-26T18:59:44.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:44 smithi097 ceph-mon[31955]: pgmap v414: 97 pgs: 97 active+clean; 929 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 643 op/s 2024-03-26T18:59:44.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:44 smithi144 ceph-mon[36053]: pgmap v414: 97 pgs: 97 active+clean; 929 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 643 op/s 2024-03-26T18:59:44.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:44 smithi105 ceph-mon[36270]: pgmap v414: 97 pgs: 97 active+clean; 929 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 643 op/s 2024-03-26T18:59:45.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 20.0 s, 2884.8 tps, lat 34.591 ms stddev 37.952 2024-03-26T18:59:46.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:46 smithi097 ceph-mon[31955]: pgmap v415: 97 pgs: 97 active+clean; 933 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 779 op/s 2024-03-26T18:59:46.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:46 smithi144 ceph-mon[36053]: pgmap v415: 97 pgs: 97 active+clean; 933 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 779 op/s 2024-03-26T18:59:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:46 smithi105 ceph-mon[36270]: pgmap v415: 97 pgs: 97 active+clean; 933 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 779 op/s 2024-03-26T18:59:48.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:48 smithi097 ceph-mon[31955]: pgmap v416: 97 pgs: 97 active+clean; 933 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 611 op/s 2024-03-26T18:59:48.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:48 smithi144 ceph-mon[36053]: pgmap v416: 97 pgs: 97 active+clean; 933 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 611 op/s 2024-03-26T18:59:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:48 smithi105 ceph-mon[36270]: pgmap v416: 97 pgs: 97 active+clean; 933 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 611 op/s 2024-03-26T18:59:50.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 25.0 s, 2450.6 tps, lat 40.745 ms stddev 43.642 2024-03-26T18:59:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.154+0000 7fd3b1c00640 1 -- 172.21.15.97:0/1001519689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac1073f0 msgr2=0x7fd3ac109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.154+0000 7fd3b1c00640 1 --2- 172.21.15.97:0/1001519689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac1073f0 0x7fd3ac109840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009ca0 tx=0x7fd3a402c120 comp rx=0 tx=0).stop 2024-03-26T18:59:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.155+0000 7fd3b1c00640 1 -- 172.21.15.97:0/1001519689 shutdown_connections 2024-03-26T18:59:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.155+0000 7fd3b1c00640 1 --2- 172.21.15.97:0/1001519689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac1073f0 0x7fd3ac109840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.155+0000 7fd3b1c00640 1 --2- 172.21.15.97:0/1001519689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac104ac0 0x7fd3ac106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.155+0000 7fd3b1c00640 1 --2- 172.21.15.97:0/1001519689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac102190 0x7fd3ac104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.155+0000 7fd3b1c00640 1 -- 172.21.15.97:0/1001519689 >> 172.21.15.97:0/1001519689 conn(0x7fd3ac0fbba0 msgr2=0x7fd3ac0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.155+0000 7fd3b1c00640 1 -- 172.21.15.97:0/1001519689 shutdown_connections 2024-03-26T18:59:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.156+0000 7fd3b1c00640 1 -- 172.21.15.97:0/1001519689 wait complete. 2024-03-26T18:59:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.157+0000 7fd3b1c00640 1 Processor -- start 2024-03-26T18:59:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.157+0000 7fd3b1c00640 1 -- start start 2024-03-26T18:59:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.159+0000 7fd3b1c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac102190 0x7fd3ac1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.159+0000 7fd3b1c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac104ac0 0x7fd3ac197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.160+0000 7fd3b1c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac1073f0 0x7fd3ac19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.160+0000 7fd3b1c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10d390 con 0x7fd3ac104ac0 2024-03-26T18:59:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.160+0000 7fd3b1c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10d210 con 0x7fd3ac102190 2024-03-26T18:59:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.160+0000 7fd3b1c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10d510 con 0x7fd3ac1073f0 2024-03-26T18:59:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.160+0000 7fd3b1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac1073f0 0x7fd3ac19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.160+0000 7fd3b1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac1073f0 0x7fd3ac19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34304/0 (socket says 172.21.15.97:34304) 2024-03-26T18:59:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.160+0000 7fd3b1200640 1 -- 172.21.15.97:0/3298339800 learned_addr learned my addr 172.21.15.97:0/3298339800 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.161+0000 7fd3abe00640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac102190 0x7fd3ac1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.161+0000 7fd3ab400640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac104ac0 0x7fd3ac197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.161+0000 7fd3b1200640 1 -- 172.21.15.97:0/3298339800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac102190 msgr2=0x7fd3ac1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.161+0000 7fd3b1200640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac102190 0x7fd3ac1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.161+0000 7fd3b1200640 1 -- 172.21.15.97:0/3298339800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac104ac0 msgr2=0x7fd3ac197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.161+0000 7fd3b1200640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac104ac0 0x7fd3ac197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.161+0000 7fd3b1200640 1 -- 172.21.15.97:0/3298339800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac19f5f0 con 0x7fd3ac1073f0 2024-03-26T18:59:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.161+0000 7fd3b1200640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac1073f0 0x7fd3ac19ef80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4037040 tx=0x7fd3a4034d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.163+0000 7fd3a8c00640 1 -- 172.21.15.97:0/3298339800 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4041020 con 0x7fd3ac1073f0 2024-03-26T18:59:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.163+0000 7fd3a8c00640 1 -- 172.21.15.97:0/3298339800 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a40328b0 con 0x7fd3ac1073f0 2024-03-26T18:59:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.163+0000 7fd3a8c00640 1 -- 172.21.15.97:0/3298339800 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4039040 con 0x7fd3ac1073f0 2024-03-26T18:59:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.163+0000 7fd3b1c00640 1 -- 172.21.15.97:0/3298339800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac0fff80 con 0x7fd3ac1073f0 2024-03-26T18:59:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.164+0000 7fd3b1c00640 1 -- 172.21.15.97:0/3298339800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac100420 con 0x7fd3ac1073f0 2024-03-26T18:59:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.172+0000 7fd3a8c00640 1 -- 172.21.15.97:0/3298339800 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137905+0+0 (secure 0 0 0) 0x7fd3a4044020 con 0x7fd3ac1073f0 2024-03-26T18:59:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.172+0000 7fd3b601b740 1 Processor -- start 2024-03-26T18:59:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.172+0000 7fd3b601b740 1 -- start start 2024-03-26T18:59:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.173+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5dff60 0x558e2c5e0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.173+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5e0950 0x558e2c5e5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.173+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.173+0000 7fd3b601b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558e2c544e80 con 0x558e2c5dff60 2024-03-26T18:59:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.173+0000 7fd3b601b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558e2c4ed660 con 0x558e2c5e0950 2024-03-26T18:59:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.173+0000 7fd3b601b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558e2c540e30 con 0x558e2c5e5ac0 2024-03-26T18:59:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.173+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5e0950 0x558e2c5e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.173+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5e0950 0x558e2c5e5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53888/0 (socket says 172.21.15.97:53888) 2024-03-26T18:59:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.173+0000 7fd3abe00640 1 -- 172.21.15.97:0/550522919 learned_addr learned my addr 172.21.15.97:0/550522919 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.174+0000 7fd3abe00640 1 -- 172.21.15.97:0/550522919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 msgr2=0x558e2c5e7f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:59:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.174+0000 7fd3abe00640 1 --2- 172.21.15.97:0/550522919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5e7f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.174+0000 7fd3abe00640 1 -- 172.21.15.97:0/550522919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5dff60 msgr2=0x558e2c5e0380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:59:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.174+0000 7fd3abe00640 1 --2- 172.21.15.97:0/550522919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5dff60 0x558e2c5e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.174+0000 7fd3abe00640 1 -- 172.21.15.97:0/550522919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e2c259210 con 0x558e2c5e0950 2024-03-26T18:59:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.175+0000 7fd3a8c00640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3900a7500 0x7fd3900a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.175+0000 7fd3abe00640 1 --2- 172.21.15.97:0/550522919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5e0950 0x558e2c5e5450 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009c80 tx=0x7fd39c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2c8ab69bdd8377 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.175+0000 7fd3ab400640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3900a7500 0x7fd3900a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.175+0000 7fd3a1600640 1 -- 172.21.15.97:0/550522919 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c012450 con 0x558e2c5e0950 2024-03-26T18:59:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.176+0000 7fd3a1600640 1 -- 172.21.15.97:0/550522919 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39c012ad0 con 0x558e2c5e0950 2024-03-26T18:59:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.176+0000 7fd3a1600640 1 -- 172.21.15.97:0/550522919 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c00fc00 con 0x558e2c5e0950 2024-03-26T18:59:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.176+0000 7fd3a8c00640 1 -- 172.21.15.97:0/3298339800 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd3a40f1600 con 0x7fd3ac1073f0 2024-03-26T18:59:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.176+0000 7fd3b601b740 1 -- 172.21.15.97:0/550522919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5e0950 msgr2=0x558e2c5e5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.176+0000 7fd3b601b740 1 --2- 172.21.15.97:0/550522919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5e0950 0x558e2c5e5450 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009c80 tx=0x7fd39c0117b0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.177+0000 7fd3ab400640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3900a7500 0x7fd3900a99c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x558e2c5e11b0 tx=0x7fd398005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.177+0000 7fd3b601b740 1 -- 172.21.15.97:0/550522919 shutdown_connections 2024-03-26T18:59:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.177+0000 7fd3b601b740 1 --2- 172.21.15.97:0/550522919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5e7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.177+0000 7fd3b601b740 1 --2- 172.21.15.97:0/550522919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5e0950 0x558e2c5e5450 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.177+0000 7fd3b601b740 1 --2- 172.21.15.97:0/550522919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5dff60 0x558e2c5e0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.177+0000 7fd3b601b740 1 -- 172.21.15.97:0/550522919 >> 172.21.15.97:0/550522919 conn(0x558e2c5db4d0 msgr2=0x558e2c5db8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.178+0000 7fd3b601b740 1 -- 172.21.15.97:0/550522919 shutdown_connections 2024-03-26T18:59:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.178+0000 7fd3b601b740 1 -- 172.21.15.97:0/550522919 wait complete. 2024-03-26T18:59:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.178+0000 7fd3b601b740 1 -- start start 2024-03-26T18:59:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.178+0000 7fd3b601b740 1 Processor -- start 2024-03-26T18:59:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.179+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.179+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 0x558e2c5ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.179+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5f1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.179+0000 7fd3b601b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558e2c5e8a70 con 0x558e2c5e0950 2024-03-26T18:59:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.179+0000 7fd3b601b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558e2c500ac0 con 0x558e2c5dff60 2024-03-26T18:59:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.179+0000 7fd3b601b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558e2c5e8bf0 con 0x558e2c5e5ac0 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.179+0000 7fd3b601b740 10 client.0 _ll_register_callbacks cb 0x558e2c500490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.179+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.179+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53892/0 (socket says 172.21.15.97:53892) 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.179+0000 7fd3abe00640 1 -- 172.21.15.97:0/3202134311 learned_addr learned my addr 172.21.15.97:0/3202134311 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd3ab400640 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 0x558e2c5ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd3abe00640 1 -- 172.21.15.97:0/3202134311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 msgr2=0x558e2c5f1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd3b1200640 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd3abe00640 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5f1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd3abe00640 1 -- 172.21.15.97:0/3202134311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 msgr2=0x558e2c5ea4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd3abe00640 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 0x558e2c5ea4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd3abe00640 1 -- 172.21.15.97:0/3202134311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2c325620 con 0x558e2c5dff60 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd3b1200640 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:59:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd3ab400640 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 0x558e2c5ea4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:59:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd3abe00640 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e9f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009590 tx=0x7fd39c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459d15d98197eb60 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.180+0000 7fd396000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:59:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.181+0000 7fd396000640 1 -- 172.21.15.97:0/3202134311 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c028020 con 0x558e2c5dff60 2024-03-26T18:59:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.181+0000 7fd3b601b740 1 -- 172.21.15.97:0/3202134311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd39c012ad0 con 0x558e2c5dff60 2024-03-26T18:59:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.181+0000 7fd396000640 1 -- 172.21.15.97:0/3202134311 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd39c012650 con 0x558e2c5dff60 2024-03-26T18:59:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.181+0000 7fd396000640 1 -- 172.21.15.97:0/3202134311 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd39c00f4f0 con 0x558e2c5dff60 2024-03-26T18:59:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.181+0000 7fd3b601b740 10 client.25349 fetch_fsmap learned FSMap version 27 2024-03-26T18:59:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.181+0000 7fd3b601b740 1 -- 172.21.15.97:0/3202134311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e2c5f48d0 con 0x558e2c5dff60 2024-03-26T18:59:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.182+0000 7fd396000640 1 -- 172.21.15.97:0/3202134311 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39c012ad0 con 0x558e2c5dff60 2024-03-26T18:59:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.183+0000 7fd396000640 1 -- 172.21.15.97:0/3202134311 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd39c0038f0 con 0x558e2c5dff60 2024-03-26T18:59:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.183+0000 7fd3b601b740 10 client.25349 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T18:59:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.183+0000 7fd3b601b740 10 client.25349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:59:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.183+0000 7fd3b601b740 20 client.25349 populate_metadata read hostname 'smithi097' 2024-03-26T18:59:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.183+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558e2c5f5ce0 0x558e2c5f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.183+0000 7fd3b601b740 4 client.25349 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:59:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.183+0000 7fd3b601b740 1 -- 172.21.15.97:0/3202134311 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e2c5523c0 con 0x558e2c5f5ce0 2024-03-26T18:59:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.183+0000 7fd3ab400640 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558e2c5f5ce0 0x558e2c5f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.184+0000 7fd3ab400640 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558e2c5f5ce0 0x558e2c5f80d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5cee08feab9614e server_cookie=2da6e8f51b7567f2 in_seq=0 out_seq=0 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.184+0000 7fd396000640 10 client.25349 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.185+0000 7fd396000640 1 -- 172.21.15.97:0/3202134311 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558e2c5523c0 con 0x558e2c5f5ce0 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.185+0000 7fd396000640 10 client.25349 handle_command_reply: tid=0 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.185+0000 7fd3b601b740 1 client.25349 shutdown 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.185+0000 7fd3b601b740 1 -- 172.21.15.97:0/3202134311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 msgr2=0x558e2c5e9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.185+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e9f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009590 tx=0x7fd39c004610 comp rx=0 tx=0).stop 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.186+0000 7fd3b601b740 1 -- 172.21.15.97:0/3202134311 shutdown_connections 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.186+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558e2c5f5ce0 0x558e2c5f80d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.186+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.186+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 0x558e2c5ea4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.186+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3202134311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e9f90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.186+0000 7fd3b601b740 1 -- 172.21.15.97:0/3202134311 >> 172.21.15.97:0/3202134311 conn(0x558e2c5db4d0 msgr2=0x558e2c5da6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.186+0000 7fd396000640 0 client.25349 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.186+0000 7fd3b601b740 1 -- 172.21.15.97:0/3202134311 shutdown_connections 2024-03-26T18:59:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.186+0000 7fd3b601b740 1 -- 172.21.15.97:0/3202134311 wait complete. 2024-03-26T18:59:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.186+0000 7fd3b601b740 20 client.25349 trim_cache size 0 max 16384 2024-03-26T18:59:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.214+0000 7fd3b601b740 1 Processor -- start 2024-03-26T18:59:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.214+0000 7fd3b601b740 1 -- start start 2024-03-26T18:59:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e0950 0x558e2c5e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e5ac0 0x558e2c5ea030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3b601b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1a6910 con 0x558e2c5e5ac0 2024-03-26T18:59:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3b601b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1a8450 con 0x558e2c5dff60 2024-03-26T18:59:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3b601b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1a6fe0 con 0x558e2c5e0950 2024-03-26T18:59:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53896/0 (socket says 172.21.15.97:53896) 2024-03-26T18:59:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3abe00640 1 -- 172.21.15.97:0/994858721 learned_addr learned my addr 172.21.15.97:0/994858721 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3ab400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e0950 0x558e2c5e9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.215+0000 7fd3b1200640 1 --2- 172.21.15.97:0/994858721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e5ac0 0x558e2c5ea030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.216+0000 7fd3abe00640 1 -- 172.21.15.97:0/994858721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e0950 msgr2=0x558e2c5e9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.216+0000 7fd3abe00640 1 --2- 172.21.15.97:0/994858721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e0950 0x558e2c5e9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.216+0000 7fd3abe00640 1 -- 172.21.15.97:0/994858721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e5ac0 msgr2=0x558e2c5ea030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.216+0000 7fd3abe00640 1 --2- 172.21.15.97:0/994858721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e5ac0 0x558e2c5ea030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.216+0000 7fd3abe00640 1 -- 172.21.15.97:0/994858721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e2c5f8660 con 0x558e2c5dff60 2024-03-26T18:59:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.216+0000 7fd3abe00640 1 --2- 172.21.15.97:0/994858721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e95b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00f4c0 tx=0x7fd39c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb9159bf98fd3a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.216+0000 7fd3a0c00640 1 -- 172.21.15.97:0/994858721 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c03d070 con 0x558e2c5dff60 2024-03-26T18:59:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.216+0000 7fd3a0c00640 1 -- 172.21.15.97:0/994858721 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39c003ad0 con 0x558e2c5dff60 2024-03-26T18:59:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.217+0000 7fd3a0c00640 1 -- 172.21.15.97:0/994858721 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c028040 con 0x558e2c5dff60 2024-03-26T18:59:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.218+0000 7fd3b601b740 1 -- 172.21.15.97:0/994858721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 msgr2=0x558e2c5e95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.218+0000 7fd3b601b740 1 --2- 172.21.15.97:0/994858721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e95b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00f4c0 tx=0x7fd39c0117b0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.218+0000 7fd3b601b740 1 -- 172.21.15.97:0/994858721 shutdown_connections 2024-03-26T18:59:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.218+0000 7fd3b601b740 1 --2- 172.21.15.97:0/994858721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e5ac0 0x558e2c5ea030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.218+0000 7fd3b601b740 1 --2- 172.21.15.97:0/994858721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e0950 0x558e2c5e9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.218+0000 7fd3b601b740 1 --2- 172.21.15.97:0/994858721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5e95b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.218+0000 7fd3b601b740 1 -- 172.21.15.97:0/994858721 >> 172.21.15.97:0/994858721 conn(0x558e2c5db4d0 msgr2=0x558e2c5e1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.219+0000 7fd3b601b740 1 -- 172.21.15.97:0/994858721 shutdown_connections 2024-03-26T18:59:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.219+0000 7fd3b601b740 1 -- 172.21.15.97:0/994858721 wait complete. 2024-03-26T18:59:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.220+0000 7fd3b601b740 1 -- start start 2024-03-26T18:59:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.220+0000 7fd3b601b740 1 Processor -- start 2024-03-26T18:59:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.221+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5eec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.221+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 0x558e2c5ef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.221+0000 7fd3b601b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5eec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5eec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53900/0 (socket says 172.21.15.97:53900) 2024-03-26T18:59:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3abe00640 1 -- 172.21.15.97:0/3954095410 learned_addr learned my addr 172.21.15.97:0/3954095410 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T18:59:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1a7d80 con 0x558e2c5e0950 2024-03-26T18:59:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1a76b0 con 0x558e2c5dff60 2024-03-26T18:59:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1a48f0 con 0x558e2c5e5ac0 2024-03-26T18:59:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3b601b740 10 client.0 _ll_register_callbacks cb 0x7fd39c0044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:59:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3ab400640 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 0x558e2c5ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3b1200640 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3b1200640 1 -- 172.21.15.97:0/3954095410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 msgr2=0x558e2c5eec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3b1200640 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5eec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3b1200640 1 -- 172.21.15.97:0/3954095410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 msgr2=0x558e2c5ef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:59:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3b1200640 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 0x558e2c5ef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.222+0000 7fd3b1200640 1 -- 172.21.15.97:0/3954095410 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2c5f0760 con 0x558e2c5e5ac0 2024-03-26T18:59:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.223+0000 7fd3abe00640 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5eec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:59:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.223+0000 7fd3b1200640 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5f0170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40ef0f0 tx=0x7fd3a40bd910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2b359454e89d79e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:59:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.223+0000 7fd396000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:59:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.224+0000 7fd396000640 1 -- 172.21.15.97:0/3954095410 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a403b4f0 con 0x558e2c5e5ac0 2024-03-26T18:59:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.224+0000 7fd396000640 1 -- 172.21.15.97:0/3954095410 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd3a40c6290 con 0x558e2c5e5ac0 2024-03-26T18:59:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.225+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd39c003ad0 con 0x558e2c5e5ac0 2024-03-26T18:59:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.225+0000 7fd396000640 1 -- 172.21.15.97:0/3954095410 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd3a40c1e60 con 0x558e2c5e5ac0 2024-03-26T18:59:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.225+0000 7fd3b601b740 10 client.25321 fetch_fsmap learned FSMap version 27 2024-03-26T18:59:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.225+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e2c664b30 con 0x558e2c5e5ac0 2024-03-26T18:59:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.225+0000 7fd396000640 1 -- 172.21.15.97:0/3954095410 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39c003ad0 con 0x558e2c5e5ac0 2024-03-26T18:59:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.225+0000 7fd396000640 1 -- 172.21.15.97:0/3954095410 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd3a403c030 con 0x558e2c5e5ac0 2024-03-26T18:59:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.226+0000 7fd3b601b740 10 client.25321 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T18:59:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.226+0000 7fd3b601b740 10 client.25321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:59:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.226+0000 7fd3b601b740 20 client.25321 populate_metadata read hostname 'smithi097' 2024-03-26T18:59:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.226+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558e2c6654a0 0x558e2c6678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:59:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.226+0000 7fd3b601b740 4 client.25321 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:59:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.226+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e2c54c710 con 0x558e2c6654a0 2024-03-26T18:59:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.226+0000 7fd3abe00640 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558e2c6654a0 0x558e2c6678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:59:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.227+0000 7fd3abe00640 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558e2c6654a0 0x558e2c6678c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ac4518a727b37b5 server_cookie=c159717f28f22287 in_seq=0 out_seq=0 2024-03-26T18:59:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.227+0000 7fd396000640 10 client.25321 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.247+0000 7fd396000640 1 -- 172.21.15.97:0/3954095410 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e2c54c710 con 0x558e2c6654a0 2024-03-26T18:59:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.247+0000 7fd396000640 10 client.25321 handle_command_reply: tid=0 2024-03-26T18:59:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.248+0000 7fd3b601b740 1 client.25321 shutdown 2024-03-26T18:59:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.248+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 msgr2=0x558e2c5f0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.248+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5f0170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40ef0f0 tx=0x7fd3a40bd910 comp rx=0 tx=0).stop 2024-03-26T18:59:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.248+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 shutdown_connections 2024-03-26T18:59:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.248+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558e2c6654a0 0x558e2c6678c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.248+0000 7fd396000640 0 client.25321 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T18:59:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.248+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e2c5e5ac0 0x558e2c5f0170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.248+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558e2c5e0950 0x558e2c5ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.248+0000 7fd3b601b740 1 --2- 172.21.15.97:0/3954095410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e2c5dff60 0x558e2c5eec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.248+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 >> 172.21.15.97:0/3954095410 conn(0x558e2c5db4d0 msgr2=0x558e2c5e11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.249+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 shutdown_connections 2024-03-26T18:59:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.249+0000 7fd3b601b740 1 -- 172.21.15.97:0/3954095410 wait complete. 2024-03-26T18:59:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.249+0000 7fd3b601b740 20 client.25321 trim_cache size 0 max 16384 2024-03-26T18:59:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T18:59:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T18:59:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T18:59:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T18:59:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.250+0000 7fd3b1c00640 1 -- 172.21.15.97:0/3298339800 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3900a7500 msgr2=0x7fd3900a99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.250+0000 7fd3b1c00640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3900a7500 0x7fd3900a99c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x558e2c5e11b0 tx=0x7fd398005f30 comp rx=0 tx=0).stop 2024-03-26T18:59:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.250+0000 7fd3b1c00640 1 -- 172.21.15.97:0/3298339800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac1073f0 msgr2=0x7fd3ac19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:59:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.250+0000 7fd3b1c00640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac1073f0 0x7fd3ac19ef80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4037040 tx=0x7fd3a4034d00 comp rx=0 tx=0).stop 2024-03-26T18:59:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.251+0000 7fd3b1c00640 1 -- 172.21.15.97:0/3298339800 shutdown_connections 2024-03-26T18:59:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.251+0000 7fd3b1c00640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3900a7500 0x7fd3900a99c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.251+0000 7fd3b1c00640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3ac1073f0 0x7fd3ac19ef80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.251+0000 7fd3b1c00640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac104ac0 0x7fd3ac197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.251+0000 7fd3b1c00640 1 --2- 172.21.15.97:0/3298339800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3ac102190 0x7fd3ac1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:59:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.251+0000 7fd3b1c00640 1 -- 172.21.15.97:0/3298339800 >> 172.21.15.97:0/3298339800 conn(0x7fd3ac0fbba0 msgr2=0x7fd3ac102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:59:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.253+0000 7fd3b1c00640 1 -- 172.21.15.97:0/3298339800 shutdown_connections 2024-03-26T18:59:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T18:59:50.253+0000 7fd3b1c00640 1 -- 172.21.15.97:0/3298339800 wait complete. 2024-03-26T18:59:50.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:50 smithi097 ceph-mon[31955]: pgmap v417: 97 pgs: 97 active+clean; 940 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 797 op/s 2024-03-26T18:59:50.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:50 smithi144 ceph-mon[36053]: pgmap v417: 97 pgs: 97 active+clean; 940 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 797 op/s 2024-03-26T18:59:50.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:50 smithi105 ceph-mon[36270]: pgmap v417: 97 pgs: 97 active+clean; 940 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 797 op/s 2024-03-26T18:59:52.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:52 smithi097 ceph-mon[31955]: pgmap v418: 97 pgs: 97 active+clean; 953 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 732 op/s 2024-03-26T18:59:52.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:52 smithi144 ceph-mon[36053]: pgmap v418: 97 pgs: 97 active+clean; 953 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 732 op/s 2024-03-26T18:59:52.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:52 smithi105 ceph-mon[36270]: pgmap v418: 97 pgs: 97 active+clean; 953 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 732 op/s 2024-03-26T18:59:54.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:54 smithi097 ceph-mon[31955]: pgmap v419: 97 pgs: 97 active+clean; 954 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 695 op/s 2024-03-26T18:59:54.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:54 smithi144 ceph-mon[36053]: pgmap v419: 97 pgs: 97 active+clean; 954 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 695 op/s 2024-03-26T18:59:54.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:54 smithi105 ceph-mon[36270]: pgmap v419: 97 pgs: 97 active+clean; 954 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 695 op/s 2024-03-26T18:59:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 30.0 s, 2816.0 tps, lat 35.653 ms stddev 41.984 2024-03-26T18:59:56.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:56 smithi097 ceph-mon[31955]: pgmap v420: 97 pgs: 97 active+clean; 973 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 825 op/s 2024-03-26T18:59:56.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:56 smithi144 ceph-mon[36053]: pgmap v420: 97 pgs: 97 active+clean; 973 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 825 op/s 2024-03-26T18:59:56.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:56 smithi105 ceph-mon[36270]: pgmap v420: 97 pgs: 97 active+clean; 973 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 825 op/s 2024-03-26T18:59:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:59:58 smithi144 ceph-mon[36053]: pgmap v421: 97 pgs: 97 active+clean; 977 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 690 op/s 2024-03-26T18:59:58.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 18:59:58 smithi105 ceph-mon[36270]: pgmap v421: 97 pgs: 97 active+clean; 977 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 690 op/s 2024-03-26T18:59:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 18:59:58 smithi097 ceph-mon[31955]: pgmap v421: 97 pgs: 97 active+clean; 977 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 690 op/s 2024-03-26T19:00:00.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 35.0 s, 2669.8 tps, lat 37.450 ms stddev 38.101 2024-03-26T19:00:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:00 smithi144 ceph-mon[36053]: pgmap v422: 97 pgs: 97 active+clean; 1001 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 825 op/s 2024-03-26T19:00:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:00 smithi144 ceph-mon[36053]: overall HEALTH_OK 2024-03-26T19:00:00.405 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:00 smithi097 ceph-mon[31955]: pgmap v422: 97 pgs: 97 active+clean; 1001 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 825 op/s 2024-03-26T19:00:00.405 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:00 smithi097 ceph-mon[31955]: overall HEALTH_OK 2024-03-26T19:00:00.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:00 smithi105 ceph-mon[36270]: pgmap v422: 97 pgs: 97 active+clean; 1001 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 825 op/s 2024-03-26T19:00:00.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:00 smithi105 ceph-mon[36270]: overall HEALTH_OK 2024-03-26T19:00:01.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:01 smithi105 ceph-mon[36270]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-26T19:00:01.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:01 smithi097 ceph-mon[31955]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-26T19:00:01.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:01 smithi144 ceph-mon[36053]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-26T19:00:02.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:02 smithi105 ceph-mon[36270]: pgmap v423: 97 pgs: 97 active+clean; 986 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 752 op/s 2024-03-26T19:00:02.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:02 smithi097 ceph-mon[31955]: pgmap v423: 97 pgs: 97 active+clean; 986 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 752 op/s 2024-03-26T19:00:02.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:02 smithi144 ceph-mon[36053]: pgmap v423: 97 pgs: 97 active+clean; 986 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 752 op/s 2024-03-26T19:00:03.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:00:03.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:00:03.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:03 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:00:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.465+0000 7fcd9ca00640 1 -- 172.21.15.97:0/872835318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd98102ae0 msgr2=0x7fcd98102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.465+0000 7fcd9ca00640 1 --2- 172.21.15.97:0/872835318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd98102ae0 0x7fcd98102f60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009900 tx=0x7fcd8402bc30 comp rx=0 tx=0).stop 2024-03-26T19:00:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.465+0000 7fcd9ca00640 1 -- 172.21.15.97:0/872835318 shutdown_connections 2024-03-26T19:00:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.465+0000 7fcd9ca00640 1 --2- 172.21.15.97:0/872835318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd981034a0 0x7fcd98109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.465+0000 7fcd9ca00640 1 --2- 172.21.15.97:0/872835318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd98102ae0 0x7fcd98102f60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.465+0000 7fcd9ca00640 1 --2- 172.21.15.97:0/872835318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd981018f0 0x7fcd98101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.465+0000 7fcd9ca00640 1 -- 172.21.15.97:0/872835318 >> 172.21.15.97:0/872835318 conn(0x7fcd980fce80 msgr2=0x7fcd980ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.466+0000 7fcd9ca00640 1 -- 172.21.15.97:0/872835318 shutdown_connections 2024-03-26T19:00:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.466+0000 7fcd9ca00640 1 -- 172.21.15.97:0/872835318 wait complete. 2024-03-26T19:00:03.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:00:03.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:00:03.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:03 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:00:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.467+0000 7fcd9ca00640 1 Processor -- start 2024-03-26T19:00:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.468+0000 7fcd9ca00640 1 -- start start 2024-03-26T19:00:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd9ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd981018f0 0x7fcd98199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd9ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd98102ae0 0x7fcd9819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd9ca00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd981034a0 0x7fcd981a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd9ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9810d620 con 0x7fcd981018f0 2024-03-26T19:00:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd9ca00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9810d4a0 con 0x7fcd981034a0 2024-03-26T19:00:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd9ca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9810d7a0 con 0x7fcd98102ae0 2024-03-26T19:00:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd96800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd981034a0 0x7fcd981a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd95e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd981018f0 0x7fcd98199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd96800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd981034a0 0x7fcd981a1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38042/0 (socket says 172.21.15.97:38042) 2024-03-26T19:00:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd96800640 1 -- 172.21.15.97:0/219300559 learned_addr learned my addr 172.21.15.97:0/219300559 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.469+0000 7fcd95400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd98102ae0 0x7fcd9819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.470+0000 7fcd96800640 1 -- 172.21.15.97:0/219300559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd98102ae0 msgr2=0x7fcd9819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.470+0000 7fcd96800640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd98102ae0 0x7fcd9819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.470+0000 7fcd96800640 1 -- 172.21.15.97:0/219300559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd981018f0 msgr2=0x7fcd98199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.470+0000 7fcd96800640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd981018f0 0x7fcd98199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.470+0000 7fcd96800640 1 -- 172.21.15.97:0/219300559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd981a19d0 con 0x7fcd981034a0 2024-03-26T19:00:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.470+0000 7fcd96800640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd981034a0 0x7fcd981a1330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000b970 tx=0x7fcd9000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.470+0000 7fcd95e00640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd981018f0 0x7fcd98199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:00:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.471+0000 7fcd8ea00640 1 -- 172.21.15.97:0/219300559 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd90019070 con 0x7fcd981034a0 2024-03-26T19:00:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.471+0000 7fcd8ea00640 1 -- 172.21.15.97:0/219300559 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9000cb40 con 0x7fcd981034a0 2024-03-26T19:00:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.471+0000 7fcd8ea00640 1 -- 172.21.15.97:0/219300559 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd90007660 con 0x7fcd981034a0 2024-03-26T19:00:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.471+0000 7fcd9ca00640 1 -- 172.21.15.97:0/219300559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd981a1cc0 con 0x7fcd981034a0 2024-03-26T19:00:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.471+0000 7fcd9ca00640 1 -- 172.21.15.97:0/219300559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd981a21c0 con 0x7fcd981034a0 2024-03-26T19:00:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.476+0000 7fcd8ea00640 1 -- 172.21.15.97:0/219300559 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fcd900149b0 con 0x7fcd981034a0 2024-03-26T19:00:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.476+0000 7fcd9f809740 1 Processor -- start 2024-03-26T19:00:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.477+0000 7fcd9f809740 1 -- start start 2024-03-26T19:00:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.479+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.479+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe49aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.479+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe4c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.479+0000 7fcd9f809740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557a6fda0e80 con 0x557a6fe4a110 2024-03-26T19:00:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.479+0000 7fcd9f809740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557a6fd49660 con 0x557a6fe445b0 2024-03-26T19:00:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.479+0000 7fcd9f809740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a6fd9ce30 con 0x557a6fe44fa0 2024-03-26T19:00:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.479+0000 7fcd95e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe49aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd8ea00640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcd7c0a3040 0x7fcd7c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd95e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe49aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:35338/0 (socket says 172.21.15.97:35338) 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd95e00640 1 -- 172.21.15.97:0/2826133379 learned_addr learned my addr 172.21.15.97:0/2826133379 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd95400640 1 --2- 172.21.15.97:0/2826133379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd8ea00640 1 -- 172.21.15.97:0/219300559 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcd900ce930 con 0x7fcd981034a0 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd95400640 1 --2- 172.21.15.97:0/2826133379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe4c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd95e00640 1 -- 172.21.15.97:0/2826133379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 msgr2=0x557a6fe449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd95e00640 1 --2- 172.21.15.97:0/2826133379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd95e00640 1 -- 172.21.15.97:0/2826133379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 msgr2=0x557a6fe4c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd96800640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcd7c0a3040 0x7fcd7c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.480+0000 7fcd95e00640 1 --2- 172.21.15.97:0/2826133379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe4c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.481+0000 7fcd95e00640 1 -- 172.21.15.97:0/2826133379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6fab5210 con 0x557a6fe44fa0 2024-03-26T19:00:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.481+0000 7fcd95400640 1 --2- 172.21.15.97:0/2826133379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe4c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:00:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.481+0000 7fcd95e00640 1 --2- 172.21.15.97:0/2826133379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe49aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd88010480 tx=0x7fcd88010940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef2f5518dc3e2bb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.481+0000 7fcd83000640 1 -- 172.21.15.97:0/2826133379 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88011760 con 0x557a6fe44fa0 2024-03-26T19:00:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.482+0000 7fcd83000640 1 -- 172.21.15.97:0/2826133379 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd88011de0 con 0x557a6fe44fa0 2024-03-26T19:00:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.482+0000 7fcd83000640 1 -- 172.21.15.97:0/2826133379 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800e430 con 0x557a6fe44fa0 2024-03-26T19:00:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.483+0000 7fcd9f809740 1 -- 172.21.15.97:0/2826133379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 msgr2=0x557a6fe49aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.483+0000 7fcd9f809740 1 --2- 172.21.15.97:0/2826133379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe49aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd88010480 tx=0x7fcd88010940 comp rx=0 tx=0).stop 2024-03-26T19:00:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.484+0000 7fcd9f809740 1 -- 172.21.15.97:0/2826133379 shutdown_connections 2024-03-26T19:00:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.484+0000 7fcd9f809740 1 --2- 172.21.15.97:0/2826133379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe4c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.484+0000 7fcd9f809740 1 --2- 172.21.15.97:0/2826133379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe49aa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.484+0000 7fcd9f809740 1 --2- 172.21.15.97:0/2826133379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.484+0000 7fcd9f809740 1 -- 172.21.15.97:0/2826133379 >> 172.21.15.97:0/2826133379 conn(0x557a6fe3fb20 msgr2=0x557a6fe3ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.484+0000 7fcd96800640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcd7c0a3040 0x7fcd7c0a5500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd900a3510 tx=0x7fcd90004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.486+0000 7fcd9f809740 1 -- 172.21.15.97:0/2826133379 shutdown_connections 2024-03-26T19:00:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.486+0000 7fcd9f809740 1 -- 172.21.15.97:0/2826133379 wait complete. 2024-03-26T19:00:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.486+0000 7fcd9f809740 1 -- start start 2024-03-26T19:00:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.487+0000 7fcd9f809740 1 Processor -- start 2024-03-26T19:00:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.487+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe4e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.487+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe4eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.487+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe55b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.487+0000 7fcd9f809740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557a6fe4d0c0 con 0x557a6fe4a110 2024-03-26T19:00:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.487+0000 7fcd9f809740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557a6fd5cac0 con 0x557a6fe445b0 2024-03-26T19:00:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.487+0000 7fcd9f809740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a6fe4d240 con 0x557a6fe44fa0 2024-03-26T19:00:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.487+0000 7fcd9f809740 10 client.0 _ll_register_callbacks cb 0x557a6fd5c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:00:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.488+0000 7fcd95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe4e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.488+0000 7fcd95e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe4eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.488+0000 7fcd95e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe4eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:35350/0 (socket says 172.21.15.97:35350) 2024-03-26T19:00:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.488+0000 7fcd95e00640 1 -- 172.21.15.97:0/3491832284 learned_addr learned my addr 172.21.15.97:0/3491832284 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.488+0000 7fcd95400640 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe55b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.489+0000 7fcd95e00640 1 -- 172.21.15.97:0/3491832284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 msgr2=0x557a6fe4e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.489+0000 7fcd95e00640 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe4e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.489+0000 7fcd95e00640 1 -- 172.21.15.97:0/3491832284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 msgr2=0x557a6fe55b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.489+0000 7fcd95e00640 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe55b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.489+0000 7fcd95e00640 1 -- 172.21.15.97:0/3491832284 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6fb81620 con 0x557a6fe44fa0 2024-03-26T19:00:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.492+0000 7fcd95e00640 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe4eb20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800b5f0 tx=0x7fcd88005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f48ca27627cd2517 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.493+0000 7fcd77400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:00:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.493+0000 7fcd77400640 1 -- 172.21.15.97:0/3491832284 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8801c890 con 0x557a6fe44fa0 2024-03-26T19:00:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.493+0000 7fcd77400640 1 -- 172.21.15.97:0/3491832284 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcd88004040 con 0x557a6fe44fa0 2024-03-26T19:00:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.494+0000 7fcd9f809740 1 -- 172.21.15.97:0/3491832284 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd88011de0 con 0x557a6fe44fa0 2024-03-26T19:00:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.498+0000 7fcd77400640 1 -- 172.21.15.97:0/3491832284 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd88004240 con 0x557a6fe44fa0 2024-03-26T19:00:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.498+0000 7fcd9f809740 10 client.25330 fetch_fsmap learned FSMap version 27 2024-03-26T19:00:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.498+0000 7fcd9f809740 1 -- 172.21.15.97:0/3491832284 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a6fe39b90 con 0x557a6fe44fa0 2024-03-26T19:00:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.498+0000 7fcd77400640 1 -- 172.21.15.97:0/3491832284 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd88011de0 con 0x557a6fe44fa0 2024-03-26T19:00:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.499+0000 7fcd77400640 1 -- 172.21.15.97:0/3491832284 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcd8802f030 con 0x557a6fe44fa0 2024-03-26T19:00:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.499+0000 7fcd9f809740 10 client.25330 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:00:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.499+0000 7fcd9f809740 10 client.25330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:00:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.499+0000 7fcd9f809740 20 client.25330 populate_metadata read hostname 'smithi097' 2024-03-26T19:00:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.499+0000 7fcd9f809740 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557a6fe3af90 0x557a6fe3d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.499+0000 7fcd9f809740 4 client.25330 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:00:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.499+0000 7fcd9f809740 1 -- 172.21.15.97:0/3491832284 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a6fdae3c0 con 0x557a6fe3af90 2024-03-26T19:00:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.502+0000 7fcd95400640 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557a6fe3af90 0x557a6fe3d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.504+0000 7fcd77400640 10 client.25330 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.504+0000 7fcd95400640 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557a6fe3af90 0x557a6fe3d3d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b04a063bd699435d server_cookie=7e15b2b10cb84563 in_seq=0 out_seq=0 2024-03-26T19:00:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.505+0000 7fcd77400640 1 -- 172.21.15.97:0/3491832284 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557a6fdae3c0 con 0x557a6fe3af90 2024-03-26T19:00:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.505+0000 7fcd77400640 10 client.25330 handle_command_reply: tid=0 2024-03-26T19:00:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.505+0000 7fcd9f809740 1 client.25330 shutdown 2024-03-26T19:00:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.506+0000 7fcd9f809740 1 -- 172.21.15.97:0/3491832284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 msgr2=0x557a6fe4eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.506+0000 7fcd9f809740 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe4eb20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800b5f0 tx=0x7fcd88005760 comp rx=0 tx=0).stop 2024-03-26T19:00:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.506+0000 7fcd9f809740 1 -- 172.21.15.97:0/3491832284 shutdown_connections 2024-03-26T19:00:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.506+0000 7fcd9f809740 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557a6fe3af90 0x557a6fe3d3d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.506+0000 7fcd9f809740 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe55b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.506+0000 7fcd9f809740 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe4eb20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.506+0000 7fcd9f809740 1 --2- 172.21.15.97:0/3491832284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe4e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.506+0000 7fcd77400640 0 client.25330 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.507+0000 7fcd9f809740 1 -- 172.21.15.97:0/3491832284 >> 172.21.15.97:0/3491832284 conn(0x557a6fe3fb20 msgr2=0x557a6fe36540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.507+0000 7fcd9f809740 1 -- 172.21.15.97:0/3491832284 shutdown_connections 2024-03-26T19:00:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.507+0000 7fcd9f809740 1 -- 172.21.15.97:0/3491832284 wait complete. 2024-03-26T19:00:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.508+0000 7fcd9f809740 20 client.25330 trim_cache size 0 max 16384 2024-03-26T19:00:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.567+0000 7fcd9f809740 1 Processor -- start 2024-03-26T19:00:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.570+0000 7fcd9f809740 1 -- start start 2024-03-26T19:00:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.570+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.570+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe55b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.570+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe45f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.570+0000 7fcd9f809740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981a6b10 con 0x557a6fe4a110 2024-03-26T19:00:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.570+0000 7fcd9f809740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981a8650 con 0x557a6fe445b0 2024-03-26T19:00:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.571+0000 7fcd9f809740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981a71e0 con 0x557a6fe44fa0 2024-03-26T19:00:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.572+0000 7fcd95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.572+0000 7fcd95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe45a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38072/0 (socket says 172.21.15.97:38072) 2024-03-26T19:00:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.572+0000 7fcd95400640 1 -- 172.21.15.97:0/3728080590 learned_addr learned my addr 172.21.15.97:0/3728080590 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.572+0000 7fcd95400640 1 --2- 172.21.15.97:0/3728080590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe45f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.573+0000 7fcd95400640 1 -- 172.21.15.97:0/3728080590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 msgr2=0x557a6fe55b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:00:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.573+0000 7fcd95400640 1 --2- 172.21.15.97:0/3728080590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe55b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.574+0000 7fcd95400640 1 -- 172.21.15.97:0/3728080590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 msgr2=0x557a6fe45f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.574+0000 7fcd95400640 1 --2- 172.21.15.97:0/3728080590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe45f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.574+0000 7fcd95400640 1 -- 172.21.15.97:0/3728080590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6fe3d960 con 0x557a6fe445b0 2024-03-26T19:00:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.574+0000 7fcd95400640 1 --2- 172.21.15.97:0/3728080590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe45a10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcd840096c0 tx=0x7fcd84009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f410427adebbb37 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.575+0000 7fcd82600640 1 -- 172.21.15.97:0/3728080590 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8404ad90 con 0x557a6fe445b0 2024-03-26T19:00:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.575+0000 7fcd82600640 1 -- 172.21.15.97:0/3728080590 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd8404a300 con 0x557a6fe445b0 2024-03-26T19:00:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.575+0000 7fcd82600640 1 -- 172.21.15.97:0/3728080590 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84005530 con 0x557a6fe445b0 2024-03-26T19:00:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.577+0000 7fcd9f809740 1 -- 172.21.15.97:0/3728080590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 msgr2=0x557a6fe45a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.577+0000 7fcd9f809740 1 --2- 172.21.15.97:0/3728080590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe45a10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcd840096c0 tx=0x7fcd84009260 comp rx=0 tx=0).stop 2024-03-26T19:00:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.577+0000 7fcd9f809740 1 -- 172.21.15.97:0/3728080590 shutdown_connections 2024-03-26T19:00:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.578+0000 7fcd9f809740 1 --2- 172.21.15.97:0/3728080590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe4a110 0x557a6fe45f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.578+0000 7fcd9f809740 1 --2- 172.21.15.97:0/3728080590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe55b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.578+0000 7fcd9f809740 1 --2- 172.21.15.97:0/3728080590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe445b0 0x557a6fe45a10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.578+0000 7fcd9f809740 1 -- 172.21.15.97:0/3728080590 >> 172.21.15.97:0/3728080590 conn(0x557a6fe3fb20 msgr2=0x557a6fe4e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.578+0000 7fcd9f809740 1 -- 172.21.15.97:0/3728080590 shutdown_connections 2024-03-26T19:00:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.578+0000 7fcd9f809740 1 -- 172.21.15.97:0/3728080590 wait complete. 2024-03-26T19:00:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.579+0000 7fcd9f809740 1 -- start start 2024-03-26T19:00:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.579+0000 7fcd9f809740 1 Processor -- start 2024-03-26T19:00:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.581+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe445b0 0x557a6fe51b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.581+0000 7fcd9f809740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe54050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.582+0000 7fcd95e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe54050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.582+0000 7fcd95e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe54050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:35354/0 (socket says 172.21.15.97:35354) 2024-03-26T19:00:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.582+0000 7fcd95400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe445b0 0x557a6fe51b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.582+0000 7fcd95400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe445b0 0x557a6fe51b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40492/0 (socket says 172.21.15.97:40492) 2024-03-26T19:00:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.582+0000 7fcd95e00640 1 -- 172.21.15.97:0/90021960 learned_addr learned my addr 172.21.15.97:0/90021960 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.583+0000 7fcd9f809740 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe4a110 0x557a6fe546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.583+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981a7f80 con 0x557a6fe445b0 2024-03-26T19:00:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.583+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981a78b0 con 0x557a6fe4a110 2024-03-26T19:00:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.583+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981a4b30 con 0x557a6fe44fa0 2024-03-26T19:00:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.584+0000 7fcd95e00640 1 -- 172.21.15.97:0/90021960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe4a110 msgr2=0x557a6fe546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.584+0000 7fcd95400640 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe4a110 0x557a6fe546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.584+0000 7fcd95e00640 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe4a110 0x557a6fe546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.584+0000 7fcd95e00640 1 -- 172.21.15.97:0/90021960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe445b0 msgr2=0x557a6fe51b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.584+0000 7fcd95e00640 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe445b0 0x557a6fe51b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.584+0000 7fcd95e00640 1 -- 172.21.15.97:0/90021960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6fe54ed0 con 0x557a6fe44fa0 2024-03-26T19:00:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.584+0000 7fcd95400640 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe4a110 0x557a6fe546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:00:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.584+0000 7fcd95e00640 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe54050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd880037d0 tx=0x7fcd88003bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96acf79810b4c009 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.584+0000 7fcd9f809740 10 client.0 _ll_register_callbacks cb 0x7fcd8800beb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:00:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.584+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd8404a300 con 0x557a6fe44fa0 2024-03-26T19:00:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.585+0000 7fcd77400640 10 client.25333 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:00:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.585+0000 7fcd77400640 1 -- 172.21.15.97:0/90021960 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8803e060 con 0x557a6fe44fa0 2024-03-26T19:00:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.585+0000 7fcd77400640 1 -- 172.21.15.97:0/90021960 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcd8801c690 con 0x557a6fe44fa0 2024-03-26T19:00:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.585+0000 7fcd77400640 1 -- 172.21.15.97:0/90021960 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd8801c8d0 con 0x557a6fe44fa0 2024-03-26T19:00:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.585+0000 7fcd9f809740 10 client.25333 fetch_fsmap learned FSMap version 27 2024-03-26T19:00:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.585+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=27}) -- 0x557a6fed9810 con 0x557a6fe44fa0 2024-03-26T19:00:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.586+0000 7fcd77400640 1 -- 172.21.15.97:0/90021960 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcd8802f040 con 0x557a6fe44fa0 2024-03-26T19:00:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.587+0000 7fcd9f809740 10 client.25333 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:00:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.587+0000 7fcd9f809740 10 client.25333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:00:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.587+0000 7fcd9f809740 20 client.25333 populate_metadata read hostname 'smithi097' 2024-03-26T19:00:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.587+0000 7fcd9f809740 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557a6feda130 0x557a6fedc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.587+0000 7fcd9f809740 4 client.25333 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:00:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.587+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a6fda8710 con 0x557a6feda130 2024-03-26T19:00:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.587+0000 7fcd95400640 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557a6feda130 0x557a6fedc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.588+0000 7fcd77400640 10 client.25333 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.589+0000 7fcd95400640 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557a6feda130 0x557a6fedc550 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61060c90242efd64 server_cookie=52669de1547562ce in_seq=0 out_seq=0 2024-03-26T19:00:03.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:00:03.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:00:03.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:03 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:00:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.617+0000 7fcd77400640 1 -- 172.21.15.97:0/90021960 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a6fda8710 con 0x557a6feda130 2024-03-26T19:00:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.618+0000 7fcd77400640 10 client.25333 handle_command_reply: tid=0 2024-03-26T19:00:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.618+0000 7fcd9f809740 1 client.25333 shutdown 2024-03-26T19:00:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.619+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 msgr2=0x557a6fe54050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.619+0000 7fcd9f809740 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe54050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd880037d0 tx=0x7fcd88003bf0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.619+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 shutdown_connections 2024-03-26T19:00:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.619+0000 7fcd9f809740 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557a6feda130 0x557a6fedc550 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.619+0000 7fcd9f809740 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a6fe4a110 0x557a6fe546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.619+0000 7fcd9f809740 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a6fe44fa0 0x557a6fe54050 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd880037d0 tx=0x7fcd88003bf0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.619+0000 7fcd9f809740 1 --2- 172.21.15.97:0/90021960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a6fe445b0 0x557a6fe51b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.619+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 >> 172.21.15.97:0/90021960 conn(0x557a6fe3fb20 msgr2=0x557a6fe42d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.619+0000 7fcd77400640 0 client.25333 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.620+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 shutdown_connections 2024-03-26T19:00:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.620+0000 7fcd9f809740 1 -- 172.21.15.97:0/90021960 wait complete. 2024-03-26T19:00:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.622+0000 7fcd9f809740 20 client.25333 trim_cache size 0 max 16384 2024-03-26T19:00:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:00:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:00:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:00:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:00:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.623+0000 7fcd9ca00640 1 -- 172.21.15.97:0/219300559 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcd7c0a3040 msgr2=0x7fcd7c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.623+0000 7fcd9ca00640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcd7c0a3040 0x7fcd7c0a5500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd900a3510 tx=0x7fcd90004b50 comp rx=0 tx=0).stop 2024-03-26T19:00:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.623+0000 7fcd9ca00640 1 -- 172.21.15.97:0/219300559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd981034a0 msgr2=0x7fcd981a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.623+0000 7fcd9ca00640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd981034a0 0x7fcd981a1330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000b970 tx=0x7fcd9000be30 comp rx=0 tx=0).stop 2024-03-26T19:00:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.624+0000 7fcd9ca00640 1 -- 172.21.15.97:0/219300559 shutdown_connections 2024-03-26T19:00:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.624+0000 7fcd9ca00640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcd7c0a3040 0x7fcd7c0a5500 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.624+0000 7fcd9ca00640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd981034a0 0x7fcd981a1330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.624+0000 7fcd9ca00640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd98102ae0 0x7fcd9819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.624+0000 7fcd9ca00640 1 --2- 172.21.15.97:0/219300559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd981018f0 0x7fcd98199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.624+0000 7fcd9ca00640 1 -- 172.21.15.97:0/219300559 >> 172.21.15.97:0/219300559 conn(0x7fcd980fce80 msgr2=0x7fcd98073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.625+0000 7fcd9ca00640 1 -- 172.21.15.97:0/219300559 shutdown_connections 2024-03-26T19:00:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:03.625+0000 7fcd9ca00640 1 -- 172.21.15.97:0/219300559 wait complete. 2024-03-26T19:00:04.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:04 smithi105 ceph-mon[36270]: pgmap v424: 97 pgs: 97 active+clean; 967 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 700 op/s 2024-03-26T19:00:04.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:04 smithi097 ceph-mon[31955]: pgmap v424: 97 pgs: 97 active+clean; 967 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 700 op/s 2024-03-26T19:00:04.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:04 smithi144 ceph-mon[36053]: pgmap v424: 97 pgs: 97 active+clean; 967 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 700 op/s 2024-03-26T19:00:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 40.0 s, 2724.1 tps, lat 36.727 ms stddev 39.652 2024-03-26T19:00:05.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:00:05.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:00:05.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:00:05.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:00:05.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:00:05.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:00:05.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:00:05.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:00:05.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:00:06.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:06 smithi105 ceph-mon[36270]: pgmap v425: 97 pgs: 97 active+clean; 976 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 821 op/s 2024-03-26T19:00:06.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:06 smithi097 ceph-mon[31955]: pgmap v425: 97 pgs: 97 active+clean; 976 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 821 op/s 2024-03-26T19:00:06.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:06 smithi144 ceph-mon[36053]: pgmap v425: 97 pgs: 97 active+clean; 976 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 821 op/s 2024-03-26T19:00:08.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:08 smithi105 ceph-mon[36270]: pgmap v426: 97 pgs: 97 active+clean; 977 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 613 op/s 2024-03-26T19:00:08.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:08 smithi097 ceph-mon[31955]: pgmap v426: 97 pgs: 97 active+clean; 977 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 613 op/s 2024-03-26T19:00:08.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:08 smithi144 ceph-mon[36053]: pgmap v426: 97 pgs: 97 active+clean; 977 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 613 op/s 2024-03-26T19:00:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 45.0 s, 2577.3 tps, lat 38.754 ms stddev 42.826 2024-03-26T19:00:10.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:10 smithi105 ceph-mon[36270]: pgmap v427: 97 pgs: 97 active+clean; 977 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 782 op/s 2024-03-26T19:00:10.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:10 smithi097 ceph-mon[31955]: pgmap v427: 97 pgs: 97 active+clean; 977 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 782 op/s 2024-03-26T19:00:10.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:10 smithi144 ceph-mon[36053]: pgmap v427: 97 pgs: 97 active+clean; 977 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 782 op/s 2024-03-26T19:00:12.440 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:12 smithi144 ceph-mon[36053]: pgmap v428: 97 pgs: 97 active+clean; 977 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 700 op/s 2024-03-26T19:00:12.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:12 smithi105 ceph-mon[36270]: pgmap v428: 97 pgs: 97 active+clean; 977 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 700 op/s 2024-03-26T19:00:12.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:12 smithi097 ceph-mon[31955]: pgmap v428: 97 pgs: 97 active+clean; 977 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 700 op/s 2024-03-26T19:00:14.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:14 smithi105 ceph-mon[36270]: pgmap v429: 97 pgs: 97 active+clean; 978 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 645 op/s 2024-03-26T19:00:14.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:14 smithi097 ceph-mon[31955]: pgmap v429: 97 pgs: 97 active+clean; 978 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 645 op/s 2024-03-26T19:00:14.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:14 smithi144 ceph-mon[36053]: pgmap v429: 97 pgs: 97 active+clean; 978 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 645 op/s 2024-03-26T19:00:15.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 50.0 s, 2515.4 tps, lat 39.726 ms stddev 43.901 2024-03-26T19:00:16.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:16 smithi105 ceph-mon[36270]: pgmap v430: 97 pgs: 97 active+clean; 982 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 776 op/s 2024-03-26T19:00:16.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:16 smithi097 ceph-mon[31955]: pgmap v430: 97 pgs: 97 active+clean; 982 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 776 op/s 2024-03-26T19:00:16.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:16 smithi144 ceph-mon[36053]: pgmap v430: 97 pgs: 97 active+clean; 982 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 776 op/s 2024-03-26T19:00:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.810+0000 7fc22b400640 1 -- 172.21.15.97:0/3676916917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc224077e60 msgr2=0x7fc224078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.810+0000 7fc22b400640 1 --2- 172.21.15.97:0/3676916917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc224077e60 0x7fc224078700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc214009860 tx=0x7fc21402bb70 comp rx=0 tx=0).stop 2024-03-26T19:00:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.811+0000 7fc22b400640 1 -- 172.21.15.97:0/3676916917 shutdown_connections 2024-03-26T19:00:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.811+0000 7fc22b400640 1 --2- 172.21.15.97:0/3676916917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc224077e60 0x7fc224078700 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.811+0000 7fc22b400640 1 --2- 172.21.15.97:0/3676916917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2240774c0 0x7fc224077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.811+0000 7fc22b400640 1 --2- 172.21.15.97:0/3676916917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc224076280 0x7fc2240766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.811+0000 7fc22b400640 1 -- 172.21.15.97:0/3676916917 >> 172.21.15.97:0/3676916917 conn(0x7fc2240ff080 msgr2=0x7fc2241014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.811+0000 7fc22b400640 1 -- 172.21.15.97:0/3676916917 shutdown_connections 2024-03-26T19:00:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.811+0000 7fc22b400640 1 -- 172.21.15.97:0/3676916917 wait complete. 2024-03-26T19:00:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.813+0000 7fc22b400640 1 Processor -- start 2024-03-26T19:00:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.813+0000 7fc22b400640 1 -- start start 2024-03-26T19:00:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.813+0000 7fc22b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc224076280 0x7fc22419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.813+0000 7fc22b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2240774c0 0x7fc22419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc22b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc224077e60 0x7fc2241a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc22b400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc224113ba0 con 0x7fc224077e60 2024-03-26T19:00:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc22b400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc224113a20 con 0x7fc2240774c0 2024-03-26T19:00:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc22b400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc224113d20 con 0x7fc224076280 2024-03-26T19:00:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc228a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc224076280 0x7fc22419e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc228a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc224076280 0x7fc22419e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:40054/0 (socket says 172.21.15.97:40054) 2024-03-26T19:00:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc228a00640 1 -- 172.21.15.97:0/2301421050 learned_addr learned my addr 172.21.15.97:0/2301421050 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc223e00640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2240774c0 0x7fc22419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc229400640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc224077e60 0x7fc2241a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc228a00640 1 -- 172.21.15.97:0/2301421050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2240774c0 msgr2=0x7fc22419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc228a00640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2240774c0 0x7fc22419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc228a00640 1 -- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc224077e60 msgr2=0x7fc2241a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc228a00640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc224077e60 0x7fc2241a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc228a00640 1 -- 172.21.15.97:0/2301421050 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2241a5d00 con 0x7fc224076280 2024-03-26T19:00:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.814+0000 7fc229400640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc224077e60 0x7fc2241a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:00:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.815+0000 7fc228a00640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc224076280 0x7fc22419e0e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00efb0 tx=0x7fc21c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.815+0000 7fc221600640 1 -- 172.21.15.97:0/2301421050 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c01b070 con 0x7fc224076280 2024-03-26T19:00:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.815+0000 7fc221600640 1 -- 172.21.15.97:0/2301421050 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc21c00ba50 con 0x7fc224076280 2024-03-26T19:00:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.815+0000 7fc22b400640 1 -- 172.21.15.97:0/2301421050 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2241a5ff0 con 0x7fc224076280 2024-03-26T19:00:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.815+0000 7fc221600640 1 -- 172.21.15.97:0/2301421050 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c016c00 con 0x7fc224076280 2024-03-26T19:00:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.815+0000 7fc22b400640 1 -- 172.21.15.97:0/2301421050 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2241a6450 con 0x7fc224076280 2024-03-26T19:00:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.820+0000 7fc221600640 1 -- 172.21.15.97:0/2301421050 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fc21c01e070 con 0x7fc224076280 2024-03-26T19:00:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.822+0000 7fc22e1d2740 1 Processor -- start 2024-03-26T19:00:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.822+0000 7fc22e1d2740 1 -- start start 2024-03-26T19:00:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.822+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fccdb0 0x55ca61fcd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.822+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fcd7a0 0x55ca61fd22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.822+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 0x55ca61fd4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.822+0000 7fc22e1d2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ca61f20e80 con 0x55ca61fcd7a0 2024-03-26T19:00:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.822+0000 7fc22e1d2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ca61ec9660 con 0x55ca61fccdb0 2024-03-26T19:00:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.822+0000 7fc22e1d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca61f1ce30 con 0x55ca61fd2910 2024-03-26T19:00:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.823+0000 7fc221600640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2080a3130 0x7fc2080a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.823+0000 7fc229400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fccdb0 0x55ca61fcd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.823+0000 7fc221600640 1 -- 172.21.15.97:0/2301421050 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc21c0cf440 con 0x7fc224076280 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.823+0000 7fc229400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fccdb0 0x55ca61fcd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:39546/0 (socket says 172.21.15.97:39546) 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.823+0000 7fc229400640 1 -- 172.21.15.97:0/4271825782 learned_addr learned my addr 172.21.15.97:0/4271825782 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.823+0000 7fc229400640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2080a3130 0x7fc2080a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.823+0000 7fc228a00640 1 --2- 172.21.15.97:0/4271825782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fcd7a0 0x55ca61fd22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.824+0000 7fc223e00640 1 --2- 172.21.15.97:0/4271825782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 0x55ca61fd4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.824+0000 7fc229400640 1 -- 172.21.15.97:0/4271825782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 msgr2=0x55ca61fd4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.824+0000 7fc229400640 1 --2- 172.21.15.97:0/4271825782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 0x55ca61fd4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.824+0000 7fc229400640 1 -- 172.21.15.97:0/4271825782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fcd7a0 msgr2=0x55ca61fd22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.824+0000 7fc229400640 1 --2- 172.21.15.97:0/4271825782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fcd7a0 0x55ca61fd22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.824+0000 7fc229400640 1 -- 172.21.15.97:0/4271825782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca61c35210 con 0x55ca61fccdb0 2024-03-26T19:00:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.824+0000 7fc223e00640 1 --2- 172.21.15.97:0/4271825782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 0x55ca61fd4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:00:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.824+0000 7fc229400640 1 --2- 172.21.15.97:0/4271825782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fccdb0 0x55ca61fcd1d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc214034b10 tx=0x7fc214035f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7322148038596cfd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.825+0000 7fc219c00640 1 -- 172.21.15.97:0/4271825782 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214039040 con 0x55ca61fccdb0 2024-03-26T19:00:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.825+0000 7fc219c00640 1 -- 172.21.15.97:0/4271825782 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc214005520 con 0x55ca61fccdb0 2024-03-26T19:00:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.825+0000 7fc219c00640 1 -- 172.21.15.97:0/4271825782 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214039040 con 0x55ca61fccdb0 2024-03-26T19:00:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.826+0000 7fc22e1d2740 1 -- 172.21.15.97:0/4271825782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fccdb0 msgr2=0x55ca61fcd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.826+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/4271825782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fccdb0 0x55ca61fcd1d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc214034b10 tx=0x7fc214035f50 comp rx=0 tx=0).stop 2024-03-26T19:00:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.826+0000 7fc229400640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2080a3130 0x7fc2080a55f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc214005e00 tx=0x7fc214049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.826+0000 7fc22e1d2740 1 -- 172.21.15.97:0/4271825782 shutdown_connections 2024-03-26T19:00:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.826+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/4271825782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 0x55ca61fd4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.826+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/4271825782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fcd7a0 0x55ca61fd22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.826+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/4271825782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fccdb0 0x55ca61fcd1d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.826+0000 7fc22e1d2740 1 -- 172.21.15.97:0/4271825782 >> 172.21.15.97:0/4271825782 conn(0x55ca61fb7310 msgr2=0x55ca61fb7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.826+0000 7fc22e1d2740 1 -- 172.21.15.97:0/4271825782 shutdown_connections 2024-03-26T19:00:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.826+0000 7fc22e1d2740 1 -- 172.21.15.97:0/4271825782 wait complete. 2024-03-26T19:00:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.827+0000 7fc22e1d2740 1 -- start start 2024-03-26T19:00:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.827+0000 7fc22e1d2740 1 Processor -- start 2024-03-26T19:00:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.828+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 0x55ca61fd6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.828+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fcd7a0 0x55ca61fd7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.828+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fd2910 0x55ca61fde270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.828+0000 7fc22e1d2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ca61fd58c0 con 0x55ca61fccdb0 2024-03-26T19:00:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.828+0000 7fc22e1d2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ca61edcac0 con 0x55ca61fd2910 2024-03-26T19:00:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.828+0000 7fc22e1d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca61fd5a40 con 0x55ca61fcd7a0 2024-03-26T19:00:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.828+0000 7fc22e1d2740 10 client.0 _ll_register_callbacks cb 0x55ca61edc490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:00:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.828+0000 7fc223e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fcd7a0 0x55ca61fd7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.828+0000 7fc223e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fcd7a0 0x55ca61fd7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:40076/0 (socket says 172.21.15.97:40076) 2024-03-26T19:00:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.828+0000 7fc223e00640 1 -- 172.21.15.97:0/219781663 learned_addr learned my addr 172.21.15.97:0/219781663 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.829+0000 7fc223e00640 1 -- 172.21.15.97:0/219781663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fd2910 msgr2=0x55ca61fde270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.829+0000 7fc223e00640 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fd2910 0x55ca61fde270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.829+0000 7fc223e00640 1 -- 172.21.15.97:0/219781663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 msgr2=0x55ca61fd6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.829+0000 7fc228a00640 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 0x55ca61fd6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.829+0000 7fc223e00640 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 0x55ca61fd6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.829+0000 7fc223e00640 1 -- 172.21.15.97:0/219781663 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca61d01620 con 0x55ca61fcd7a0 2024-03-26T19:00:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.829+0000 7fc223e00640 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fcd7a0 0x55ca61fd7230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0136e0 tx=0x7fc20c013ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa298998f16dccf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.829+0000 7fc212000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:00:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.830+0000 7fc212000640 1 -- 172.21.15.97:0/219781663 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20c011960 con 0x55ca61fcd7a0 2024-03-26T19:00:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.830+0000 7fc212000640 1 -- 172.21.15.97:0/219781663 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc20c011b00 con 0x55ca61fcd7a0 2024-03-26T19:00:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.830+0000 7fc22e1d2740 1 -- 172.21.15.97:0/219781663 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc214005520 con 0x55ca61fcd7a0 2024-03-26T19:00:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.830+0000 7fc212000640 1 -- 172.21.15.97:0/219781663 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc20c011d00 con 0x55ca61fcd7a0 2024-03-26T19:00:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.831+0000 7fc22e1d2740 10 client.25342 fetch_fsmap learned FSMap version 27 2024-03-26T19:00:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.831+0000 7fc22e1d2740 1 -- 172.21.15.97:0/219781663 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca61fe1650 con 0x55ca61fcd7a0 2024-03-26T19:00:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.831+0000 7fc212000640 1 -- 172.21.15.97:0/219781663 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc214005520 con 0x55ca61fcd7a0 2024-03-26T19:00:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.832+0000 7fc212000640 1 -- 172.21.15.97:0/219781663 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc20c01a070 con 0x55ca61fcd7a0 2024-03-26T19:00:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.832+0000 7fc22e1d2740 10 client.25342 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:00:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.832+0000 7fc22e1d2740 10 client.25342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:00:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.832+0000 7fc22e1d2740 20 client.25342 populate_metadata read hostname 'smithi097' 2024-03-26T19:00:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.832+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ca61fe2e50 0x55ca61fe5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.832+0000 7fc22e1d2740 4 client.25342 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:00:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.832+0000 7fc22e1d2740 1 -- 172.21.15.97:0/219781663 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca61f2e3c0 con 0x55ca61fe2e50 2024-03-26T19:00:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.832+0000 7fc228a00640 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ca61fe2e50 0x55ca61fe5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.833+0000 7fc228a00640 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ca61fe2e50 0x55ca61fe5310 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abcef532914e7585 server_cookie=76f4643a71f0aaac in_seq=0 out_seq=0 2024-03-26T19:00:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.833+0000 7fc212000640 10 client.25342 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.834+0000 7fc212000640 1 -- 172.21.15.97:0/219781663 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ca61f2e3c0 con 0x55ca61fe2e50 2024-03-26T19:00:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.834+0000 7fc212000640 10 client.25342 handle_command_reply: tid=0 2024-03-26T19:00:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.834+0000 7fc22e1d2740 1 client.25342 shutdown 2024-03-26T19:00:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.835+0000 7fc22e1d2740 1 -- 172.21.15.97:0/219781663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fcd7a0 msgr2=0x55ca61fd7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.835+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fcd7a0 0x55ca61fd7230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0136e0 tx=0x7fc20c013ba0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.835+0000 7fc22e1d2740 1 -- 172.21.15.97:0/219781663 shutdown_connections 2024-03-26T19:00:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.835+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ca61fe2e50 0x55ca61fe5310 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.835+0000 7fc212000640 0 client.25342 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.836+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fd2910 0x55ca61fde270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.836+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fcd7a0 0x55ca61fd7230 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.836+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/219781663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 0x55ca61fd6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.836+0000 7fc22e1d2740 1 -- 172.21.15.97:0/219781663 >> 172.21.15.97:0/219781663 conn(0x55ca61fb7310 msgr2=0x55ca61fb6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.836+0000 7fc22e1d2740 1 -- 172.21.15.97:0/219781663 shutdown_connections 2024-03-26T19:00:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.837+0000 7fc22e1d2740 1 -- 172.21.15.97:0/219781663 wait complete. 2024-03-26T19:00:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.837+0000 7fc22e1d2740 20 client.25342 trim_cache size 0 max 16384 2024-03-26T19:00:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.864+0000 7fc22e1d2740 1 Processor -- start 2024-03-26T19:00:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.865+0000 7fc22e1d2740 1 -- start start 2024-03-26T19:00:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.865+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fccdb0 0x55ca61fb5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.865+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 0x55ca61fde230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.865+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fd2910 0x55ca61fb61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.865+0000 7fc22e1d2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc20000e3b0 con 0x55ca61fd2910 2024-03-26T19:00:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.865+0000 7fc22e1d2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc20000fef0 con 0x55ca61fcd7a0 2024-03-26T19:00:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.865+0000 7fc22e1d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc20000ea80 con 0x55ca61fccdb0 2024-03-26T19:00:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.866+0000 7fc229400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fd2910 0x55ca61fb61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.866+0000 7fc228a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fccdb0 0x55ca61fb5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.866+0000 7fc229400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fd2910 0x55ca61fb61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32936/0 (socket says 172.21.15.97:32936) 2024-03-26T19:00:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.866+0000 7fc223e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 0x55ca61fde230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.866+0000 7fc229400640 1 -- 172.21.15.97:0/1514406977 learned_addr learned my addr 172.21.15.97:0/1514406977 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.867+0000 7fc229400640 1 -- 172.21.15.97:0/1514406977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fccdb0 msgr2=0x55ca61fb5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.867+0000 7fc229400640 1 --2- 172.21.15.97:0/1514406977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fccdb0 0x55ca61fb5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.867+0000 7fc229400640 1 -- 172.21.15.97:0/1514406977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 msgr2=0x55ca61fde230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.867+0000 7fc229400640 1 --2- 172.21.15.97:0/1514406977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 0x55ca61fde230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.867+0000 7fc229400640 1 -- 172.21.15.97:0/1514406977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca61fb8080 con 0x55ca61fd2910 2024-03-26T19:00:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.867+0000 7fc229400640 1 --2- 172.21.15.97:0/1514406977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fd2910 0x55ca61fb61f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc2140349e0 tx=0x7fc21403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205f333692ba55c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.868+0000 7fc219200640 1 -- 172.21.15.97:0/1514406977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214058020 con 0x55ca61fd2910 2024-03-26T19:00:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.868+0000 7fc219200640 1 -- 172.21.15.97:0/1514406977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc214005520 con 0x55ca61fd2910 2024-03-26T19:00:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.868+0000 7fc219200640 1 -- 172.21.15.97:0/1514406977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214039040 con 0x55ca61fd2910 2024-03-26T19:00:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.869+0000 7fc22e1d2740 1 -- 172.21.15.97:0/1514406977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fd2910 msgr2=0x55ca61fb61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.870+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/1514406977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fd2910 0x55ca61fb61f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc2140349e0 tx=0x7fc21403c000 comp rx=0 tx=0).stop 2024-03-26T19:00:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.870+0000 7fc22e1d2740 1 -- 172.21.15.97:0/1514406977 shutdown_connections 2024-03-26T19:00:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.870+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/1514406977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fd2910 0x55ca61fb61f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.870+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/1514406977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 0x55ca61fde230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.870+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/1514406977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fccdb0 0x55ca61fb5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.870+0000 7fc22e1d2740 1 -- 172.21.15.97:0/1514406977 >> 172.21.15.97:0/1514406977 conn(0x55ca61fb7310 msgr2=0x55ca61fb76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.870+0000 7fc22e1d2740 1 -- 172.21.15.97:0/1514406977 shutdown_connections 2024-03-26T19:00:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.870+0000 7fc22e1d2740 1 -- 172.21.15.97:0/1514406977 wait complete. 2024-03-26T19:00:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.871+0000 7fc22e1d2740 1 -- start start 2024-03-26T19:00:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.871+0000 7fc22e1d2740 1 Processor -- start 2024-03-26T19:00:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.872+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 0x55ca61fdb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.872+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 0x55ca61fdbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.872+0000 7fc22e1d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 0x55ca61fdcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.872+0000 7fc228a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 0x55ca61fdb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.872+0000 7fc223e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 0x55ca61fdbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.872+0000 7fc223e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 0x55ca61fdbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:39574/0 (socket says 172.21.15.97:39574) 2024-03-26T19:00:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc22e1d2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc20000f820 con 0x55ca61fccdb0 2024-03-26T19:00:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc223e00640 1 -- 172.21.15.97:0/2222400660 learned_addr learned my addr 172.21.15.97:0/2222400660 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc22e1d2740 1 -- 172.21.15.97:0/2222400660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc20000f150 con 0x55ca61fcd7a0 2024-03-26T19:00:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc22e1d2740 1 -- 172.21.15.97:0/2222400660 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc20000c390 con 0x55ca61fd2910 2024-03-26T19:00:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc229400640 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 0x55ca61fdcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc22e1d2740 10 client.0 _ll_register_callbacks cb 0x7fc20c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:00:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc223e00640 1 -- 172.21.15.97:0/2222400660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 msgr2=0x55ca61fdcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc223e00640 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 0x55ca61fdcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc223e00640 1 -- 172.21.15.97:0/2222400660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 msgr2=0x55ca61fdb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc223e00640 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 0x55ca61fdb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.873+0000 7fc223e00640 1 -- 172.21.15.97:0/2222400660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca61fdd320 con 0x55ca61fcd7a0 2024-03-26T19:00:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.874+0000 7fc229400640 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 0x55ca61fdcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:00:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.874+0000 7fc228a00640 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 0x55ca61fdb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:00:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.874+0000 7fc223e00640 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 0x55ca61fdbd90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55ca61fb8250 tx=0x7fc20c005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac109394386dd48 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.874+0000 7fc212000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:00:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.875+0000 7fc212000640 1 -- 172.21.15.97:0/2222400660 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20c026b40 con 0x55ca61fcd7a0 2024-03-26T19:00:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.875+0000 7fc212000640 1 -- 172.21.15.97:0/2222400660 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc20c026ce0 con 0x55ca61fcd7a0 2024-03-26T19:00:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.875+0000 7fc22e1d2740 1 -- 172.21.15.97:0/2222400660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc214005520 con 0x55ca61fcd7a0 2024-03-26T19:00:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.875+0000 7fc212000640 1 -- 172.21.15.97:0/2222400660 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc20c0035f0 con 0x55ca61fcd7a0 2024-03-26T19:00:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.875+0000 7fc22e1d2740 10 client.25382 fetch_fsmap learned FSMap version 27 2024-03-26T19:00:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.876+0000 7fc22e1d2740 1 -- 172.21.15.97:0/2222400660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca62049250 con 0x55ca61fcd7a0 2024-03-26T19:00:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.876+0000 7fc212000640 1 -- 172.21.15.97:0/2222400660 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc214005520 con 0x55ca61fcd7a0 2024-03-26T19:00:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.876+0000 7fc212000640 1 -- 172.21.15.97:0/2222400660 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc20c01a040 con 0x55ca61fcd7a0 2024-03-26T19:00:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.876+0000 7fc22e1d2740 10 client.25382 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:00:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.876+0000 7fc22e1d2740 10 client.25382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:00:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.876+0000 7fc22e1d2740 20 client.25382 populate_metadata read hostname 'smithi097' 2024-03-26T19:00:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.876+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ca61fe2e50 0x55ca6204baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.876+0000 7fc22e1d2740 4 client.25382 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:00:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.876+0000 7fc22e1d2740 1 -- 172.21.15.97:0/2222400660 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca61f28710 con 0x55ca61fe2e50 2024-03-26T19:00:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.877+0000 7fc228a00640 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ca61fe2e50 0x55ca6204baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.877+0000 7fc228a00640 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ca61fe2e50 0x55ca6204baa0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5930fc0bdb87bc server_cookie=29fef0eb98fad4f3 in_seq=0 out_seq=0 2024-03-26T19:00:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.877+0000 7fc212000640 10 client.25382 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.909+0000 7fc212000640 1 -- 172.21.15.97:0/2222400660 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca61f28710 con 0x55ca61fe2e50 2024-03-26T19:00:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.909+0000 7fc212000640 10 client.25382 handle_command_reply: tid=0 2024-03-26T19:00:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.909+0000 7fc22e1d2740 1 client.25382 shutdown 2024-03-26T19:00:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.909+0000 7fc22e1d2740 1 -- 172.21.15.97:0/2222400660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 msgr2=0x55ca61fdbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.909+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 0x55ca61fdbd90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55ca61fb8250 tx=0x7fc20c005750 comp rx=0 tx=0).stop 2024-03-26T19:00:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.910+0000 7fc22e1d2740 1 -- 172.21.15.97:0/2222400660 shutdown_connections 2024-03-26T19:00:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.910+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ca61fe2e50 0x55ca6204baa0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.910+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca61fd2910 0x55ca61fdcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.910+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca61fcd7a0 0x55ca61fdbd90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.910+0000 7fc22e1d2740 1 --2- 172.21.15.97:0/2222400660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca61fccdb0 0x55ca61fdb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.910+0000 7fc22e1d2740 1 -- 172.21.15.97:0/2222400660 >> 172.21.15.97:0/2222400660 conn(0x55ca61fb7310 msgr2=0x55ca61fba570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.910+0000 7fc212000640 0 client.25382 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.910+0000 7fc22e1d2740 1 -- 172.21.15.97:0/2222400660 shutdown_connections 2024-03-26T19:00:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.911+0000 7fc22e1d2740 1 -- 172.21.15.97:0/2222400660 wait complete. 2024-03-26T19:00:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.911+0000 7fc22e1d2740 20 client.25382 trim_cache size 0 max 16384 2024-03-26T19:00:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:00:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:00:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:00:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:00:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 -- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2080a3130 msgr2=0x7fc2080a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2080a3130 0x7fc2080a55f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc214005e00 tx=0x7fc214049000 comp rx=0 tx=0).stop 2024-03-26T19:00:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 -- 172.21.15.97:0/2301421050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc224076280 msgr2=0x7fc22419e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc224076280 0x7fc22419e0e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00efb0 tx=0x7fc21c00c800 comp rx=0 tx=0).stop 2024-03-26T19:00:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 -- 172.21.15.97:0/2301421050 shutdown_connections 2024-03-26T19:00:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc2080a3130 0x7fc2080a55f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc224077e60 0x7fc2241a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2240774c0 0x7fc22419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 --2- 172.21.15.97:0/2301421050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc224076280 0x7fc22419e0e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 -- 172.21.15.97:0/2301421050 >> 172.21.15.97:0/2301421050 conn(0x7fc2240ff080 msgr2=0x7fc224100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 -- 172.21.15.97:0/2301421050 shutdown_connections 2024-03-26T19:00:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:16.912+0000 7fc22b400640 1 -- 172.21.15.97:0/2301421050 wait complete. 2024-03-26T19:00:18.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:18 smithi105 ceph-mon[36270]: pgmap v431: 97 pgs: 97 active+clean; 986 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.8 MiB/s wr, 640 op/s 2024-03-26T19:00:18.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:18 smithi097 ceph-mon[31955]: pgmap v431: 97 pgs: 97 active+clean; 986 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.8 MiB/s wr, 640 op/s 2024-03-26T19:00:18.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:18 smithi144 ceph-mon[36053]: pgmap v431: 97 pgs: 97 active+clean; 986 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.8 MiB/s wr, 640 op/s 2024-03-26T19:00:20.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 55.0 s, 2968.4 tps, lat 33.615 ms stddev 39.872 2024-03-26T19:00:20.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:20 smithi105 ceph-mon[36270]: pgmap v432: 97 pgs: 97 active+clean; 995 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 733 op/s 2024-03-26T19:00:20.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:20 smithi097 ceph-mon[31955]: pgmap v432: 97 pgs: 97 active+clean; 995 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 733 op/s 2024-03-26T19:00:20.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:20 smithi144 ceph-mon[36053]: pgmap v432: 97 pgs: 97 active+clean; 995 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 733 op/s 2024-03-26T19:00:22.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:22 smithi105 ceph-mon[36270]: pgmap v433: 97 pgs: 97 active+clean; 995 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.3 MiB/s wr, 726 op/s 2024-03-26T19:00:22.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:22 smithi097 ceph-mon[31955]: pgmap v433: 97 pgs: 97 active+clean; 995 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.3 MiB/s wr, 726 op/s 2024-03-26T19:00:22.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:22 smithi144 ceph-mon[36053]: pgmap v433: 97 pgs: 97 active+clean; 995 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.3 MiB/s wr, 726 op/s 2024-03-26T19:00:24.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:24 smithi097 ceph-mon[31955]: pgmap v434: 97 pgs: 97 active+clean; 995 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 750 op/s 2024-03-26T19:00:24.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:24 smithi144 ceph-mon[36053]: pgmap v434: 97 pgs: 97 active+clean; 995 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 750 op/s 2024-03-26T19:00:24.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:24 smithi105 ceph-mon[36270]: pgmap v434: 97 pgs: 97 active+clean; 995 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 750 op/s 2024-03-26T19:00:25.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 60.0 s, 2498.6 tps, lat 40.051 ms stddev 48.626 2024-03-26T19:00:26.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:26 smithi097 ceph-mon[31955]: pgmap v435: 97 pgs: 97 active+clean; 996 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.9 MiB/s wr, 775 op/s 2024-03-26T19:00:26.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:26 smithi144 ceph-mon[36053]: pgmap v435: 97 pgs: 97 active+clean; 996 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.9 MiB/s wr, 775 op/s 2024-03-26T19:00:26.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:26 smithi105 ceph-mon[36270]: pgmap v435: 97 pgs: 97 active+clean; 996 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.9 MiB/s wr, 775 op/s 2024-03-26T19:00:28.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:28 smithi097 ceph-mon[31955]: pgmap v436: 97 pgs: 97 active+clean; 996 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 630 op/s 2024-03-26T19:00:28.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:28 smithi144 ceph-mon[36053]: pgmap v436: 97 pgs: 97 active+clean; 996 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 630 op/s 2024-03-26T19:00:28.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:28 smithi105 ceph-mon[36270]: pgmap v436: 97 pgs: 97 active+clean; 996 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 630 op/s 2024-03-26T19:00:29.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:29 smithi097 ceph-mon[31955]: pgmap v437: 97 pgs: 97 active+clean; 1005 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.1 MiB/s wr, 793 op/s 2024-03-26T19:00:29.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:29 smithi144 ceph-mon[36053]: pgmap v437: 97 pgs: 97 active+clean; 1005 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.1 MiB/s wr, 793 op/s 2024-03-26T19:00:29.710 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:29 smithi105 ceph-mon[36270]: pgmap v437: 97 pgs: 97 active+clean; 1005 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.1 MiB/s wr, 793 op/s 2024-03-26T19:00:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 65.0 s, 2299.4 tps, lat 43.358 ms stddev 47.911 2024-03-26T19:00:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.104+0000 7ffa25000640 1 -- 172.21.15.97:0/490967012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa20102ae0 msgr2=0x7ffa20102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.104+0000 7ffa25000640 1 --2- 172.21.15.97:0/490967012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa20102ae0 0x7ffa20102f60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009900 tx=0x7ffa0c02bc30 comp rx=0 tx=0).stop 2024-03-26T19:00:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.105+0000 7ffa25000640 1 -- 172.21.15.97:0/490967012 shutdown_connections 2024-03-26T19:00:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.105+0000 7ffa25000640 1 --2- 172.21.15.97:0/490967012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa201034a0 0x7ffa20109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.105+0000 7ffa25000640 1 --2- 172.21.15.97:0/490967012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa20102ae0 0x7ffa20102f60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.105+0000 7ffa25000640 1 --2- 172.21.15.97:0/490967012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa201018f0 0x7ffa20101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.105+0000 7ffa25000640 1 -- 172.21.15.97:0/490967012 >> 172.21.15.97:0/490967012 conn(0x7ffa200fce80 msgr2=0x7ffa200ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.105+0000 7ffa25000640 1 -- 172.21.15.97:0/490967012 shutdown_connections 2024-03-26T19:00:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.105+0000 7ffa25000640 1 -- 172.21.15.97:0/490967012 wait complete. 2024-03-26T19:00:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.106+0000 7ffa25000640 1 Processor -- start 2024-03-26T19:00:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.107+0000 7ffa25000640 1 -- start start 2024-03-26T19:00:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.108+0000 7ffa25000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa201018f0 0x7ffa2019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.108+0000 7ffa25000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa20102ae0 0x7ffa2019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.108+0000 7ffa25000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa201034a0 0x7ffa201a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.108+0000 7ffa25000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2010d5a0 con 0x7ffa20102ae0 2024-03-26T19:00:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.108+0000 7ffa25000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2010d420 con 0x7ffa201018f0 2024-03-26T19:00:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.108+0000 7ffa25000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2010d720 con 0x7ffa201034a0 2024-03-26T19:00:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa201018f0 0x7ffa2019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa201034a0 0x7ffa201a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa201018f0 0x7ffa2019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:40246/0 (socket says 172.21.15.97:40246) 2024-03-26T19:00:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1de00640 1 -- 172.21.15.97:0/397297385 learned_addr learned my addr 172.21.15.97:0/397297385 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1d400640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa20102ae0 0x7ffa2019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1de00640 1 -- 172.21.15.97:0/397297385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa201034a0 msgr2=0x7ffa201a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1de00640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa201034a0 0x7ffa201a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1de00640 1 -- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa20102ae0 msgr2=0x7ffa2019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1de00640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa20102ae0 0x7ffa2019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1de00640 1 -- 172.21.15.97:0/397297385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa201a5d70 con 0x7ffa201018f0 2024-03-26T19:00:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.109+0000 7ffa1d400640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa20102ae0 0x7ffa2019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:00:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.110+0000 7ffa1de00640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa201018f0 0x7ffa2019e150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9c0 tx=0x7ffa1000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.110+0000 7ffa16a00640 1 -- 172.21.15.97:0/397297385 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1000cd80 con 0x7ffa201018f0 2024-03-26T19:00:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.110+0000 7ffa16a00640 1 -- 172.21.15.97:0/397297385 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa10010910 con 0x7ffa201018f0 2024-03-26T19:00:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.110+0000 7ffa16a00640 1 -- 172.21.15.97:0/397297385 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10018960 con 0x7ffa201018f0 2024-03-26T19:00:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.110+0000 7ffa25000640 1 -- 172.21.15.97:0/397297385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa201a6060 con 0x7ffa201018f0 2024-03-26T19:00:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.111+0000 7ffa25000640 1 -- 172.21.15.97:0/397297385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa201a6530 con 0x7ffa201018f0 2024-03-26T19:00:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.115+0000 7ffa16a00640 1 -- 172.21.15.97:0/397297385 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7ffa1001f020 con 0x7ffa201018f0 2024-03-26T19:00:30.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.117+0000 7ffa27dee740 1 Processor -- start 2024-03-26T19:00:30.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.117+0000 7ffa27dee740 1 -- start start 2024-03-26T19:00:30.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.117+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d662491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa16a00640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ffa040a31b0 0x7ffa040a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d6624e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 0x556d66250d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa27dee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556d661ade80 con 0x556d66249790 2024-03-26T19:00:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa27dee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556d66156660 con 0x556d66248da0 2024-03-26T19:00:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa27dee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d661a9e30 con 0x556d6624e900 2024-03-26T19:00:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d662491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa16a00640 1 -- 172.21.15.97:0/397297385 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ffa100cf4f0 con 0x7ffa201018f0 2024-03-26T19:00:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d662491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:40248/0 (socket says 172.21.15.97:40248) 2024-03-26T19:00:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1e800640 1 -- 172.21.15.97:0/4123678635 learned_addr learned my addr 172.21.15.97:0/4123678635 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1e800640 1 --2- 172.21.15.97:0/4123678635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 0x556d66250d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1d400640 1 --2- 172.21.15.97:0/4123678635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d6624e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1e800640 1 -- 172.21.15.97:0/4123678635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 msgr2=0x556d66250d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1e800640 1 --2- 172.21.15.97:0/4123678635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 0x556d66250d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1e800640 1 -- 172.21.15.97:0/4123678635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 msgr2=0x556d6624e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1e800640 1 --2- 172.21.15.97:0/4123678635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d6624e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1e800640 1 -- 172.21.15.97:0/4123678635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d65ec2210 con 0x556d66248da0 2024-03-26T19:00:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.118+0000 7ffa1d400640 1 --2- 172.21.15.97:0/4123678635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d6624e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:00:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.119+0000 7ffa1de00640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ffa040a31b0 0x7ffa040a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.119+0000 7ffa1e800640 1 --2- 172.21.15.97:0/4123678635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d662491c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009b80 tx=0x7ffa18009af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92235009502c60b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.119+0000 7ffa0b400640 1 -- 172.21.15.97:0/4123678635 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1800fc20 con 0x556d66248da0 2024-03-26T19:00:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.119+0000 7ffa0b400640 1 -- 172.21.15.97:0/4123678635 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa18003e50 con 0x556d66248da0 2024-03-26T19:00:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.119+0000 7ffa0b400640 1 -- 172.21.15.97:0/4123678635 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa180044f0 con 0x556d66248da0 2024-03-26T19:00:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.121+0000 7ffa27dee740 1 -- 172.21.15.97:0/4123678635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 msgr2=0x556d662491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.121+0000 7ffa1de00640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ffa040a31b0 0x7ffa040a5670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa1009bb80 tx=0x7ffa1000cf20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.121+0000 7ffa27dee740 1 --2- 172.21.15.97:0/4123678635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d662491c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009b80 tx=0x7ffa18009af0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.122+0000 7ffa27dee740 1 -- 172.21.15.97:0/4123678635 shutdown_connections 2024-03-26T19:00:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.122+0000 7ffa27dee740 1 --2- 172.21.15.97:0/4123678635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 0x556d66250d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.122+0000 7ffa27dee740 1 --2- 172.21.15.97:0/4123678635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d6624e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.122+0000 7ffa27dee740 1 --2- 172.21.15.97:0/4123678635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d662491c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.122+0000 7ffa27dee740 1 -- 172.21.15.97:0/4123678635 >> 172.21.15.97:0/4123678635 conn(0x556d66244310 msgr2=0x556d66244710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.123+0000 7ffa27dee740 1 -- 172.21.15.97:0/4123678635 shutdown_connections 2024-03-26T19:00:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.123+0000 7ffa27dee740 1 -- 172.21.15.97:0/4123678635 wait complete. 2024-03-26T19:00:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.124+0000 7ffa27dee740 1 -- start start 2024-03-26T19:00:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.124+0000 7ffa27dee740 1 Processor -- start 2024-03-26T19:00:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.125+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 0x556d66252dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.125+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d66253310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.125+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d6624e900 0x556d6625a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.125+0000 7ffa27dee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556d662518b0 con 0x556d66249790 2024-03-26T19:00:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.125+0000 7ffa27dee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556d66169ac0 con 0x556d6624e900 2024-03-26T19:00:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.125+0000 7ffa27dee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d66251a30 con 0x556d66248da0 2024-03-26T19:00:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.125+0000 7ffa27dee740 10 client.0 _ll_register_callbacks cb 0x556d66169490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:00:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.126+0000 7ffa1e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d6624e900 0x556d6625a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.126+0000 7ffa1e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 0x556d66252dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.126+0000 7ffa1e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d6624e900 0x556d6625a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:40262/0 (socket says 172.21.15.97:40262) 2024-03-26T19:00:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.126+0000 7ffa1e800640 1 -- 172.21.15.97:0/775529051 learned_addr learned my addr 172.21.15.97:0/775529051 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.126+0000 7ffa1d400640 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d66253310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.127+0000 7ffa1e800640 1 -- 172.21.15.97:0/775529051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 msgr2=0x556d66252dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.127+0000 7ffa1e800640 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 0x556d66252dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.127+0000 7ffa1e800640 1 -- 172.21.15.97:0/775529051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 msgr2=0x556d66253310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.127+0000 7ffa1e800640 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d66253310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.127+0000 7ffa1e800640 1 -- 172.21.15.97:0/775529051 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d65f8e620 con 0x556d6624e900 2024-03-26T19:00:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.127+0000 7ffa1d400640 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d66253310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:00:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.127+0000 7ffa1e800640 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d6624e900 0x556d6625a350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa18005fd0 tx=0x7ffa18009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b88cb52a5963e847 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.127+0000 7ff9ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:00:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.128+0000 7ff9ffe00640 1 -- 172.21.15.97:0/775529051 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18026030 con 0x556d6624e900 2024-03-26T19:00:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.128+0000 7ff9ffe00640 1 -- 172.21.15.97:0/775529051 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ffa18011220 con 0x556d6624e900 2024-03-26T19:00:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.128+0000 7ffa27dee740 1 -- 172.21.15.97:0/775529051 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa18003e50 con 0x556d6624e900 2024-03-26T19:00:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.129+0000 7ff9ffe00640 1 -- 172.21.15.97:0/775529051 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa18029040 con 0x556d6624e900 2024-03-26T19:00:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.129+0000 7ffa27dee740 10 client.25394 fetch_fsmap learned FSMap version 27 2024-03-26T19:00:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.129+0000 7ffa27dee740 1 -- 172.21.15.97:0/775529051 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d6625d630 con 0x556d6624e900 2024-03-26T19:00:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.129+0000 7ff9ffe00640 1 -- 172.21.15.97:0/775529051 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa18003e50 con 0x556d6624e900 2024-03-26T19:00:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.130+0000 7ff9ffe00640 1 -- 172.21.15.97:0/775529051 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffa1802d070 con 0x556d6624e900 2024-03-26T19:00:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.130+0000 7ffa27dee740 10 client.25394 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:00:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.130+0000 7ffa27dee740 10 client.25394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:00:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.130+0000 7ffa27dee740 20 client.25394 populate_metadata read hostname 'smithi097' 2024-03-26T19:00:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.130+0000 7ffa27dee740 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556d6625eaa0 0x556d66260e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.130+0000 7ffa27dee740 4 client.25394 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:00:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.130+0000 7ffa27dee740 1 -- 172.21.15.97:0/775529051 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d661bb3c0 con 0x556d6625eaa0 2024-03-26T19:00:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.130+0000 7ffa1d400640 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556d6625eaa0 0x556d66260e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.131+0000 7ffa1d400640 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556d6625eaa0 0x556d66260e90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdba6874bb58e0a0 server_cookie=5753a7618cd6a6db in_seq=0 out_seq=0 2024-03-26T19:00:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.131+0000 7ff9ffe00640 10 client.25394 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.132+0000 7ff9ffe00640 1 -- 172.21.15.97:0/775529051 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556d661bb3c0 con 0x556d6625eaa0 2024-03-26T19:00:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.132+0000 7ff9ffe00640 10 client.25394 handle_command_reply: tid=0 2024-03-26T19:00:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.132+0000 7ffa27dee740 1 client.25394 shutdown 2024-03-26T19:00:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ffa27dee740 1 -- 172.21.15.97:0/775529051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d6624e900 msgr2=0x556d6625a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ffa27dee740 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d6624e900 0x556d6625a350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa18005fd0 tx=0x7ffa18009500 comp rx=0 tx=0).stop 2024-03-26T19:00:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ffa27dee740 1 -- 172.21.15.97:0/775529051 shutdown_connections 2024-03-26T19:00:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ffa27dee740 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556d6625eaa0 0x556d66260e90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ffa27dee740 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d6624e900 0x556d6625a350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ffa27dee740 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d66253310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ffa27dee740 1 --2- 172.21.15.97:0/775529051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 0x556d66252dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ffa27dee740 1 -- 172.21.15.97:0/775529051 >> 172.21.15.97:0/775529051 conn(0x556d66244310 msgr2=0x556d66243540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ff9ffe00640 0 client.25394 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ffa27dee740 1 -- 172.21.15.97:0/775529051 shutdown_connections 2024-03-26T19:00:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.133+0000 7ffa27dee740 1 -- 172.21.15.97:0/775529051 wait complete. 2024-03-26T19:00:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.134+0000 7ffa27dee740 20 client.25394 trim_cache size 0 max 16384 2024-03-26T19:00:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.160+0000 7ffa27dee740 1 Processor -- start 2024-03-26T19:00:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.161+0000 7ffa27dee740 1 -- start start 2024-03-26T19:00:30.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.162+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d66247570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.162+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d6625a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.162+0000 7ffa1e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d66247570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.162+0000 7ffa1e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d66247570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:40278/0 (socket says 172.21.15.97:40278) 2024-03-26T19:00:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.162+0000 7ffa1e800640 1 -- 172.21.15.97:0/4168808459 learned_addr learned my addr 172.21.15.97:0/4168808459 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.162+0000 7ffa27dee740 1 --2- 172.21.15.97:0/4168808459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 0x556d66247ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.162+0000 7ffa27dee740 1 -- 172.21.15.97:0/4168808459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201af370 con 0x556d66249790 2024-03-26T19:00:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.162+0000 7ffa27dee740 1 -- 172.21.15.97:0/4168808459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201b0eb0 con 0x556d66248da0 2024-03-26T19:00:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.162+0000 7ffa27dee740 1 -- 172.21.15.97:0/4168808459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201afa40 con 0x556d6624e900 2024-03-26T19:00:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.163+0000 7ffa1d400640 1 --2- 172.21.15.97:0/4168808459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d6625a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.163+0000 7ffa1e800640 1 --2- 172.21.15.97:0/4168808459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 0x556d66247ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.164+0000 7ffa1e800640 1 -- 172.21.15.97:0/4168808459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 msgr2=0x556d66247570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:30.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.164+0000 7ffa1e800640 1 --2- 172.21.15.97:0/4168808459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d66247570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.164+0000 7ffa1e800640 1 -- 172.21.15.97:0/4168808459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 msgr2=0x556d6625a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.164+0000 7ffa1e800640 1 --2- 172.21.15.97:0/4168808459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d6625a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.165+0000 7ffa1e800640 1 -- 172.21.15.97:0/4168808459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d66243cc0 con 0x556d6624e900 2024-03-26T19:00:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.165+0000 7ffa1e800640 1 --2- 172.21.15.97:0/4168808459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 0x556d66247ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa18004910 tx=0x7ffa18009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e64a126c78ef1b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.165+0000 7ffa0aa00640 1 -- 172.21.15.97:0/4168808459 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18026030 con 0x556d6624e900 2024-03-26T19:00:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.165+0000 7ffa0aa00640 1 -- 172.21.15.97:0/4168808459 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa180114d0 con 0x556d6624e900 2024-03-26T19:00:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.165+0000 7ffa0aa00640 1 -- 172.21.15.97:0/4168808459 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18034080 con 0x556d6624e900 2024-03-26T19:00:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.167+0000 7ffa27dee740 1 -- 172.21.15.97:0/4168808459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 msgr2=0x556d66247ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.167+0000 7ffa27dee740 1 --2- 172.21.15.97:0/4168808459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 0x556d66247ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa18004910 tx=0x7ffa18009820 comp rx=0 tx=0).stop 2024-03-26T19:00:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.168+0000 7ffa27dee740 1 -- 172.21.15.97:0/4168808459 shutdown_connections 2024-03-26T19:00:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.168+0000 7ffa27dee740 1 --2- 172.21.15.97:0/4168808459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6624e900 0x556d66247ab0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.168+0000 7ffa27dee740 1 --2- 172.21.15.97:0/4168808459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d66249790 0x556d6625a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.168+0000 7ffa27dee740 1 --2- 172.21.15.97:0/4168808459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66248da0 0x556d66247570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.168+0000 7ffa27dee740 1 -- 172.21.15.97:0/4168808459 >> 172.21.15.97:0/4168808459 conn(0x556d66244310 msgr2=0x556d662446f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.168+0000 7ffa27dee740 1 -- 172.21.15.97:0/4168808459 shutdown_connections 2024-03-26T19:00:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.168+0000 7ffa27dee740 1 -- 172.21.15.97:0/4168808459 wait complete. 2024-03-26T19:00:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.169+0000 7ffa27dee740 1 -- start start 2024-03-26T19:00:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.169+0000 7ffa27dee740 1 Processor -- start 2024-03-26T19:00:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.170+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 0x556d662575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.170+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66249790 0x556d66257b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.170+0000 7ffa27dee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d6624e900 0x556d66258b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.170+0000 7ffa27dee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201b07e0 con 0x556d6624e900 2024-03-26T19:00:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.170+0000 7ffa27dee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201b0110 con 0x556d66249790 2024-03-26T19:00:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.170+0000 7ffa27dee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201ad350 con 0x556d66248da0 2024-03-26T19:00:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.170+0000 7ffa27dee740 10 client.0 _ll_register_callbacks cb 0x7ffa1800fe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:00:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.170+0000 7ffa1e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 0x556d662575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.170+0000 7ffa1e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 0x556d662575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44736/0 (socket says 172.21.15.97:44736) 2024-03-26T19:00:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.170+0000 7ffa1e800640 1 -- 172.21.15.97:0/3313218762 learned_addr learned my addr 172.21.15.97:0/3313218762 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.171+0000 7ffa1d400640 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66249790 0x556d66257b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.171+0000 7ffa1e800640 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d6624e900 0x556d66258b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.172+0000 7ffa1d400640 1 -- 172.21.15.97:0/3313218762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 msgr2=0x556d662575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.172+0000 7ffa1d400640 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 0x556d662575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.172+0000 7ffa1d400640 1 -- 172.21.15.97:0/3313218762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d6624e900 msgr2=0x556d66258b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.172+0000 7ffa1d400640 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d6624e900 0x556d66258b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.172+0000 7ffa1d400640 1 -- 172.21.15.97:0/3313218762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d66259110 con 0x556d66249790 2024-03-26T19:00:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.172+0000 7ffa1e800640 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d6624e900 0x556d66258b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:00:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.173+0000 7ffa1d400640 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66249790 0x556d66257b10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c036040 tx=0x7ffa0c005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2b442c5e64a6a53 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.173+0000 7ff9ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.173+0000 7ff9ffe00640 1 -- 172.21.15.97:0/3313218762 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c04b070 con 0x556d66249790 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.173+0000 7ff9ffe00640 1 -- 172.21.15.97:0/3313218762 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ffa0c004040 con 0x556d66249790 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.173+0000 7ffa27dee740 1 -- 172.21.15.97:0/3313218762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d661b5710 con 0x556d66249790 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.173+0000 7ff9ffe00640 1 -- 172.21.15.97:0/3313218762 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa0c02cd20 con 0x556d66249790 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.173+0000 7ffa27dee740 10 client.25400 fetch_fsmap learned FSMap version 27 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.173+0000 7ffa27dee740 1 -- 172.21.15.97:0/3313218762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d6625a490 con 0x556d66249790 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.174+0000 7ff9ffe00640 1 -- 172.21.15.97:0/3313218762 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556d661b5710 con 0x556d66249790 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.174+0000 7ff9ffe00640 1 -- 172.21.15.97:0/3313218762 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffa0c0051d0 con 0x556d66249790 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.174+0000 7ffa27dee740 10 client.25400 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.174+0000 7ffa27dee740 10 client.25400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:00:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.174+0000 7ffa27dee740 20 client.25400 populate_metadata read hostname 'smithi097' 2024-03-26T19:00:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.174+0000 7ffa27dee740 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556d662c53b0 0x556d662c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.174+0000 7ffa27dee740 4 client.25400 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:00:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.174+0000 7ffa27dee740 1 -- 172.21.15.97:0/3313218762 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d66133630 con 0x556d662c53b0 2024-03-26T19:00:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.174+0000 7ffa1e800640 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556d662c53b0 0x556d662c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.175+0000 7ff9ffe00640 10 client.25400 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.175+0000 7ffa1e800640 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556d662c53b0 0x556d662c77d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6143db59d6dde062 server_cookie=b47c4a62c600eff1 in_seq=0 out_seq=0 2024-03-26T19:00:30.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.195+0000 7ff9ffe00640 1 -- 172.21.15.97:0/3313218762 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d66133630 con 0x556d662c53b0 2024-03-26T19:00:30.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.195+0000 7ff9ffe00640 10 client.25400 handle_command_reply: tid=0 2024-03-26T19:00:30.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.195+0000 7ffa27dee740 1 client.25400 shutdown 2024-03-26T19:00:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.196+0000 7ffa27dee740 1 -- 172.21.15.97:0/3313218762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66249790 msgr2=0x556d66257b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.196+0000 7ffa27dee740 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66249790 0x556d66257b10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c036040 tx=0x7ffa0c005b50 comp rx=0 tx=0).stop 2024-03-26T19:00:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.196+0000 7ffa27dee740 1 -- 172.21.15.97:0/3313218762 shutdown_connections 2024-03-26T19:00:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.196+0000 7ffa27dee740 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556d662c53b0 0x556d662c77d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.196+0000 7ffa27dee740 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556d6624e900 0x556d66258b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.196+0000 7ffa27dee740 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d66249790 0x556d66257b10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.196+0000 7ffa27dee740 1 --2- 172.21.15.97:0/3313218762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d66248da0 0x556d662575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.196+0000 7ffa27dee740 1 -- 172.21.15.97:0/3313218762 >> 172.21.15.97:0/3313218762 conn(0x556d66244310 msgr2=0x556d66247570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.196+0000 7ff9ffe00640 0 client.25400 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.197+0000 7ffa27dee740 1 -- 172.21.15.97:0/3313218762 shutdown_connections 2024-03-26T19:00:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.197+0000 7ffa27dee740 1 -- 172.21.15.97:0/3313218762 wait complete. 2024-03-26T19:00:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.197+0000 7ffa27dee740 20 client.25400 trim_cache size 0 max 16384 2024-03-26T19:00:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:00:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:00:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:00:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.198+0000 7ffa25000640 1 -- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ffa040a31b0 msgr2=0x7ffa040a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.198+0000 7ffa25000640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ffa040a31b0 0x7ffa040a5670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa1009bb80 tx=0x7ffa1000cf20 comp rx=0 tx=0).stop 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.198+0000 7ffa25000640 1 -- 172.21.15.97:0/397297385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa201018f0 msgr2=0x7ffa2019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.198+0000 7ffa25000640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa201018f0 0x7ffa2019e150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9c0 tx=0x7ffa1000edc0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.199+0000 7ffa25000640 1 -- 172.21.15.97:0/397297385 shutdown_connections 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.199+0000 7ffa25000640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ffa040a31b0 0x7ffa040a5670 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.199+0000 7ffa25000640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa201034a0 0x7ffa201a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.199+0000 7ffa25000640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa20102ae0 0x7ffa2019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.199+0000 7ffa25000640 1 --2- 172.21.15.97:0/397297385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa201018f0 0x7ffa2019e150 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.199+0000 7ffa25000640 1 -- 172.21.15.97:0/397297385 >> 172.21.15.97:0/397297385 conn(0x7ffa200fce80 msgr2=0x7ffa20103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.199+0000 7ffa25000640 1 -- 172.21.15.97:0/397297385 shutdown_connections 2024-03-26T19:00:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:30.199+0000 7ffa25000640 1 -- 172.21.15.97:0/397297385 wait complete. 2024-03-26T19:00:32.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:32 smithi097 ceph-mon[31955]: pgmap v438: 97 pgs: 97 active+clean; 1013 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 745 op/s 2024-03-26T19:00:32.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:32 smithi144 ceph-mon[36053]: pgmap v438: 97 pgs: 97 active+clean; 1013 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 745 op/s 2024-03-26T19:00:32.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:32 smithi105 ceph-mon[36270]: pgmap v438: 97 pgs: 97 active+clean; 1013 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 745 op/s 2024-03-26T19:00:34.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:34 smithi097 ceph-mon[31955]: pgmap v439: 97 pgs: 97 active+clean; 1015 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 648 op/s 2024-03-26T19:00:34.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:34 smithi144 ceph-mon[36053]: pgmap v439: 97 pgs: 97 active+clean; 1015 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 648 op/s 2024-03-26T19:00:34.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:34 smithi105 ceph-mon[36270]: pgmap v439: 97 pgs: 97 active+clean; 1015 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 648 op/s 2024-03-26T19:00:35.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 70.0 s, 2671.4 tps, lat 37.503 ms stddev 39.906 2024-03-26T19:00:36.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:36 smithi097 ceph-mon[31955]: pgmap v440: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 712 op/s 2024-03-26T19:00:36.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:36 smithi144 ceph-mon[36053]: pgmap v440: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 712 op/s 2024-03-26T19:00:36.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:36 smithi105 ceph-mon[36270]: pgmap v440: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 712 op/s 2024-03-26T19:00:38.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:38 smithi097 ceph-mon[31955]: pgmap v441: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 669 op/s 2024-03-26T19:00:38.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:38 smithi144 ceph-mon[36053]: pgmap v441: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 669 op/s 2024-03-26T19:00:38.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:38 smithi105 ceph-mon[36270]: pgmap v441: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 669 op/s 2024-03-26T19:00:40.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 75.0 s, 2447.9 tps, lat 40.793 ms stddev 44.264 2024-03-26T19:00:40.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:40 smithi097 ceph-mon[31955]: pgmap v442: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 775 op/s 2024-03-26T19:00:40.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:40 smithi144 ceph-mon[36053]: pgmap v442: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 775 op/s 2024-03-26T19:00:40.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:40 smithi105 ceph-mon[36270]: pgmap v442: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 775 op/s 2024-03-26T19:00:42.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:42 smithi097 ceph-mon[31955]: pgmap v443: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 MiB/s wr, 676 op/s 2024-03-26T19:00:42.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:42 smithi144 ceph-mon[36053]: pgmap v443: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 MiB/s wr, 676 op/s 2024-03-26T19:00:42.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:42 smithi105 ceph-mon[36270]: pgmap v443: 97 pgs: 97 active+clean; 1016 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 MiB/s wr, 676 op/s 2024-03-26T19:00:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.396+0000 7fb5b2a00640 1 -- 172.21.15.97:0/1886590387 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a002cc80 con 0x7fb5b4102ae0 2024-03-26T19:00:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.397+0000 7fb5bb200640 1 -- 172.21.15.97:0/1886590387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b4102ae0 msgr2=0x7fb5b4102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.397+0000 7fb5bb200640 1 --2- 172.21.15.97:0/1886590387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b4102ae0 0x7fb5b4102f60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009900 tx=0x7fb5a002bc30 comp rx=0 tx=0).stop 2024-03-26T19:00:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.397+0000 7fb5bb200640 1 -- 172.21.15.97:0/1886590387 shutdown_connections 2024-03-26T19:00:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.397+0000 7fb5bb200640 1 --2- 172.21.15.97:0/1886590387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5b41034a0 0x7fb5b4109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.397+0000 7fb5bb200640 1 --2- 172.21.15.97:0/1886590387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b4102ae0 0x7fb5b4102f60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.397+0000 7fb5bb200640 1 --2- 172.21.15.97:0/1886590387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b41018f0 0x7fb5b4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.397+0000 7fb5bb200640 1 -- 172.21.15.97:0/1886590387 >> 172.21.15.97:0/1886590387 conn(0x7fb5b40fce80 msgr2=0x7fb5b40ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.398+0000 7fb5bb200640 1 -- 172.21.15.97:0/1886590387 shutdown_connections 2024-03-26T19:00:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.398+0000 7fb5bb200640 1 -- 172.21.15.97:0/1886590387 wait complete. 2024-03-26T19:00:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.399+0000 7fb5bb200640 1 Processor -- start 2024-03-26T19:00:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.399+0000 7fb5bb200640 1 -- start start 2024-03-26T19:00:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.399+0000 7fb5bb200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b41018f0 0x7fb5b419e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.399+0000 7fb5bb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5b4102ae0 0x7fb5b419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.399+0000 7fb5bb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b41034a0 0x7fb5b41a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.399+0000 7fb5bb200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b410b3d0 con 0x7fb5b4102ae0 2024-03-26T19:00:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.399+0000 7fb5bb200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b410b250 con 0x7fb5b41018f0 2024-03-26T19:00:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.399+0000 7fb5bb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b410b550 con 0x7fb5b41034a0 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.400+0000 7fb5b3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b41018f0 0x7fb5b419e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.400+0000 7fb5b3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b41018f0 0x7fb5b419e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:48254/0 (socket says 172.21.15.97:48254) 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.400+0000 7fb5b9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b41034a0 0x7fb5b41a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.400+0000 7fb5b9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b41034a0 0x7fb5b41a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37664/0 (socket says 172.21.15.97:37664) 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.400+0000 7fb5b9200640 1 -- 172.21.15.97:0/1753228019 learned_addr learned my addr 172.21.15.97:0/1753228019 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.400+0000 7fb5b3400640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5b4102ae0 0x7fb5b419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.400+0000 7fb5b3e00640 1 -- 172.21.15.97:0/1753228019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b41034a0 msgr2=0x7fb5b41a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.401+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b41034a0 0x7fb5b41a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.401+0000 7fb5b3e00640 1 -- 172.21.15.97:0/1753228019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5b4102ae0 msgr2=0x7fb5b419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.401+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5b4102ae0 0x7fb5b419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.401+0000 7fb5b3e00640 1 -- 172.21.15.97:0/1753228019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b406b410 con 0x7fb5b41018f0 2024-03-26T19:00:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.401+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b41018f0 0x7fb5b419e150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac005fd0 tx=0x7fb5ac00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.401+0000 7fb5b0c00640 1 -- 172.21.15.97:0/1753228019 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac010630 con 0x7fb5b41018f0 2024-03-26T19:00:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.401+0000 7fb5b0c00640 1 -- 172.21.15.97:0/1753228019 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ac010cb0 con 0x7fb5b41018f0 2024-03-26T19:00:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.401+0000 7fb5b0c00640 1 -- 172.21.15.97:0/1753228019 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac018d90 con 0x7fb5b41018f0 2024-03-26T19:00:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.401+0000 7fb5bb200640 1 -- 172.21.15.97:0/1753228019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b406b700 con 0x7fb5b41018f0 2024-03-26T19:00:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.402+0000 7fb5bb200640 1 -- 172.21.15.97:0/1753228019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b406bc00 con 0x7fb5b41018f0 2024-03-26T19:00:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.406+0000 7fb5b0c00640 1 -- 172.21.15.97:0/1753228019 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fb5ac01f020 con 0x7fb5b41018f0 2024-03-26T19:00:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.407+0000 7fb5b0c00640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5980a3130 0x7fb5980a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.408+0000 7fb5b0c00640 1 -- 172.21.15.97:0/1753228019 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb5ac0cf7e0 con 0x7fb5b41018f0 2024-03-26T19:00:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.408+0000 7fb5bde28740 1 Processor -- start 2024-03-26T19:00:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.409+0000 7fb5b3400640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5980a3130 0x7fb5980a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.409+0000 7fb5bde28740 1 -- start start 2024-03-26T19:00:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.411+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 0x5637e6c401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.411+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c40790 0x5637e6c45290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.411+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c47d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.411+0000 7fb5bde28740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5637e6ba4e80 con 0x5637e6c40790 2024-03-26T19:00:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.411+0000 7fb5bde28740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5637e6b4d660 con 0x5637e6c45900 2024-03-26T19:00:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.411+0000 7fb5bde28740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637e6ba0e30 con 0x5637e6c3fda0 2024-03-26T19:00:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.412+0000 7fb5b3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 0x5637e6c401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.412+0000 7fb5b3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 0x5637e6c401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37668/0 (socket says 172.21.15.97:37668) 2024-03-26T19:00:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.412+0000 7fb5b3e00640 1 -- 172.21.15.97:0/2937491734 learned_addr learned my addr 172.21.15.97:0/2937491734 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.412+0000 7fb5b9200640 1 --2- 172.21.15.97:0/2937491734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c47d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.412+0000 7fb5b3e00640 1 -- 172.21.15.97:0/2937491734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 msgr2=0x5637e6c47d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.412+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/2937491734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c47d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.412+0000 7fb5b3e00640 1 -- 172.21.15.97:0/2937491734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c40790 msgr2=0x5637e6c45290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.412+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/2937491734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c40790 0x5637e6c45290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.412+0000 7fb5b3e00640 1 -- 172.21.15.97:0/2937491734 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e68b9210 con 0x5637e6c3fda0 2024-03-26T19:00:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.413+0000 7fb5b3400640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5980a3130 0x7fb5980a55f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00096c0 tx=0x7fb5a002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.413+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/2937491734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 0x5637e6c401c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0a4420 tx=0x7fb5ac09b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a6e2d53a50c8a0d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.414+0000 7fb5a9600640 1 -- 172.21.15.97:0/2937491734 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0b5070 con 0x5637e6c3fda0 2024-03-26T19:00:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.414+0000 7fb5a9600640 1 -- 172.21.15.97:0/2937491734 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ac0a0090 con 0x5637e6c3fda0 2024-03-26T19:00:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.414+0000 7fb5a9600640 1 -- 172.21.15.97:0/2937491734 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0a44c0 con 0x5637e6c3fda0 2024-03-26T19:00:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.415+0000 7fb5bde28740 1 -- 172.21.15.97:0/2937491734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 msgr2=0x5637e6c401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.415+0000 7fb5bde28740 1 --2- 172.21.15.97:0/2937491734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 0x5637e6c401c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0a4420 tx=0x7fb5ac09b960 comp rx=0 tx=0).stop 2024-03-26T19:00:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.415+0000 7fb5bde28740 1 -- 172.21.15.97:0/2937491734 shutdown_connections 2024-03-26T19:00:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.415+0000 7fb5bde28740 1 --2- 172.21.15.97:0/2937491734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c47d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.415+0000 7fb5bde28740 1 --2- 172.21.15.97:0/2937491734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c40790 0x5637e6c45290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.415+0000 7fb5bde28740 1 --2- 172.21.15.97:0/2937491734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 0x5637e6c401c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.415+0000 7fb5bde28740 1 -- 172.21.15.97:0/2937491734 >> 172.21.15.97:0/2937491734 conn(0x5637e6c3b310 msgr2=0x5637e6c3b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.415+0000 7fb5bde28740 1 -- 172.21.15.97:0/2937491734 shutdown_connections 2024-03-26T19:00:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.415+0000 7fb5bde28740 1 -- 172.21.15.97:0/2937491734 wait complete. 2024-03-26T19:00:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.416+0000 7fb5bde28740 1 -- start start 2024-03-26T19:00:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.416+0000 7fb5bde28740 1 Processor -- start 2024-03-26T19:00:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.417+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 0x5637e6c49e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.417+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c40790 0x5637e6c4a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.417+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c45900 0x5637e6c51390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.417+0000 7fb5bde28740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5637e6c488b0 con 0x5637e6c3fda0 2024-03-26T19:00:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.417+0000 7fb5bde28740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5637e6b60ac0 con 0x5637e6c40790 2024-03-26T19:00:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.417+0000 7fb5bde28740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637e6c48a30 con 0x5637e6c45900 2024-03-26T19:00:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.417+0000 7fb5bde28740 10 client.0 _ll_register_callbacks cb 0x5637e6b60490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:00:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.417+0000 7fb5b9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c45900 0x5637e6c51390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.417+0000 7fb5b9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c45900 0x5637e6c51390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37684/0 (socket says 172.21.15.97:37684) 2024-03-26T19:00:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.417+0000 7fb5b9200640 1 -- 172.21.15.97:0/1771844924 learned_addr learned my addr 172.21.15.97:0/1771844924 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.418+0000 7fb5b3400640 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c40790 0x5637e6c4a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.418+0000 7fb5b9200640 1 -- 172.21.15.97:0/1771844924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c40790 msgr2=0x5637e6c4a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.418+0000 7fb5b9200640 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c40790 0x5637e6c4a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.418+0000 7fb5b9200640 1 -- 172.21.15.97:0/1771844924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 msgr2=0x5637e6c49e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.418+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 0x5637e6c49e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.418+0000 7fb5b9200640 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 0x5637e6c49e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.418+0000 7fb5b9200640 1 -- 172.21.15.97:0/1771844924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e6985620 con 0x5637e6c45900 2024-03-26T19:00:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.419+0000 7fb5b3400640 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c40790 0x5637e6c4a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:00:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.419+0000 7fb59e000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:00:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.419+0000 7fb5b9200640 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c45900 0x5637e6c51390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400bf30 tx=0x7fb5a40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bdee724beb834f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.419+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 0x5637e6c49e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:00:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.420+0000 7fb59e000640 1 -- 172.21.15.97:0/1771844924 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a401f060 con 0x5637e6c45900 2024-03-26T19:00:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.420+0000 7fb5bde28740 1 -- 172.21.15.97:0/1771844924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ac0a0090 con 0x5637e6c45900 2024-03-26T19:00:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.420+0000 7fb59e000640 1 -- 172.21.15.97:0/1771844924 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb5a4010700 con 0x5637e6c45900 2024-03-26T19:00:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.421+0000 7fb59e000640 1 -- 172.21.15.97:0/1771844924 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb5a4010900 con 0x5637e6c45900 2024-03-26T19:00:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.421+0000 7fb5bde28740 10 client.25378 fetch_fsmap learned FSMap version 27 2024-03-26T19:00:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.421+0000 7fb5bde28740 1 -- 172.21.15.97:0/1771844924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637e6c54820 con 0x5637e6c45900 2024-03-26T19:00:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.421+0000 7fb59e000640 1 -- 172.21.15.97:0/1771844924 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ac0a0090 con 0x5637e6c45900 2024-03-26T19:00:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.422+0000 7fb59e000640 1 -- 172.21.15.97:0/1771844924 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb5a401ac20 con 0x5637e6c45900 2024-03-26T19:00:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.422+0000 7fb5bde28740 10 client.25378 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:00:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.422+0000 7fb5bde28740 10 client.25378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:00:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.422+0000 7fb5bde28740 20 client.25378 populate_metadata read hostname 'smithi097' 2024-03-26T19:00:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.422+0000 7fb5bde28740 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5637e6c55f10 0x5637e6c58380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.422+0000 7fb5bde28740 4 client.25378 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:00:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.422+0000 7fb5bde28740 1 -- 172.21.15.97:0/1771844924 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637e6bb23c0 con 0x5637e6c55f10 2024-03-26T19:00:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.422+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5637e6c55f10 0x5637e6c58380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.423+0000 7fb59e000640 10 client.25378 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.423+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5637e6c55f10 0x5637e6c58380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c855e805deb6e54 server_cookie=17597a8809915cea in_seq=0 out_seq=0 2024-03-26T19:00:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.424+0000 7fb59e000640 1 -- 172.21.15.97:0/1771844924 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5637e6bb23c0 con 0x5637e6c55f10 2024-03-26T19:00:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.424+0000 7fb59e000640 10 client.25378 handle_command_reply: tid=0 2024-03-26T19:00:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.424+0000 7fb5bde28740 1 client.25378 shutdown 2024-03-26T19:00:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.424+0000 7fb5bde28740 1 -- 172.21.15.97:0/1771844924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c45900 msgr2=0x5637e6c51390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.424+0000 7fb5bde28740 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c45900 0x5637e6c51390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400bf30 tx=0x7fb5a40118d0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.425+0000 7fb5bde28740 1 -- 172.21.15.97:0/1771844924 shutdown_connections 2024-03-26T19:00:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.425+0000 7fb5bde28740 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5637e6c55f10 0x5637e6c58380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.425+0000 7fb5bde28740 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c45900 0x5637e6c51390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.425+0000 7fb5bde28740 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c40790 0x5637e6c4a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.425+0000 7fb5bde28740 1 --2- 172.21.15.97:0/1771844924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 0x5637e6c49e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.425+0000 7fb5bde28740 1 -- 172.21.15.97:0/1771844924 >> 172.21.15.97:0/1771844924 conn(0x5637e6c3b310 msgr2=0x5637e6c3a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.425+0000 7fb59e000640 0 client.25378 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.425+0000 7fb5bde28740 1 -- 172.21.15.97:0/1771844924 shutdown_connections 2024-03-26T19:00:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.425+0000 7fb5bde28740 1 -- 172.21.15.97:0/1771844924 wait complete. 2024-03-26T19:00:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.425+0000 7fb5bde28740 20 client.25378 trim_cache size 0 max 16384 2024-03-26T19:00:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.451+0000 7fb5bde28740 1 Processor -- start 2024-03-26T19:00:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.451+0000 7fb5bde28740 1 -- start start 2024-03-26T19:00:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.452+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 0x5637e6c41940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.452+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c40790 0x5637e6c51350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.452+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c5cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.452+0000 7fb5bde28740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41b7ac0 con 0x5637e6c40790 2024-03-26T19:00:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.452+0000 7fb5bde28740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41b9600 con 0x5637e6c45900 2024-03-26T19:00:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.452+0000 7fb5bde28740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41b8190 con 0x5637e6c3fda0 2024-03-26T19:00:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.452+0000 7fb5b9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c5cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.452+0000 7fb5b9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c5cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:48278/0 (socket says 172.21.15.97:48278) 2024-03-26T19:00:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.452+0000 7fb5b9200640 1 -- 172.21.15.97:0/836266447 learned_addr learned my addr 172.21.15.97:0/836266447 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.453+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/836266447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 0x5637e6c41940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.453+0000 7fb5b3400640 1 --2- 172.21.15.97:0/836266447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c40790 0x5637e6c51350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.453+0000 7fb5b9200640 1 -- 172.21.15.97:0/836266447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 msgr2=0x5637e6c41940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.453+0000 7fb5b9200640 1 --2- 172.21.15.97:0/836266447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 0x5637e6c41940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.453+0000 7fb5b9200640 1 -- 172.21.15.97:0/836266447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c40790 msgr2=0x5637e6c51350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.453+0000 7fb5b9200640 1 --2- 172.21.15.97:0/836266447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c40790 0x5637e6c51350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.453+0000 7fb5b9200640 1 -- 172.21.15.97:0/836266447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e6c39ab0 con 0x5637e6c45900 2024-03-26T19:00:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.453+0000 7fb5b3400640 1 --2- 172.21.15.97:0/836266447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c40790 0x5637e6c51350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:00:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.454+0000 7fb5b9200640 1 --2- 172.21.15.97:0/836266447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c5cc80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b500 tx=0x7fb5a4010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8455d4c0304248 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.455+0000 7fb5a8c00640 1 -- 172.21.15.97:0/836266447 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a402a030 con 0x5637e6c45900 2024-03-26T19:00:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.455+0000 7fb5a8c00640 1 -- 172.21.15.97:0/836266447 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a402c070 con 0x5637e6c45900 2024-03-26T19:00:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.455+0000 7fb5a8c00640 1 -- 172.21.15.97:0/836266447 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4010900 con 0x5637e6c45900 2024-03-26T19:00:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.456+0000 7fb5bde28740 1 -- 172.21.15.97:0/836266447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 msgr2=0x5637e6c5cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.456+0000 7fb5bde28740 1 --2- 172.21.15.97:0/836266447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c5cc80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b500 tx=0x7fb5a4010890 comp rx=0 tx=0).stop 2024-03-26T19:00:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.457+0000 7fb5bde28740 1 -- 172.21.15.97:0/836266447 shutdown_connections 2024-03-26T19:00:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.457+0000 7fb5bde28740 1 --2- 172.21.15.97:0/836266447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c5cc80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.457+0000 7fb5bde28740 1 --2- 172.21.15.97:0/836266447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c40790 0x5637e6c51350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.457+0000 7fb5bde28740 1 --2- 172.21.15.97:0/836266447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c3fda0 0x5637e6c41940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.457+0000 7fb5bde28740 1 -- 172.21.15.97:0/836266447 >> 172.21.15.97:0/836266447 conn(0x5637e6c3b310 msgr2=0x5637e6c41230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.458+0000 7fb5bde28740 1 -- 172.21.15.97:0/836266447 shutdown_connections 2024-03-26T19:00:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.458+0000 7fb5bde28740 1 -- 172.21.15.97:0/836266447 wait complete. 2024-03-26T19:00:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.459+0000 7fb5bde28740 1 -- start start 2024-03-26T19:00:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.459+0000 7fb5bde28740 1 Processor -- start 2024-03-26T19:00:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.460+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 0x5637e6c4ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.460+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c40790 0x5637e6c4f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.460+0000 7fb5bde28740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.460+0000 7fb5bde28740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41b8f30 con 0x5637e6c3fda0 2024-03-26T19:00:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.460+0000 7fb5bde28740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41b8860 con 0x5637e6c45900 2024-03-26T19:00:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.460+0000 7fb5bde28740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4072920 con 0x5637e6c40790 2024-03-26T19:00:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.460+0000 7fb5bde28740 10 client.0 _ll_register_callbacks cb 0x7fb5a400bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:00:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.461+0000 7fb5b9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.461+0000 7fb5b3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 0x5637e6c4ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.461+0000 7fb5b9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:48290/0 (socket says 172.21.15.97:48290) 2024-03-26T19:00:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.461+0000 7fb5b9200640 1 -- 172.21.15.97:0/2847311590 learned_addr learned my addr 172.21.15.97:0/2847311590 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.461+0000 7fb5b3400640 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c40790 0x5637e6c4f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.462+0000 7fb5b9200640 1 -- 172.21.15.97:0/2847311590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c40790 msgr2=0x5637e6c4f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.462+0000 7fb5b9200640 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c40790 0x5637e6c4f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.462+0000 7fb5b9200640 1 -- 172.21.15.97:0/2847311590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 msgr2=0x5637e6c4ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.462+0000 7fb5b9200640 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 0x5637e6c4ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.462+0000 7fb5b9200640 1 -- 172.21.15.97:0/2847311590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e6c506b0 con 0x5637e6c45900 2024-03-26T19:00:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.462+0000 7fb5b3400640 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c40790 0x5637e6c4f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:00:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.462+0000 7fb5b9200640 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c500c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400bc50 tx=0x7fb5a40058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c1b85e7157c4508 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.462+0000 7fb59e000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:00:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.463+0000 7fb59e000640 1 -- 172.21.15.97:0/2847311590 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a403d060 con 0x5637e6c45900 2024-03-26T19:00:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.463+0000 7fb59e000640 1 -- 172.21.15.97:0/2847311590 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb5a4010bd0 con 0x5637e6c45900 2024-03-26T19:00:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.463+0000 7fb5bde28740 1 -- 172.21.15.97:0/2847311590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a402c070 con 0x5637e6c45900 2024-03-26T19:00:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.463+0000 7fb59e000640 1 -- 172.21.15.97:0/2847311590 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb5a402a070 con 0x5637e6c45900 2024-03-26T19:00:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.463+0000 7fb5bde28740 10 client.25415 fetch_fsmap learned FSMap version 27 2024-03-26T19:00:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.463+0000 7fb5bde28740 1 -- 172.21.15.97:0/2847311590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637e6ccd330 con 0x5637e6c45900 2024-03-26T19:00:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.464+0000 7fb59e000640 1 -- 172.21.15.97:0/2847311590 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a402c070 con 0x5637e6c45900 2024-03-26T19:00:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.464+0000 7fb59e000640 1 -- 172.21.15.97:0/2847311590 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb5a40264c0 con 0x5637e6c45900 2024-03-26T19:00:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.464+0000 7fb5bde28740 10 client.25415 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:00:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.464+0000 7fb5bde28740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:00:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.464+0000 7fb5bde28740 20 client.25415 populate_metadata read hostname 'smithi097' 2024-03-26T19:00:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.464+0000 7fb5bde28740 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5637e6ccdbd0 0x5637e6ccfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.464+0000 7fb5bde28740 4 client.25415 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:00:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.464+0000 7fb5bde28740 1 -- 172.21.15.97:0/2847311590 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637e6bac710 con 0x5637e6ccdbd0 2024-03-26T19:00:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.464+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5637e6ccdbd0 0x5637e6ccfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.465+0000 7fb59e000640 10 client.25415 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.465+0000 7fb5b3e00640 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5637e6ccdbd0 0x5637e6ccfff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73177932c7cfe00b server_cookie=9498e79f32bbf693 in_seq=0 out_seq=0 2024-03-26T19:00:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb59e000640 1 -- 172.21.15.97:0/2847311590 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637e6bac710 con 0x5637e6ccdbd0 2024-03-26T19:00:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb59e000640 10 client.25415 handle_command_reply: tid=0 2024-03-26T19:00:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb5bde28740 1 client.25415 shutdown 2024-03-26T19:00:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb5bde28740 1 -- 172.21.15.97:0/2847311590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 msgr2=0x5637e6c500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb5bde28740 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c500c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400bc50 tx=0x7fb5a40058d0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb5bde28740 1 -- 172.21.15.97:0/2847311590 shutdown_connections 2024-03-26T19:00:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb5bde28740 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5637e6ccdbd0 0x5637e6ccfff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb5bde28740 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637e6c45900 0x5637e6c500c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb5bde28740 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637e6c40790 0x5637e6c4f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb5bde28740 1 --2- 172.21.15.97:0/2847311590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5637e6c3fda0 0x5637e6c4ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb59e000640 0 client.25415 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.480+0000 7fb5bde28740 1 -- 172.21.15.97:0/2847311590 >> 172.21.15.97:0/2847311590 conn(0x5637e6c3b310 msgr2=0x5637e6c39cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.481+0000 7fb5bde28740 1 -- 172.21.15.97:0/2847311590 shutdown_connections 2024-03-26T19:00:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.481+0000 7fb5bde28740 1 -- 172.21.15.97:0/2847311590 wait complete. 2024-03-26T19:00:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.481+0000 7fb5bde28740 20 client.25415 trim_cache size 0 max 16384 2024-03-26T19:00:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:00:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:00:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:00:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:00:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.482+0000 7fb5bb200640 1 -- 172.21.15.97:0/1753228019 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5980a3130 msgr2=0x7fb5980a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.482+0000 7fb5bb200640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5980a3130 0x7fb5980a55f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00096c0 tx=0x7fb5a002df90 comp rx=0 tx=0).stop 2024-03-26T19:00:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.482+0000 7fb5bb200640 1 -- 172.21.15.97:0/1753228019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b41018f0 msgr2=0x7fb5b419e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.482+0000 7fb5bb200640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b41018f0 0x7fb5b419e150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac005fd0 tx=0x7fb5ac00c6a0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.482+0000 7fb5bb200640 1 -- 172.21.15.97:0/1753228019 shutdown_connections 2024-03-26T19:00:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.482+0000 7fb5bb200640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb5980a3130 0x7fb5980a55f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.482+0000 7fb5bb200640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b41034a0 0x7fb5b41a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.482+0000 7fb5bb200640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5b4102ae0 0x7fb5b419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.482+0000 7fb5bb200640 1 --2- 172.21.15.97:0/1753228019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b41018f0 0x7fb5b419e150 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.482+0000 7fb5bb200640 1 -- 172.21.15.97:0/1753228019 >> 172.21.15.97:0/1753228019 conn(0x7fb5b40fce80 msgr2=0x7fb5b410dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.483+0000 7fb5bb200640 1 -- 172.21.15.97:0/1753228019 shutdown_connections 2024-03-26T19:00:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:43.483+0000 7fb5bb200640 1 -- 172.21.15.97:0/1753228019 wait complete. 2024-03-26T19:00:44.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:44 smithi097 ceph-mon[31955]: pgmap v444: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 653 op/s 2024-03-26T19:00:44.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:44 smithi144 ceph-mon[36053]: pgmap v444: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 653 op/s 2024-03-26T19:00:44.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:44 smithi105 ceph-mon[36270]: pgmap v444: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 653 op/s 2024-03-26T19:00:45.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 80.0 s, 2895.5 tps, lat 34.507 ms stddev 36.590 2024-03-26T19:00:46.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:46 smithi097 ceph-mon[31955]: pgmap v445: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 771 op/s 2024-03-26T19:00:46.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:46 smithi144 ceph-mon[36053]: pgmap v445: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 771 op/s 2024-03-26T19:00:46.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:46 smithi105 ceph-mon[36270]: pgmap v445: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 771 op/s 2024-03-26T19:00:48.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:48 smithi097 ceph-mon[31955]: pgmap v446: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 626 op/s 2024-03-26T19:00:48.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:48 smithi144 ceph-mon[36053]: pgmap v446: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 626 op/s 2024-03-26T19:00:48.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:48 smithi105 ceph-mon[36270]: pgmap v446: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 626 op/s 2024-03-26T19:00:50.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 85.0 s, 2530.4 tps, lat 39.629 ms stddev 44.539 2024-03-26T19:00:50.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:50 smithi097 ceph-mon[31955]: pgmap v447: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 MiB/s wr, 755 op/s 2024-03-26T19:00:50.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:50 smithi144 ceph-mon[36053]: pgmap v447: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 MiB/s wr, 755 op/s 2024-03-26T19:00:50.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:50 smithi105 ceph-mon[36270]: pgmap v447: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 MiB/s wr, 755 op/s 2024-03-26T19:00:52.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:52 smithi097 ceph-mon[31955]: pgmap v448: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 741 op/s 2024-03-26T19:00:52.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:52 smithi144 ceph-mon[36053]: pgmap v448: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 741 op/s 2024-03-26T19:00:52.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:52 smithi105 ceph-mon[36270]: pgmap v448: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 741 op/s 2024-03-26T19:00:54.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:54 smithi097 ceph-mon[31955]: pgmap v449: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 724 op/s 2024-03-26T19:00:54.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:54 smithi144 ceph-mon[36053]: pgmap v449: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 724 op/s 2024-03-26T19:00:54.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:54 smithi105 ceph-mon[36270]: pgmap v449: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 724 op/s 2024-03-26T19:00:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 90.0 s, 2739.6 tps, lat 36.493 ms stddev 41.994 2024-03-26T19:00:56.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:56 smithi097 ceph-mon[31955]: pgmap v450: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 784 op/s 2024-03-26T19:00:56.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:56 smithi144 ceph-mon[36053]: pgmap v450: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 784 op/s 2024-03-26T19:00:56.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:56 smithi105 ceph-mon[36270]: pgmap v450: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 784 op/s 2024-03-26T19:00:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.676+0000 7f9f87c00640 1 -- 172.21.15.97:0/799452356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f80077e60 msgr2=0x7f9f80078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.676+0000 7f9f87c00640 1 --2- 172.21.15.97:0/799452356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f80077e60 0x7f9f80078700 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009ca0 tx=0x7f9f7802c100 comp rx=0 tx=0).stop 2024-03-26T19:00:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.676+0000 7f9f87c00640 1 -- 172.21.15.97:0/799452356 shutdown_connections 2024-03-26T19:00:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.676+0000 7f9f87c00640 1 --2- 172.21.15.97:0/799452356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f80077e60 0x7f9f80078700 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.676+0000 7f9f87c00640 1 --2- 172.21.15.97:0/799452356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f800774c0 0x7f9f80077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.676+0000 7f9f87c00640 1 --2- 172.21.15.97:0/799452356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f80076280 0x7f9f800766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.676+0000 7f9f87c00640 1 -- 172.21.15.97:0/799452356 >> 172.21.15.97:0/799452356 conn(0x7f9f800ff0b0 msgr2=0x7f9f801014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.677+0000 7f9f87c00640 1 -- 172.21.15.97:0/799452356 shutdown_connections 2024-03-26T19:00:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.677+0000 7f9f87c00640 1 -- 172.21.15.97:0/799452356 wait complete. 2024-03-26T19:00:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.680+0000 7f9f87c00640 1 Processor -- start 2024-03-26T19:00:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.681+0000 7f9f87c00640 1 -- start start 2024-03-26T19:00:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.681+0000 7f9f87c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f80076280 0x7f9f8019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.681+0000 7f9f87c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f800774c0 0x7f9f8019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.681+0000 7f9f87c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f80077e60 0x7f9f801a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.682+0000 7f9f85200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f80076280 0x7f9f8019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.682+0000 7f9f85200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f80076280 0x7f9f8019e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:39512/0 (socket says 172.21.15.97:39512) 2024-03-26T19:00:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.682+0000 7f9f85200640 1 -- 172.21.15.97:0/3127775349 learned_addr learned my addr 172.21.15.97:0/3127775349 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.682+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80113bb0 con 0x7f9f80077e60 2024-03-26T19:00:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.682+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80113a30 con 0x7f9f800774c0 2024-03-26T19:00:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.682+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80113d30 con 0x7f9f80076280 2024-03-26T19:00:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.683+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f800774c0 0x7f9f8019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.683+0000 7f9f85c00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f80077e60 0x7f9f801a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.684+0000 7f9f85200640 1 -- 172.21.15.97:0/3127775349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f800774c0 msgr2=0x7f9f8019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.684+0000 7f9f85200640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f800774c0 0x7f9f8019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.684+0000 7f9f85200640 1 -- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f80077e60 msgr2=0x7f9f801a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.684+0000 7f9f85200640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f80077e60 0x7f9f801a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.684+0000 7f9f85200640 1 -- 172.21.15.97:0/3127775349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f801a5cf0 con 0x7f9f80076280 2024-03-26T19:00:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.684+0000 7f9f85c00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f80077e60 0x7f9f801a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:00:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.685+0000 7f9f85200640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f80076280 0x7f9f8019e0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00ea60 tx=0x7f9f6c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.686+0000 7f9f7d600640 1 -- 172.21.15.97:0/3127775349 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c00cde0 con 0x7f9f80076280 2024-03-26T19:00:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.687+0000 7f9f7d600640 1 -- 172.21.15.97:0/3127775349 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c0035f0 con 0x7f9f80076280 2024-03-26T19:00:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.687+0000 7f9f7d600640 1 -- 172.21.15.97:0/3127775349 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c010670 con 0x7f9f80076280 2024-03-26T19:00:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.687+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f801a5fe0 con 0x7f9f80076280 2024-03-26T19:00:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.687+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f801a6390 con 0x7f9f80076280 2024-03-26T19:00:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.698+0000 7f9f7d600640 1 -- 172.21.15.97:0/3127775349 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f9f6c0037a0 con 0x7f9f80076280 2024-03-26T19:00:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.700+0000 7f9f7d600640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f640a3040 0x7f9f640a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.701+0000 7f9f7d600640 1 -- 172.21.15.97:0/3127775349 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9f6c0cf6a0 con 0x7f9f80076280 2024-03-26T19:00:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.703+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f640a3040 0x7f9f640a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.707+0000 7f9f8aa0b740 1 Processor -- start 2024-03-26T19:00:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.708+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f640a3040 0x7f9f640a5500 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9f8019f5e0 tx=0x7f9f70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.708+0000 7f9f8aa0b740 1 -- start start 2024-03-26T19:00:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.708+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11cda0 0x564b9d11d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.708+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11d790 0x564b9d122290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.708+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d122900 0x564b9d124d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.708+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564b9d081e80 con 0x564b9d11d790 2024-03-26T19:00:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.708+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564b9d02a660 con 0x564b9d11cda0 2024-03-26T19:00:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.708+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564b9d07de30 con 0x564b9d122900 2024-03-26T19:00:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.710+0000 7f9f85c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d122900 0x564b9d124d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.710+0000 7f9f85c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d122900 0x564b9d124d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:39514/0 (socket says 172.21.15.97:39514) 2024-03-26T19:00:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.710+0000 7f9f85c00640 1 -- 172.21.15.97:0/226594974 learned_addr learned my addr 172.21.15.97:0/226594974 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.710+0000 7f9f85200640 1 --2- 172.21.15.97:0/226594974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11cda0 0x564b9d11d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.710+0000 7f9f85200640 1 -- 172.21.15.97:0/226594974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d122900 msgr2=0x564b9d124d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.710+0000 7f9f85200640 1 --2- 172.21.15.97:0/226594974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d122900 0x564b9d124d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.710+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/226594974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11d790 0x564b9d122290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.710+0000 7f9f85200640 1 -- 172.21.15.97:0/226594974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11d790 msgr2=0x564b9d122290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.711+0000 7f9f85200640 1 --2- 172.21.15.97:0/226594974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11d790 0x564b9d122290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.711+0000 7f9f85200640 1 -- 172.21.15.97:0/226594974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9cd96210 con 0x564b9d11cda0 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.711+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/226594974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11d790 0x564b9d122290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.711+0000 7f9f85200640 1 --2- 172.21.15.97:0/226594974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11cda0 0x564b9d11d1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0cd220 tx=0x7f9f6c09ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28fc97011a49d6a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.711+0000 7f9f76000640 1 -- 172.21.15.97:0/226594974 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c09be00 con 0x564b9d11cda0 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.712+0000 7f9f76000640 1 -- 172.21.15.97:0/226594974 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c09ff50 con 0x564b9d11cda0 2024-03-26T19:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.712+0000 7f9f76000640 1 -- 172.21.15.97:0/226594974 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c09be00 con 0x564b9d11cda0 2024-03-26T19:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.714+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/226594974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11cda0 msgr2=0x564b9d11d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.714+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/226594974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11cda0 0x564b9d11d1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0cd220 tx=0x7f9f6c09ba00 comp rx=0 tx=0).stop 2024-03-26T19:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.715+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/226594974 shutdown_connections 2024-03-26T19:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.715+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/226594974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d122900 0x564b9d124d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.715+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/226594974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11d790 0x564b9d122290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.715+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/226594974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11cda0 0x564b9d11d1c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.715+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/226594974 >> 172.21.15.97:0/226594974 conn(0x564b9d118310 msgr2=0x564b9d118710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.717+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/226594974 shutdown_connections 2024-03-26T19:00:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.718+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/226594974 wait complete. 2024-03-26T19:00:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.720+0000 7f9f8aa0b740 1 -- start start 2024-03-26T19:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.721+0000 7f9f8aa0b740 1 Processor -- start 2024-03-26T19:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.723+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11cda0 0x564b9d126d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.723+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11d790 0x564b9d1272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.725+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11d790 0x564b9d1272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.725+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11d790 0x564b9d1272b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:39526/0 (socket says 172.21.15.97:39526) 2024-03-26T19:00:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.725+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 0x564b9d12e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.725+0000 7f9f85200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11cda0 0x564b9d126d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.725+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564b9d1257f0 con 0x564b9d11cda0 2024-03-26T19:00:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.725+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564b9d03dac0 con 0x564b9d122900 2024-03-26T19:00:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.725+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564b9d125970 con 0x564b9d11d790 2024-03-26T19:00:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.725+0000 7f9f7fe00640 1 -- 172.21.15.97:0/3251668181 learned_addr learned my addr 172.21.15.97:0/3251668181 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.725+0000 7f9f8aa0b740 10 client.0 _ll_register_callbacks cb 0x564b9d03d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:00:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.726+0000 7f9f85200640 1 -- 172.21.15.97:0/3251668181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11d790 msgr2=0x564b9d1272b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.726+0000 7f9f85200640 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11d790 0x564b9d1272b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.726+0000 7f9f85200640 1 -- 172.21.15.97:0/3251668181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 msgr2=0x564b9d12e2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:00:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.726+0000 7f9f85200640 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 0x564b9d12e2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.726+0000 7f9f85200640 1 -- 172.21.15.97:0/3251668181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9ce62620 con 0x564b9d11cda0 2024-03-26T19:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.728+0000 7f9f85200640 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11cda0 0x564b9d126d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0bc1a0 tx=0x7f9f6c09ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f32078bfe7c6d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.728+0000 7f9f6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T19:00:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.729+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3251668181 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c099ea0 con 0x564b9d11cda0 2024-03-26T19:00:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.729+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3251668181 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9f6c09f070 con 0x564b9d11cda0 2024-03-26T19:00:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.730+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3251668181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f6c09ff50 con 0x564b9d11cda0 2024-03-26T19:00:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.733+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3251668181 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f6c09a100 con 0x564b9d11cda0 2024-03-26T19:00:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.733+0000 7f9f8aa0b740 10 client.15264 fetch_fsmap learned FSMap version 27 2024-03-26T19:00:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.734+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3251668181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b9d1315d0 con 0x564b9d11cda0 2024-03-26T19:00:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.735+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3251668181 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c09ff50 con 0x564b9d11cda0 2024-03-26T19:00:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.735+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3251668181 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9f6c097b50 con 0x564b9d11cda0 2024-03-26T19:00:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.736+0000 7f9f8aa0b740 10 client.15264 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:00:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.736+0000 7f9f8aa0b740 10 client.15264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:00:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.736+0000 7f9f8aa0b740 20 client.15264 populate_metadata read hostname 'smithi097' 2024-03-26T19:00:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.736+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564b9d132920 0x564b9d134d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.736+0000 7f9f8aa0b740 4 client.15264 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:00:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.736+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3251668181 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b9d08f3c0 con 0x564b9d132920 2024-03-26T19:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.736+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564b9d132920 0x564b9d134d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.739+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564b9d132920 0x564b9d134d10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=727f6f74cee0d483 server_cookie=d70ac623ac851f7a in_seq=0 out_seq=0 2024-03-26T19:00:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.739+0000 7f9f6aa00640 10 client.15264 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.740+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3251668181 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564b9d08f3c0 con 0x564b9d132920 2024-03-26T19:00:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.740+0000 7f9f6aa00640 10 client.15264 handle_command_reply: tid=0 2024-03-26T19:00:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.740+0000 7f9f8aa0b740 1 client.15264 shutdown 2024-03-26T19:00:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.742+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3251668181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11cda0 msgr2=0x564b9d126d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.743+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11cda0 0x564b9d126d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0bc1a0 tx=0x7f9f6c09ba70 comp rx=0 tx=0).stop 2024-03-26T19:00:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.743+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3251668181 shutdown_connections 2024-03-26T19:00:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.743+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564b9d132920 0x564b9d134d10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.743+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 0x564b9d12e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.744+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11d790 0x564b9d1272b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.744+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3251668181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11cda0 0x564b9d126d70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.744+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3251668181 >> 172.21.15.97:0/3251668181 conn(0x564b9d118310 msgr2=0x564b9d117540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.744+0000 7f9f6aa00640 0 client.15264 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.744+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3251668181 shutdown_connections 2024-03-26T19:00:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.745+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3251668181 wait complete. 2024-03-26T19:00:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.745+0000 7f9f8aa0b740 20 client.15264 trim_cache size 0 max 16384 2024-03-26T19:00:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.773+0000 7f9f8aa0b740 1 Processor -- start 2024-03-26T19:00:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.773+0000 7f9f8aa0b740 1 -- start start 2024-03-26T19:00:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.774+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 0x564b9d127090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.774+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11d790 0x564b9d12e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.774+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 0x564b9d11e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.774+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801af400 con 0x564b9d11d790 2024-03-26T19:00:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.774+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801b0f40 con 0x564b9d122900 2024-03-26T19:00:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.774+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801afad0 con 0x564b9d11cda0 2024-03-26T19:00:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.774+0000 7f9f85c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 0x564b9d11e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.774+0000 7f9f85c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 0x564b9d11e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34760/0 (socket says 172.21.15.97:34760) 2024-03-26T19:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.774+0000 7f9f85c00640 1 -- 172.21.15.97:0/615480346 learned_addr learned my addr 172.21.15.97:0/615480346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.774+0000 7f9f85200640 1 --2- 172.21.15.97:0/615480346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 0x564b9d127090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.775+0000 7f9f85c00640 1 -- 172.21.15.97:0/615480346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 msgr2=0x564b9d127090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.775+0000 7f9f85c00640 1 --2- 172.21.15.97:0/615480346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 0x564b9d127090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.775+0000 7f9f85c00640 1 -- 172.21.15.97:0/615480346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11d790 msgr2=0x564b9d12e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.775+0000 7f9f85c00640 1 --2- 172.21.15.97:0/615480346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11d790 0x564b9d12e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.775+0000 7f9f85c00640 1 -- 172.21.15.97:0/615480346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9d119080 con 0x564b9d122900 2024-03-26T19:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.775+0000 7f9f85c00640 1 --2- 172.21.15.97:0/615480346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 0x564b9d11e020 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009bf0 tx=0x7f9f78009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb10efb85d94159e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.776+0000 7f9f75600640 1 -- 172.21.15.97:0/615480346 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78044030 con 0x564b9d122900 2024-03-26T19:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.776+0000 7f9f75600640 1 -- 172.21.15.97:0/615480346 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78032c90 con 0x564b9d122900 2024-03-26T19:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.776+0000 7f9f75600640 1 -- 172.21.15.97:0/615480346 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78039040 con 0x564b9d122900 2024-03-26T19:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.777+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/615480346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 msgr2=0x564b9d11e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.777+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/615480346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 0x564b9d11e020 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009bf0 tx=0x7f9f78009800 comp rx=0 tx=0).stop 2024-03-26T19:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.777+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/615480346 shutdown_connections 2024-03-26T19:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.778+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/615480346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d122900 0x564b9d11e020 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.778+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/615480346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d11d790 0x564b9d12e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.778+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/615480346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 0x564b9d127090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.778+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/615480346 >> 172.21.15.97:0/615480346 conn(0x564b9d118310 msgr2=0x564b9d126980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.778+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/615480346 shutdown_connections 2024-03-26T19:00:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.778+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/615480346 wait complete. 2024-03-26T19:00:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.779+0000 7f9f8aa0b740 1 -- start start 2024-03-26T19:00:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.779+0000 7f9f8aa0b740 1 Processor -- start 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.779+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 0x564b9d129810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.779+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11d790 0x564b9d129d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.779+0000 7f9f8aa0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d122900 0x564b9d12cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.779+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801b0870 con 0x564b9d122900 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.779+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801b01a0 con 0x564b9d11d790 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.779+0000 7f9f8aa0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801ad3e0 con 0x564b9d11cda0 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.779+0000 7f9f8aa0b740 10 client.0 _ll_register_callbacks cb 0x7f9f6c003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 0x564b9d129810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 0x564b9d129810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:39540/0 (socket says 172.21.15.97:39540) 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85200640 1 -- 172.21.15.97:0/3410176788 learned_addr learned my addr 172.21.15.97:0/3410176788 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11d790 0x564b9d129d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85c00640 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d122900 0x564b9d12cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85200640 1 -- 172.21.15.97:0/3410176788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11d790 msgr2=0x564b9d129d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85200640 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11d790 0x564b9d129d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85200640 1 -- 172.21.15.97:0/3410176788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d122900 msgr2=0x564b9d12cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85200640 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d122900 0x564b9d12cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85200640 1 -- 172.21.15.97:0/3410176788 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9d12d310 con 0x564b9d11cda0 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11d790 0x564b9d129d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85c00640 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d122900 0x564b9d12cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.780+0000 7f9f85200640 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 0x564b9d129810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0cd540 tx=0x7f9f6c0bc090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1664f40c2212d3b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.781+0000 7f9f6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.781+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3410176788 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c09ff50 con 0x564b9d11cda0 2024-03-26T19:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.781+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3410176788 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9f6c09f070 con 0x564b9d11cda0 2024-03-26T19:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.781+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3410176788 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f78032c90 con 0x564b9d11cda0 2024-03-26T19:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.782+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3410176788 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f6c09be00 con 0x564b9d11cda0 2024-03-26T19:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.782+0000 7f9f8aa0b740 10 client.25399 fetch_fsmap learned FSMap version 27 2024-03-26T19:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.782+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3410176788 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b9d1a9ab0 con 0x564b9d11cda0 2024-03-26T19:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.783+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3410176788 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78032c90 con 0x564b9d11cda0 2024-03-26T19:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.783+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3410176788 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9f6c09b250 con 0x564b9d11cda0 2024-03-26T19:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.783+0000 7f9f8aa0b740 10 client.25399 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.783+0000 7f9f8aa0b740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.783+0000 7f9f8aa0b740 20 client.25399 populate_metadata read hostname 'smithi097' 2024-03-26T19:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.783+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564b9d1aa000 0x564b9d1ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.783+0000 7f9f8aa0b740 4 client.25399 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.783+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3410176788 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b9d089710 con 0x564b9d1aa000 2024-03-26T19:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.784+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564b9d1aa000 0x564b9d1ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:00:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.785+0000 7f9f6aa00640 10 client.25399 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.785+0000 7f9f7fe00640 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564b9d1aa000 0x564b9d1ac420 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4059da6914be25f5 server_cookie=6b8a2f814c4274d7 in_seq=0 out_seq=0 2024-03-26T19:00:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.803+0000 7f9f6aa00640 1 -- 172.21.15.97:0/3410176788 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b9d089710 con 0x564b9d1aa000 2024-03-26T19:00:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.803+0000 7f9f6aa00640 10 client.25399 handle_command_reply: tid=0 2024-03-26T19:00:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.804+0000 7f9f8aa0b740 1 client.25399 shutdown 2024-03-26T19:00:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3410176788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 msgr2=0x564b9d129810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 0x564b9d129810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0cd540 tx=0x7f9f6c0bc090 comp rx=0 tx=0).stop 2024-03-26T19:00:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3410176788 shutdown_connections 2024-03-26T19:00:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564b9d1aa000 0x564b9d1ac420 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b9d122900 0x564b9d12cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b9d11d790 0x564b9d129d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f6aa00640 0 client.25399 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:00:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 1 --2- 172.21.15.97:0/3410176788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b9d11cda0 0x564b9d129810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3410176788 >> 172.21.15.97:0/3410176788 conn(0x564b9d118310 msgr2=0x564b9d116cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3410176788 shutdown_connections 2024-03-26T19:00:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 1 -- 172.21.15.97:0/3410176788 wait complete. 2024-03-26T19:00:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.805+0000 7f9f8aa0b740 20 client.25399 trim_cache size 0 max 16384 2024-03-26T19:00:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:00:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:00:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:00:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:00:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.806+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f640a3040 msgr2=0x7f9f640a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.806+0000 7f9f87c00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f640a3040 0x7f9f640a5500 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9f8019f5e0 tx=0x7f9f70005f90 comp rx=0 tx=0).stop 2024-03-26T19:00:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.806+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f80076280 msgr2=0x7f9f8019e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:00:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.806+0000 7f9f87c00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f80076280 0x7f9f8019e0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00ea60 tx=0x7f9f6c00ef20 comp rx=0 tx=0).stop 2024-03-26T19:00:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.806+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 shutdown_connections 2024-03-26T19:00:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.806+0000 7f9f87c00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f640a3040 0x7f9f640a5500 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.806+0000 7f9f87c00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f80077e60 0x7f9f801a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.806+0000 7f9f87c00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f800774c0 0x7f9f8019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.806+0000 7f9f87c00640 1 --2- 172.21.15.97:0/3127775349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f80076280 0x7f9f8019e0d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:00:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.806+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 >> 172.21.15.97:0/3127775349 conn(0x7f9f800ff0b0 msgr2=0x7f9f80100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:00:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.807+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 shutdown_connections 2024-03-26T19:00:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:00:56.807+0000 7f9f87c00640 1 -- 172.21.15.97:0/3127775349 wait complete. 2024-03-26T19:00:58.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:00:58 smithi144 ceph-mon[36053]: pgmap v451: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 614 op/s 2024-03-26T19:00:58.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:00:58 smithi105 ceph-mon[36270]: pgmap v451: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 614 op/s 2024-03-26T19:00:58.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:00:58 smithi097 ceph-mon[31955]: pgmap v451: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 614 op/s 2024-03-26T19:01:00.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 95.0 s, 2539.0 tps, lat 39.213 ms stddev 47.465 2024-03-26T19:01:00.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:00 smithi144 ceph-mon[36053]: pgmap v452: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 834 op/s 2024-03-26T19:01:00.460 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:00 smithi105 ceph-mon[36270]: pgmap v452: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 834 op/s 2024-03-26T19:01:00.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:00 smithi097 ceph-mon[31955]: pgmap v452: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 834 op/s 2024-03-26T19:01:02.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:02 smithi144 ceph-mon[36053]: pgmap v453: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 721 op/s 2024-03-26T19:01:02.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:02 smithi105 ceph-mon[36270]: pgmap v453: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 721 op/s 2024-03-26T19:01:02.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:02 smithi097 ceph-mon[31955]: pgmap v453: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 721 op/s 2024-03-26T19:01:04.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:04 smithi144 ceph-mon[36053]: pgmap v454: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 643 op/s 2024-03-26T19:01:04.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:04 smithi105 ceph-mon[36270]: pgmap v454: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 643 op/s 2024-03-26T19:01:04.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:04 smithi097 ceph-mon[31955]: pgmap v454: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 643 op/s 2024-03-26T19:01:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 100.0 s, 2946.2 tps, lat 34.069 ms stddev 38.149 2024-03-26T19:01:05.376 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:05 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:01:05.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:05 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:01:05.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:05 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:01:06.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:06 smithi105 ceph-mon[36270]: pgmap v455: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 766 op/s 2024-03-26T19:01:06.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:06 smithi097 ceph-mon[31955]: pgmap v455: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 766 op/s 2024-03-26T19:01:06.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:06 smithi144 ceph-mon[36053]: pgmap v455: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 766 op/s 2024-03-26T19:01:07.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:01:07.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:01:07.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:01:07.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:01:07.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:01:07.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:01:07.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:01:07.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:01:07.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:01:08.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:08 smithi105 ceph-mon[36270]: pgmap v456: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 683 op/s 2024-03-26T19:01:08.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:08 smithi097 ceph-mon[31955]: pgmap v456: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 683 op/s 2024-03-26T19:01:08.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:08 smithi144 ceph-mon[36053]: pgmap v456: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 683 op/s 2024-03-26T19:01:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:09.999+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3684871039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf4109a30 msgr2=0x7fcbf4110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:09.999+0000 7fcbf9c00640 1 --2- 172.21.15.97:0/3684871039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf4109a30 0x7fcbf4110350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009ca0 tx=0x7fcbec02c100 comp rx=0 tx=0).stop 2024-03-26T19:01:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:09.999+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3684871039 shutdown_connections 2024-03-26T19:01:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:09.999+0000 7fcbf9c00640 1 --2- 172.21.15.97:0/3684871039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf4109a30 0x7fcbf4110350 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:09.999+0000 7fcbf9c00640 1 --2- 172.21.15.97:0/3684871039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf4109070 0x7fcbf41094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.000+0000 7fcbf9c00640 1 --2- 172.21.15.97:0/3684871039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf4107f20 0x7fcbf4108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.000+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3684871039 >> 172.21.15.97:0/3684871039 conn(0x7fcbf4076ed0 msgr2=0x7fcbf40792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.000+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3684871039 shutdown_connections 2024-03-26T19:01:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.000+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3684871039 wait complete. 2024-03-26T19:01:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.003+0000 7fcbf9c00640 1 Processor -- start 2024-03-26T19:01:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.003+0000 7fcbf9c00640 1 -- start start 2024-03-26T19:01:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.004+0000 7fcbf9c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf4107f20 0x7fcbf407ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.004+0000 7fcbf9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf4109070 0x7fcbf407f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.004+0000 7fcbf9c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf4109a30 0x7fcbf41ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.004+0000 7fcbf9c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf4113bc0 con 0x7fcbf4109070 2024-03-26T19:01:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.004+0000 7fcbf9c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf4113a40 con 0x7fcbf4107f20 2024-03-26T19:01:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.004+0000 7fcbf9c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf4113d40 con 0x7fcbf4109a30 2024-03-26T19:01:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.004+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf4107f20 0x7fcbf407ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.004+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf4107f20 0x7fcbf407ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34504/0 (socket says 172.21.15.97:34504) 2024-03-26T19:01:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.004+0000 7fcbf2a00640 1 -- 172.21.15.97:0/3539741451 learned_addr learned my addr 172.21.15.97:0/3539741451 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbf2a00640 1 -- 172.21.15.97:0/3539741451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf4109a30 msgr2=0x7fcbf41ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbf3400640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf4109a30 0x7fcbf41ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf4109a30 0x7fcbf41ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbf2a00640 1 -- 172.21.15.97:0/3539741451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf4109070 msgr2=0x7fcbf407f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf4109070 0x7fcbf407f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbf2a00640 1 -- 172.21.15.97:0/3539741451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf4082630 con 0x7fcbf4107f20 2024-03-26T19:01:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbf3400640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf4109a30 0x7fcbf41ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:01:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf4107f20 0x7fcbf407ef70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400d950 tx=0x7fcbe400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbeb400640 1 -- 172.21.15.97:0/3539741451 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4017030 con 0x7fcbf4107f20 2024-03-26T19:01:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbeb400640 1 -- 172.21.15.97:0/3539741451 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe4004020 con 0x7fcbf4107f20 2024-03-26T19:01:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.005+0000 7fcbeb400640 1 -- 172.21.15.97:0/3539741451 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4003d80 con 0x7fcbf4107f20 2024-03-26T19:01:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.006+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3539741451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf41ae810 con 0x7fcbf4107f20 2024-03-26T19:01:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.006+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3539741451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf41aed10 con 0x7fcbf4107f20 2024-03-26T19:01:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.011+0000 7fcbfc87b740 1 Processor -- start 2024-03-26T19:01:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.011+0000 7fcbfc87b740 1 -- start start 2024-03-26T19:01:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.013+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.013+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 0x55bdfb0fa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.013+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 0x55bdfb0fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.013+0000 7fcbfc87b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bdfb059e80 con 0x55bdfb0f4da0 2024-03-26T19:01:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.013+0000 7fcbfc87b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bdfb002660 con 0x55bdfb0fa900 2024-03-26T19:01:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.013+0000 7fcbfc87b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bdfb055e30 con 0x55bdfb0f5790 2024-03-26T19:01:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.014+0000 7fcbebe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 0x55bdfb0fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.014+0000 7fcbebe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 0x55bdfb0fcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34512/0 (socket says 172.21.15.97:34512) 2024-03-26T19:01:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.015+0000 7fcbebe00640 1 -- 172.21.15.97:0/1238249527 learned_addr learned my addr 172.21.15.97:0/1238249527 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.016+0000 7fcbf3400640 1 --2- 172.21.15.97:0/1238249527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.017+0000 7fcbeb400640 1 -- 172.21.15.97:0/3539741451 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fcbe401a070 con 0x7fcbf4107f20 2024-03-26T19:01:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.017+0000 7fcbebe00640 1 -- 172.21.15.97:0/1238249527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 msgr2=0x55bdfb0fa290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:01:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.017+0000 7fcbebe00640 1 --2- 172.21.15.97:0/1238249527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 0x55bdfb0fa290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.017+0000 7fcbebe00640 1 -- 172.21.15.97:0/1238249527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0f4da0 msgr2=0x55bdfb0f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.017+0000 7fcbebe00640 1 --2- 172.21.15.97:0/1238249527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.017+0000 7fcbebe00640 1 -- 172.21.15.97:0/1238249527 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdfad6e210 con 0x55bdfb0fa900 2024-03-26T19:01:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.018+0000 7fcbebe00640 1 --2- 172.21.15.97:0/1238249527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 0x55bdfb0fcd90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009b40 tx=0x7fcbdc00e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5ef0f1b4bd8eccb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.020+0000 7fcbe3e00640 1 -- 172.21.15.97:0/1238249527 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc00ed80 con 0x55bdfb0fa900 2024-03-26T19:01:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.020+0000 7fcbe3e00640 1 -- 172.21.15.97:0/1238249527 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc005500 con 0x55bdfb0fa900 2024-03-26T19:01:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.020+0000 7fcbe3e00640 1 -- 172.21.15.97:0/1238249527 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc010690 con 0x55bdfb0fa900 2024-03-26T19:01:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.021+0000 7fcbfc87b740 1 -- 172.21.15.97:0/1238249527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 msgr2=0x55bdfb0fcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.021+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/1238249527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 0x55bdfb0fcd90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009b40 tx=0x7fcbdc00e350 comp rx=0 tx=0).stop 2024-03-26T19:01:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.023+0000 7fcbeb400640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcbd80a3040 0x7fcbd80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.023+0000 7fcbf3400640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcbd80a3040 0x7fcbd80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.024+0000 7fcbeb400640 1 -- 172.21.15.97:0/3539741451 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcbe40ce340 con 0x7fcbf4107f20 2024-03-26T19:01:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.025+0000 7fcbfc87b740 1 -- 172.21.15.97:0/1238249527 shutdown_connections 2024-03-26T19:01:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.025+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/1238249527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 0x55bdfb0fcd90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.026+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/1238249527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 0x55bdfb0fa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.026+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/1238249527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.026+0000 7fcbfc87b740 1 -- 172.21.15.97:0/1238249527 >> 172.21.15.97:0/1238249527 conn(0x55bdfb0f0310 msgr2=0x55bdfb0f0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.027+0000 7fcbf3400640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcbd80a3040 0x7fcbd80a5500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0324c0 tx=0x7fcbec004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.028+0000 7fcbfc87b740 1 -- 172.21.15.97:0/1238249527 shutdown_connections 2024-03-26T19:01:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.028+0000 7fcbfc87b740 1 -- 172.21.15.97:0/1238249527 wait complete. 2024-03-26T19:01:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.029+0000 7fcbfc87b740 1 -- start start 2024-03-26T19:01:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.029+0000 7fcbfc87b740 1 Processor -- start 2024-03-26T19:01:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.031+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0fede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 105.0 s, 2495.0 tps, lat 39.840 ms stddev 41.749 2024-03-26T19:01:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.031+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0f5790 0x55bdfb0ff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.031+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 0x55bdfb106360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.031+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.031+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0fede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52498/0 (socket says 172.21.15.97:52498) 2024-03-26T19:01:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.032+0000 7fcbf2a00640 1 -- 172.21.15.97:0/538222276 learned_addr learned my addr 172.21.15.97:0/538222276 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.032+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bdfb0fd8b0 con 0x55bdfb0f5790 2024-03-26T19:01:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.032+0000 7fcbf3400640 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 0x55bdfb106360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.032+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bdfb015ac0 con 0x55bdfb0fa900 2024-03-26T19:01:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.032+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bdfb0fda30 con 0x55bdfb0f4da0 2024-03-26T19:01:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.032+0000 7fcbfc87b740 10 client.0 _ll_register_callbacks cb 0x55bdfb015490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:01:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.033+0000 7fcbf2a00640 1 -- 172.21.15.97:0/538222276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 msgr2=0x55bdfb106360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.033+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 0x55bdfb106360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.033+0000 7fcbf2a00640 1 -- 172.21.15.97:0/538222276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0f5790 msgr2=0x55bdfb0ff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.033+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0f5790 0x55bdfb0ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.033+0000 7fcbf2a00640 1 -- 172.21.15.97:0/538222276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdfae3a620 con 0x55bdfb0f4da0 2024-03-26T19:01:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.033+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0fede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40a2f80 tx=0x7fcbe409a550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5f4c7e05b0c703a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.033+0000 7fcbe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:01:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.033+0000 7fcbe0c00640 1 -- 172.21.15.97:0/538222276 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe409be60 con 0x55bdfb0f4da0 2024-03-26T19:01:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.033+0000 7fcbe0c00640 1 -- 172.21.15.97:0/538222276 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcbe40971a0 con 0x55bdfb0f4da0 2024-03-26T19:01:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.034+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbdc005500 con 0x55bdfb0f4da0 2024-03-26T19:01:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.036+0000 7fcbe0c00640 1 -- 172.21.15.97:0/538222276 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcbe40a34b0 con 0x55bdfb0f4da0 2024-03-26T19:01:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.036+0000 7fcbfc87b740 10 client.25405 fetch_fsmap learned FSMap version 27 2024-03-26T19:01:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.036+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bdfb1097a0 con 0x55bdfb0f4da0 2024-03-26T19:01:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.037+0000 7fcbe0c00640 1 -- 172.21.15.97:0/538222276 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc005500 con 0x55bdfb0f4da0 2024-03-26T19:01:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.038+0000 7fcbe0c00640 1 -- 172.21.15.97:0/538222276 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcbe4096880 con 0x55bdfb0f4da0 2024-03-26T19:01:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.038+0000 7fcbfc87b740 10 client.25405 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:01:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.038+0000 7fcbfc87b740 10 client.25405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:01:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.039+0000 7fcbfc87b740 20 client.25405 populate_metadata read hostname 'smithi097' 2024-03-26T19:01:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.039+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55bdfb10ae90 0x55bdfb10d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.039+0000 7fcbfc87b740 4 client.25405 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:01:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.039+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdfb0673c0 con 0x55bdfb10ae90 2024-03-26T19:01:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.039+0000 7fcbebe00640 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55bdfb10ae90 0x55bdfb10d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.041+0000 7fcbe0c00640 10 client.25405 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.041+0000 7fcbebe00640 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55bdfb10ae90 0x55bdfb10d2a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c666257da28387 server_cookie=185114e2052009ec in_seq=0 out_seq=0 2024-03-26T19:01:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.042+0000 7fcbe0c00640 1 -- 172.21.15.97:0/538222276 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bdfb0673c0 con 0x55bdfb10ae90 2024-03-26T19:01:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.042+0000 7fcbe0c00640 10 client.25405 handle_command_reply: tid=0 2024-03-26T19:01:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.043+0000 7fcbfc87b740 1 client.25405 shutdown 2024-03-26T19:01:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.043+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 msgr2=0x55bdfb0fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.043+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0fede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40a2f80 tx=0x7fcbe409a550 comp rx=0 tx=0).stop 2024-03-26T19:01:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.044+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 shutdown_connections 2024-03-26T19:01:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.044+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55bdfb10ae90 0x55bdfb10d2a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.044+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0fa900 0x55bdfb106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.044+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0f5790 0x55bdfb0ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.044+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/538222276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0fede0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.044+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 >> 172.21.15.97:0/538222276 conn(0x55bdfb0f0310 msgr2=0x55bdfb0ef540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.044+0000 7fcbe0c00640 0 client.25405 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.044+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 shutdown_connections 2024-03-26T19:01:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.044+0000 7fcbfc87b740 1 -- 172.21.15.97:0/538222276 wait complete. 2024-03-26T19:01:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.046+0000 7fcbfc87b740 20 client.25405 trim_cache size 0 max 16384 2024-03-26T19:01:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.084+0000 7fcbfc87b740 1 Processor -- start 2024-03-26T19:01:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.084+0000 7fcbfc87b740 1 -- start start 2024-03-26T19:01:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0ff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f5790 0x55bdfb0eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 0x55bdfb0ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbfc87b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41b7a40 con 0x55bdfb0fa900 2024-03-26T19:01:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbfc87b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41b9580 con 0x55bdfb0f5790 2024-03-26T19:01:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbfc87b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41b8110 con 0x55bdfb0f4da0 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbebe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f5790 0x55bdfb0eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbebe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f5790 0x55bdfb0eecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34538/0 (socket says 172.21.15.97:34538) 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbebe00640 1 -- 172.21.15.97:0/3679650693 learned_addr learned my addr 172.21.15.97:0/3679650693 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbf3400640 1 --2- 172.21.15.97:0/3679650693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 0x55bdfb0ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbf2a00640 1 -- 172.21.15.97:0/3679650693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f5790 msgr2=0x55bdfb0eecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/3679650693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f5790 0x55bdfb0eecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbf2a00640 1 -- 172.21.15.97:0/3679650693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 msgr2=0x55bdfb0ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/3679650693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 0x55bdfb0ef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbf2a00640 1 -- 172.21.15.97:0/3679650693 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdfb10d920 con 0x55bdfb0f4da0 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbebe00640 1 --2- 172.21.15.97:0/3679650693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f5790 0x55bdfb0eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.085+0000 7fcbf3400640 1 --2- 172.21.15.97:0/3679650693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 0x55bdfb0ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.086+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/3679650693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0ff100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40cbec0 tx=0x7fcbe4099040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71d1e9a3d13e3267 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.086+0000 7fcbe3400640 1 -- 172.21.15.97:0/3679650693 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe409be60 con 0x55bdfb0f4da0 2024-03-26T19:01:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.086+0000 7fcbe3400640 1 -- 172.21.15.97:0/3679650693 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe4099cf0 con 0x55bdfb0f4da0 2024-03-26T19:01:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.086+0000 7fcbe3400640 1 -- 172.21.15.97:0/3679650693 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40a8080 con 0x55bdfb0f4da0 2024-03-26T19:01:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.087+0000 7fcbfc87b740 1 -- 172.21.15.97:0/3679650693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 msgr2=0x55bdfb0ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.087+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/3679650693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0ff100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40cbec0 tx=0x7fcbe4099040 comp rx=0 tx=0).stop 2024-03-26T19:01:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.088+0000 7fcbfc87b740 1 -- 172.21.15.97:0/3679650693 shutdown_connections 2024-03-26T19:01:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.088+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/3679650693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 0x55bdfb0ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.088+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/3679650693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f5790 0x55bdfb0eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.088+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/3679650693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f4da0 0x55bdfb0ff100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.088+0000 7fcbfc87b740 1 -- 172.21.15.97:0/3679650693 >> 172.21.15.97:0/3679650693 conn(0x55bdfb0f0310 msgr2=0x55bdfb0fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.088+0000 7fcbfc87b740 1 -- 172.21.15.97:0/3679650693 shutdown_connections 2024-03-26T19:01:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.088+0000 7fcbfc87b740 1 -- 172.21.15.97:0/3679650693 wait complete. 2024-03-26T19:01:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.089+0000 7fcbfc87b740 1 -- start start 2024-03-26T19:01:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.089+0000 7fcbfc87b740 1 Processor -- start 2024-03-26T19:01:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.090+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f4da0 0x55bdfb1037c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.090+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 0x55bdfb103d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.090+0000 7fcbfc87b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 0x55bdfb104d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.091+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f4da0 0x55bdfb1037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.091+0000 7fcbebe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 0x55bdfb103d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.091+0000 7fcbebe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 0x55bdfb103d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52514/0 (socket says 172.21.15.97:52514) 2024-03-26T19:01:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.091+0000 7fcbebe00640 1 -- 172.21.15.97:0/917020642 learned_addr learned my addr 172.21.15.97:0/917020642 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.091+0000 7fcbf3400640 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 0x55bdfb104d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.092+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41b8eb0 con 0x55bdfb0fa900 2024-03-26T19:01:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.092+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41b87e0 con 0x55bdfb0f4da0 2024-03-26T19:01:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.092+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf41b5a20 con 0x55bdfb0f5790 2024-03-26T19:01:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.092+0000 7fcbfc87b740 10 client.0 _ll_register_callbacks cb 0x7fcbe409a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:01:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.092+0000 7fcbebe00640 1 -- 172.21.15.97:0/917020642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f4da0 msgr2=0x55bdfb1037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.092+0000 7fcbebe00640 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f4da0 0x55bdfb1037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.092+0000 7fcbebe00640 1 -- 172.21.15.97:0/917020642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 msgr2=0x55bdfb104d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.092+0000 7fcbebe00640 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 0x55bdfb104d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.093+0000 7fcbebe00640 1 -- 172.21.15.97:0/917020642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdfb105300 con 0x55bdfb0f5790 2024-03-26T19:01:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.093+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f4da0 0x55bdfb1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:01:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.093+0000 7fcbebe00640 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 0x55bdfb103d00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc005940 tx=0x7fcbdc009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c18138cd0f82269 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.093+0000 7fcbe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:01:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.093+0000 7fcbe0c00640 1 -- 172.21.15.97:0/917020642 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc02e060 con 0x55bdfb0f5790 2024-03-26T19:01:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.093+0000 7fcbe0c00640 1 -- 172.21.15.97:0/917020642 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcbdc00ed80 con 0x55bdfb0f5790 2024-03-26T19:01:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.094+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe4099cf0 con 0x55bdfb0f5790 2024-03-26T19:01:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.094+0000 7fcbe0c00640 1 -- 172.21.15.97:0/917020642 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcbdc01d040 con 0x55bdfb0f5790 2024-03-26T19:01:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.094+0000 7fcbfc87b740 10 client.25411 fetch_fsmap learned FSMap version 27 2024-03-26T19:01:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.094+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bdfb171210 con 0x55bdfb0f5790 2024-03-26T19:01:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.095+0000 7fcbe0c00640 1 -- 172.21.15.97:0/917020642 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe4099cf0 con 0x55bdfb0f5790 2024-03-26T19:01:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.095+0000 7fcbe0c00640 1 -- 172.21.15.97:0/917020642 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcbdc005000 con 0x55bdfb0f5790 2024-03-26T19:01:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.096+0000 7fcbfc87b740 10 client.25411 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:01:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.096+0000 7fcbfc87b740 10 client.25411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:01:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.096+0000 7fcbfc87b740 20 client.25411 populate_metadata read hostname 'smithi097' 2024-03-26T19:01:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.096+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55bdfb171a30 0x55bdfb173e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.096+0000 7fcbfc87b740 4 client.25411 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:01:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.096+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdfb061710 con 0x55bdfb171a30 2024-03-26T19:01:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.096+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55bdfb171a30 0x55bdfb173e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.097+0000 7fcbf2a00640 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55bdfb171a30 0x55bdfb173e50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a20e5b2c30f1653 server_cookie=870c166638f7018d in_seq=0 out_seq=0 2024-03-26T19:01:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.097+0000 7fcbe0c00640 10 client.25411 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.114+0000 7fcbe0c00640 1 -- 172.21.15.97:0/917020642 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdfb061710 con 0x55bdfb171a30 2024-03-26T19:01:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.114+0000 7fcbe0c00640 10 client.25411 handle_command_reply: tid=0 2024-03-26T19:01:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.114+0000 7fcbfc87b740 1 client.25411 shutdown 2024-03-26T19:01:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.114+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 msgr2=0x55bdfb103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.114+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 0x55bdfb103d00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc005940 tx=0x7fcbdc009760 comp rx=0 tx=0).stop 2024-03-26T19:01:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.115+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 shutdown_connections 2024-03-26T19:01:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.115+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55bdfb171a30 0x55bdfb173e50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.115+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdfb0fa900 0x55bdfb104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.115+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdfb0f5790 0x55bdfb103d00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.115+0000 7fcbfc87b740 1 --2- 172.21.15.97:0/917020642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdfb0f4da0 0x55bdfb1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.115+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 >> 172.21.15.97:0/917020642 conn(0x55bdfb0f0310 msgr2=0x55bdfb0f61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.115+0000 7fcbe0c00640 0 client.25411 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.115+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 shutdown_connections 2024-03-26T19:01:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.115+0000 7fcbfc87b740 1 -- 172.21.15.97:0/917020642 wait complete. 2024-03-26T19:01:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.116+0000 7fcbfc87b740 20 client.25411 trim_cache size 0 max 16384 2024-03-26T19:01:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:01:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:01:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:01:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:01:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.117+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3539741451 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcbd80a3040 msgr2=0x7fcbd80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.117+0000 7fcbf9c00640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcbd80a3040 0x7fcbd80a5500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0324c0 tx=0x7fcbec004a40 comp rx=0 tx=0).stop 2024-03-26T19:01:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.117+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3539741451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf4107f20 msgr2=0x7fcbf407ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.117+0000 7fcbf9c00640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf4107f20 0x7fcbf407ef70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400d950 tx=0x7fcbe400de10 comp rx=0 tx=0).stop 2024-03-26T19:01:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.117+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3539741451 shutdown_connections 2024-03-26T19:01:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.117+0000 7fcbf9c00640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fcbd80a3040 0x7fcbd80a5500 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.117+0000 7fcbf9c00640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf4109a30 0x7fcbf41ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.117+0000 7fcbf9c00640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf4109070 0x7fcbf407f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.117+0000 7fcbf9c00640 1 --2- 172.21.15.97:0/3539741451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf4107f20 0x7fcbf407ef70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.117+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3539741451 >> 172.21.15.97:0/3539741451 conn(0x7fcbf4076ed0 msgr2=0x7fcbf410c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.118+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3539741451 shutdown_connections 2024-03-26T19:01:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:10.118+0000 7fcbf9c00640 1 -- 172.21.15.97:0/3539741451 wait complete. 2024-03-26T19:01:10.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:10 smithi105 ceph-mon[36270]: pgmap v457: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 826 op/s 2024-03-26T19:01:10.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:10 smithi097 ceph-mon[31955]: pgmap v457: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 826 op/s 2024-03-26T19:01:10.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:10 smithi144 ceph-mon[36053]: pgmap v457: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 826 op/s 2024-03-26T19:01:12.429 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:12 smithi144 ceph-mon[36053]: pgmap v458: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 688 op/s 2024-03-26T19:01:12.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:12 smithi105 ceph-mon[36270]: pgmap v458: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 688 op/s 2024-03-26T19:01:12.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:12 smithi097 ceph-mon[31955]: pgmap v458: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 688 op/s 2024-03-26T19:01:14.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:14 smithi105 ceph-mon[36270]: pgmap v459: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 MiB/s wr, 687 op/s 2024-03-26T19:01:14.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:14 smithi097 ceph-mon[31955]: pgmap v459: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 MiB/s wr, 687 op/s 2024-03-26T19:01:14.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:14 smithi144 ceph-mon[36053]: pgmap v459: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 MiB/s wr, 687 op/s 2024-03-26T19:01:15.023 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 110.0 s, 2534.4 tps, lat 39.402 ms stddev 45.150 2024-03-26T19:01:16.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:16 smithi105 ceph-mon[36270]: pgmap v460: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 775 op/s 2024-03-26T19:01:16.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:16 smithi097 ceph-mon[31955]: pgmap v460: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 775 op/s 2024-03-26T19:01:16.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:16 smithi144 ceph-mon[36053]: pgmap v460: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 775 op/s 2024-03-26T19:01:18.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:18 smithi105 ceph-mon[36270]: pgmap v461: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.8 MiB/s wr, 603 op/s 2024-03-26T19:01:18.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:18 smithi097 ceph-mon[31955]: pgmap v461: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.8 MiB/s wr, 603 op/s 2024-03-26T19:01:18.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:18 smithi144 ceph-mon[36053]: pgmap v461: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.8 MiB/s wr, 603 op/s 2024-03-26T19:01:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 115.0 s, 2867.2 tps, lat 35.167 ms stddev 39.798 2024-03-26T19:01:20.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:20 smithi105 ceph-mon[36270]: pgmap v462: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 779 op/s 2024-03-26T19:01:20.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:20 smithi097 ceph-mon[31955]: pgmap v462: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 779 op/s 2024-03-26T19:01:20.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:20 smithi144 ceph-mon[36053]: pgmap v462: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 779 op/s 2024-03-26T19:01:22.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:22 smithi105 ceph-mon[36270]: pgmap v463: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-03-26T19:01:22.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:22 smithi097 ceph-mon[31955]: pgmap v463: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-03-26T19:01:22.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:22 smithi144 ceph-mon[36053]: pgmap v463: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-03-26T19:01:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.303+0000 7fb3c6400640 1 -- 172.21.15.97:0/804588984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0102930 msgr2=0x7fb3c0102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.303+0000 7fb3c6400640 1 --2- 172.21.15.97:0/804588984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0102930 0x7fb3c0102db0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009900 tx=0x7fb3ac02bc30 comp rx=0 tx=0).stop 2024-03-26T19:01:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.303+0000 7fb3c6400640 1 -- 172.21.15.97:0/804588984 shutdown_connections 2024-03-26T19:01:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.303+0000 7fb3c6400640 1 --2- 172.21.15.97:0/804588984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c01032f0 0x7fb3c0109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.303+0000 7fb3c6400640 1 --2- 172.21.15.97:0/804588984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c0102930 0x7fb3c0102db0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.303+0000 7fb3c6400640 1 --2- 172.21.15.97:0/804588984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c0101740 0x7fb3c0101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.303+0000 7fb3c6400640 1 -- 172.21.15.97:0/804588984 >> 172.21.15.97:0/804588984 conn(0x7fb3c00fccd0 msgr2=0x7fb3c00ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.304+0000 7fb3c6400640 1 -- 172.21.15.97:0/804588984 shutdown_connections 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.304+0000 7fb3c6400640 1 -- 172.21.15.97:0/804588984 wait complete. 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c6400640 1 Processor -- start 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c6400640 1 -- start start 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c6400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c0101740 0x7fb3c006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c6400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c0102930 0x7fb3c006c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c6400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c01032f0 0x7fb3c01ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c6400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c0075800 con 0x7fb3c0102930 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c6400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c0075680 con 0x7fb3c01032f0 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c6400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c0075980 con 0x7fb3c0101740 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c0101740 0x7fb3c006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c0101740 0x7fb3c006c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51118/0 (socket says 172.21.15.97:51118) 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3c5000640 1 -- 172.21.15.97:0/3423320511 learned_addr learned my addr 172.21.15.97:0/3423320511 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.305+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c0102930 0x7fb3c006c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.306+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c01032f0 0x7fb3c01ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.306+0000 7fb3c5000640 1 -- 172.21.15.97:0/3423320511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c01032f0 msgr2=0x7fb3c01ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.306+0000 7fb3c5000640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c01032f0 0x7fb3c01ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.306+0000 7fb3c5000640 1 -- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c0102930 msgr2=0x7fb3c006c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.306+0000 7fb3c5000640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c0102930 0x7fb3c006c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.306+0000 7fb3c5000640 1 -- 172.21.15.97:0/3423320511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c01ae560 con 0x7fb3c0101740 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.306+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c0102930 0x7fb3c006c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.306+0000 7fb3c5000640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c0101740 0x7fb3c006c360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400e9c0 tx=0x7fb3b400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.307+0000 7fb3bd600640 1 -- 172.21.15.97:0/3423320511 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400cde0 con 0x7fb3c0101740 2024-03-26T19:01:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.307+0000 7fb3bd600640 1 -- 172.21.15.97:0/3423320511 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b4010910 con 0x7fb3c0101740 2024-03-26T19:01:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.307+0000 7fb3bd600640 1 -- 172.21.15.97:0/3423320511 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4018a00 con 0x7fb3c0101740 2024-03-26T19:01:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.307+0000 7fb3c6400640 1 -- 172.21.15.97:0/3423320511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c01ae7f0 con 0x7fb3c0101740 2024-03-26T19:01:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.307+0000 7fb3c6400640 1 -- 172.21.15.97:0/3423320511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c01aecc0 con 0x7fb3c0101740 2024-03-26T19:01:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.312+0000 7fb3ca610740 1 Processor -- start 2024-03-26T19:01:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.312+0000 7fb3ca610740 1 -- start start 2024-03-26T19:01:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.312+0000 7fb3bd600640 1 -- 172.21.15.97:0/3423320511 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fb3b401f020 con 0x7fb3c0101740 2024-03-26T19:01:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.312+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c022f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.312+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c0234450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.312+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 0x5581c0236f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.312+0000 7fb3ca610740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5581c0193e80 con 0x5581c022f950 2024-03-26T19:01:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.312+0000 7fb3ca610740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5581c013c660 con 0x5581c0234ac0 2024-03-26T19:01:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.312+0000 7fb3ca610740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5581c018fe30 con 0x5581c022ef60 2024-03-26T19:01:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3c5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c022f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3c5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c0234450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3c5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c022f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51124/0 (socket says 172.21.15.97:51124) 2024-03-26T19:01:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3c5a00640 1 -- 172.21.15.97:0/2696314271 learned_addr learned my addr 172.21.15.97:0/2696314271 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/2696314271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 0x5581c0236f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3c5a00640 1 -- 172.21.15.97:0/2696314271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 msgr2=0x5581c0236f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/2696314271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 0x5581c0236f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3c5a00640 1 -- 172.21.15.97:0/2696314271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 msgr2=0x5581c0234450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/2696314271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c0234450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3c5a00640 1 -- 172.21.15.97:0/2696314271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581bfea8210 con 0x5581c022ef60 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.313+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/2696314271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 0x5581c0236f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.314+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/2696314271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c022f380 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009fd0 tx=0x7fb3b80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a31b40e21b3e62 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.314+0000 7fb3b2000640 1 -- 172.21.15.97:0/2696314271 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8004d20 con 0x5581c022ef60 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.314+0000 7fb3bd600640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3a80a2fd0 0x7fb3a80a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.314+0000 7fb3b2000640 1 -- 172.21.15.97:0/2696314271 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8004050 con 0x5581c022ef60 2024-03-26T19:01:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.314+0000 7fb3b2000640 1 -- 172.21.15.97:0/2696314271 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8005260 con 0x5581c022ef60 2024-03-26T19:01:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.314+0000 7fb3bd600640 1 -- 172.21.15.97:0/3423320511 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb3b40cf680 con 0x7fb3c0101740 2024-03-26T19:01:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.314+0000 7fb3c5000640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3a80a2fd0 0x7fb3a80a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.315+0000 7fb3ca610740 1 -- 172.21.15.97:0/2696314271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 msgr2=0x5581c022f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.315+0000 7fb3ca610740 1 --2- 172.21.15.97:0/2696314271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c022f380 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009fd0 tx=0x7fb3b80118d0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.315+0000 7fb3c5000640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3a80a2fd0 0x7fb3a80a5490 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40cd520 tx=0x7fb3b40d5550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.315+0000 7fb3ca610740 1 -- 172.21.15.97:0/2696314271 shutdown_connections 2024-03-26T19:01:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.315+0000 7fb3ca610740 1 --2- 172.21.15.97:0/2696314271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 0x5581c0236f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.315+0000 7fb3ca610740 1 --2- 172.21.15.97:0/2696314271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c0234450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.315+0000 7fb3ca610740 1 --2- 172.21.15.97:0/2696314271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c022f380 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.315+0000 7fb3ca610740 1 -- 172.21.15.97:0/2696314271 >> 172.21.15.97:0/2696314271 conn(0x5581c022a4d0 msgr2=0x5581c022a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.315+0000 7fb3ca610740 1 -- 172.21.15.97:0/2696314271 shutdown_connections 2024-03-26T19:01:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.316+0000 7fb3ca610740 1 -- 172.21.15.97:0/2696314271 wait complete. 2024-03-26T19:01:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.316+0000 7fb3ca610740 1 -- start start 2024-03-26T19:01:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.316+0000 7fb3ca610740 1 Processor -- start 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.317+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0238f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.317+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c02394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.317+0000 7fb3c5a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0238f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.318+0000 7fb3c5a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0238f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:36972/0 (socket says 172.21.15.97:36972) 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.318+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c0234ac0 0x5581c02404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.318+0000 7fb3ca610740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5581c0237a70 con 0x5581c022f950 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.318+0000 7fb3ca610740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5581c014fac0 con 0x5581c022ef60 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.318+0000 7fb3ca610740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5581c0237bf0 con 0x5581c0234ac0 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.318+0000 7fb3ca610740 10 client.0 _ll_register_callbacks cb 0x5581c014f490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.318+0000 7fb3c5a00640 1 -- 172.21.15.97:0/405410298 learned_addr learned my addr 172.21.15.97:0/405410298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.318+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c02394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.319+0000 7fb3bfe00640 1 -- 172.21.15.97:0/405410298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c0234ac0 msgr2=0x5581c02404e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:01:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.319+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c0234ac0 0x5581c02404e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.319+0000 7fb3bfe00640 1 -- 172.21.15.97:0/405410298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 msgr2=0x5581c0238f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.319+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0238f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.319+0000 7fb3bfe00640 1 -- 172.21.15.97:0/405410298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581bff74620 con 0x5581c022f950 2024-03-26T19:01:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.320+0000 7fb3a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T19:01:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.320+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c02394a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009850 tx=0x7fb3ac005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32fb85317207592 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.320+0000 7fb3a6a00640 1 -- 172.21.15.97:0/405410298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac046070 con 0x5581c022f950 2024-03-26T19:01:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.320+0000 7fb3ca610740 1 -- 172.21.15.97:0/405410298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b8004050 con 0x5581c022f950 2024-03-26T19:01:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.321+0000 7fb3a6a00640 1 -- 172.21.15.97:0/405410298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb3ac0055d0 con 0x5581c022f950 2024-03-26T19:01:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.321+0000 7fb3a6a00640 1 -- 172.21.15.97:0/405410298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb3ac050050 con 0x5581c022f950 2024-03-26T19:01:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.321+0000 7fb3ca610740 10 client.15285 fetch_fsmap learned FSMap version 27 2024-03-26T19:01:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.321+0000 7fb3ca610740 1 -- 172.21.15.97:0/405410298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581c0243910 con 0x5581c022f950 2024-03-26T19:01:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.322+0000 7fb3a6a00640 1 -- 172.21.15.97:0/405410298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8004050 con 0x5581c022f950 2024-03-26T19:01:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.322+0000 7fb3a6a00640 1 -- 172.21.15.97:0/405410298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb3ac053020 con 0x5581c022f950 2024-03-26T19:01:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.322+0000 7fb3ca610740 10 client.15285 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:01:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.322+0000 7fb3ca610740 10 client.15285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:01:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.322+0000 7fb3ca610740 20 client.15285 populate_metadata read hostname 'smithi097' 2024-03-26T19:01:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.322+0000 7fb3ca610740 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5581c0244f90 0x5581c0247380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.322+0000 7fb3ca610740 4 client.15285 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:01:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.322+0000 7fb3ca610740 1 -- 172.21.15.97:0/405410298 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581c01a13c0 con 0x5581c0244f90 2024-03-26T19:01:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.322+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5581c0244f90 0x5581c0247380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.323+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5581c0244f90 0x5581c0247380 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc5ce6913cf7041 server_cookie=13c33be21e144b40 in_seq=0 out_seq=0 2024-03-26T19:01:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.323+0000 7fb3a6a00640 10 client.15285 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.324+0000 7fb3a6a00640 1 -- 172.21.15.97:0/405410298 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5581c01a13c0 con 0x5581c0244f90 2024-03-26T19:01:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.324+0000 7fb3a6a00640 10 client.15285 handle_command_reply: tid=0 2024-03-26T19:01:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.324+0000 7fb3ca610740 1 client.15285 shutdown 2024-03-26T19:01:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.324+0000 7fb3ca610740 1 -- 172.21.15.97:0/405410298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 msgr2=0x5581c02394a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.324+0000 7fb3ca610740 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c02394a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009850 tx=0x7fb3ac005250 comp rx=0 tx=0).stop 2024-03-26T19:01:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.325+0000 7fb3ca610740 1 -- 172.21.15.97:0/405410298 shutdown_connections 2024-03-26T19:01:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.325+0000 7fb3ca610740 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5581c0244f90 0x5581c0247380 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.325+0000 7fb3ca610740 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c0234ac0 0x5581c02404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.325+0000 7fb3ca610740 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c02394a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.325+0000 7fb3ca610740 1 --2- 172.21.15.97:0/405410298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0238f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.325+0000 7fb3ca610740 1 -- 172.21.15.97:0/405410298 >> 172.21.15.97:0/405410298 conn(0x5581c022a4d0 msgr2=0x5581c02296f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.325+0000 7fb3a6a00640 0 client.15285 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.326+0000 7fb3ca610740 1 -- 172.21.15.97:0/405410298 shutdown_connections 2024-03-26T19:01:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.326+0000 7fb3ca610740 1 -- 172.21.15.97:0/405410298 wait complete. 2024-03-26T19:01:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.326+0000 7fb3ca610740 20 client.15285 trim_cache size 0 max 16384 2024-03-26T19:01:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.351+0000 7fb3ca610740 1 Processor -- start 2024-03-26T19:01:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.351+0000 7fb3ca610740 1 -- start start 2024-03-26T19:01:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.351+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0247010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.351+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022f950 0x5581c02404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.351+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c0234ac0 0x5581c0228e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.351+0000 7fb3ca610740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01b7b40 con 0x5581c0234ac0 2024-03-26T19:01:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.351+0000 7fb3ca610740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01b9680 con 0x5581c022ef60 2024-03-26T19:01:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.351+0000 7fb3ca610740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01b8210 con 0x5581c022f950 2024-03-26T19:01:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.351+0000 7fb3c5a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0247010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.352+0000 7fb3c5a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0247010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:36976/0 (socket says 172.21.15.97:36976) 2024-03-26T19:01:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.352+0000 7fb3c5a00640 1 -- 172.21.15.97:0/709236099 learned_addr learned my addr 172.21.15.97:0/709236099 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.352+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/709236099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c0234ac0 0x5581c0228e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.352+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/709236099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022f950 0x5581c02404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.352+0000 7fb3c5a00640 1 -- 172.21.15.97:0/709236099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022f950 msgr2=0x5581c02404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.352+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/709236099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022f950 0x5581c02404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.352+0000 7fb3c5a00640 1 -- 172.21.15.97:0/709236099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c0234ac0 msgr2=0x5581c0228e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.352+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/709236099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c0234ac0 0x5581c0228e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.352+0000 7fb3c5a00640 1 -- 172.21.15.97:0/709236099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c02476f0 con 0x5581c022ef60 2024-03-26T19:01:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.352+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/709236099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0247010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3b801b040 tx=0x7fb3b8009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79869d12034b621a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.353+0000 7fb3b1600640 1 -- 172.21.15.97:0/709236099 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8003bd0 con 0x5581c022ef60 2024-03-26T19:01:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.353+0000 7fb3b1600640 1 -- 172.21.15.97:0/709236099 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8005530 con 0x5581c022ef60 2024-03-26T19:01:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.353+0000 7fb3b1600640 1 -- 172.21.15.97:0/709236099 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800f8a0 con 0x5581c022ef60 2024-03-26T19:01:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.353+0000 7fb3ca610740 1 -- 172.21.15.97:0/709236099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 msgr2=0x5581c0247010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.353+0000 7fb3ca610740 1 --2- 172.21.15.97:0/709236099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0247010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3b801b040 tx=0x7fb3b8009500 comp rx=0 tx=0).stop 2024-03-26T19:01:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.354+0000 7fb3ca610740 1 -- 172.21.15.97:0/709236099 shutdown_connections 2024-03-26T19:01:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.354+0000 7fb3ca610740 1 --2- 172.21.15.97:0/709236099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c0234ac0 0x5581c0228e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.354+0000 7fb3ca610740 1 --2- 172.21.15.97:0/709236099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022f950 0x5581c02404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.354+0000 7fb3ca610740 1 --2- 172.21.15.97:0/709236099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c022ef60 0x5581c0247010 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.354+0000 7fb3ca610740 1 -- 172.21.15.97:0/709236099 >> 172.21.15.97:0/709236099 conn(0x5581c022a4d0 msgr2=0x5581c022dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.354+0000 7fb3ca610740 1 -- 172.21.15.97:0/709236099 shutdown_connections 2024-03-26T19:01:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.354+0000 7fb3ca610740 1 -- 172.21.15.97:0/709236099 wait complete. 2024-03-26T19:01:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.355+0000 7fb3ca610740 1 -- start start 2024-03-26T19:01:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.356+0000 7fb3ca610740 1 Processor -- start 2024-03-26T19:01:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.357+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c023df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.357+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c023e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.357+0000 7fb3ca610740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 0x5581c023f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.357+0000 7fb3ca610740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01b8fb0 con 0x5581c022f950 2024-03-26T19:01:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.357+0000 7fb3ca610740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01b88e0 con 0x5581c0234ac0 2024-03-26T19:01:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.357+0000 7fb3ca610740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01b5ac0 con 0x5581c022ef60 2024-03-26T19:01:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.357+0000 7fb3ca610740 10 client.0 _ll_register_callbacks cb 0x7fb3ac004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:01:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.358+0000 7fb3c5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c023df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.358+0000 7fb3c5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c023df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51150/0 (socket says 172.21.15.97:51150) 2024-03-26T19:01:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.358+0000 7fb3c5a00640 1 -- 172.21.15.97:0/3309680899 learned_addr learned my addr 172.21.15.97:0/3309680899 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.358+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c023e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.358+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 0x5581c023f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.358+0000 7fb3c5a00640 1 -- 172.21.15.97:0/3309680899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 msgr2=0x5581c023f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.358+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 0x5581c023f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.358+0000 7fb3c5a00640 1 -- 172.21.15.97:0/3309680899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 msgr2=0x5581c023e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.358+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c023e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.358+0000 7fb3c5a00640 1 -- 172.21.15.97:0/3309680899 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c023fa70 con 0x5581c022ef60 2024-03-26T19:01:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.359+0000 7fb3c5a00640 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c023df30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009fc0 tx=0x7fb3b800f4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ebf8b28a7984e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.359+0000 7fb3a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.359+0000 7fb3a6a00640 1 -- 172.21.15.97:0/3309680899 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8005760 con 0x5581c022ef60 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.359+0000 7fb3a6a00640 1 -- 172.21.15.97:0/3309680899 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb3b8005900 con 0x5581c022ef60 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.359+0000 7fb3ca610740 1 -- 172.21.15.97:0/3309680899 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b8005530 con 0x5581c022ef60 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.359+0000 7fb3a6a00640 1 -- 172.21.15.97:0/3309680899 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb3b80050c0 con 0x5581c022ef60 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.359+0000 7fb3ca610740 10 client.25426 fetch_fsmap learned FSMap version 27 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.360+0000 7fb3ca610740 1 -- 172.21.15.97:0/3309680899 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581c02ab5c0 con 0x5581c022ef60 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.360+0000 7fb3a6a00640 1 -- 172.21.15.97:0/3309680899 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8005530 con 0x5581c022ef60 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.360+0000 7fb3a6a00640 1 -- 172.21.15.97:0/3309680899 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b8032020 con 0x5581c022ef60 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.360+0000 7fb3ca610740 10 client.25426 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.360+0000 7fb3ca610740 10 client.25426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.360+0000 7fb3ca610740 20 client.25426 populate_metadata read hostname 'smithi097' 2024-03-26T19:01:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.360+0000 7fb3ca610740 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5581c02abd10 0x5581c02ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.360+0000 7fb3ca610740 4 client.25426 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:01:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.360+0000 7fb3ca610740 1 -- 172.21.15.97:0/3309680899 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581c019b710 con 0x5581c02abd10 2024-03-26T19:01:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.360+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5581c02abd10 0x5581c02ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.361+0000 7fb3bfe00640 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5581c02abd10 0x5581c02ae100 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=656f054721cc8bd0 server_cookie=7a9239b441cdd1a6 in_seq=0 out_seq=0 2024-03-26T19:01:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.361+0000 7fb3a6a00640 10 client.25426 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.382+0000 7fb3a6a00640 1 -- 172.21.15.97:0/3309680899 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581c019b710 con 0x5581c02abd10 2024-03-26T19:01:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3a6a00640 10 client.25426 handle_command_reply: tid=0 2024-03-26T19:01:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3ca610740 1 client.25426 shutdown 2024-03-26T19:01:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3ca610740 1 -- 172.21.15.97:0/3309680899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 msgr2=0x5581c023df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3ca610740 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c023df30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009fc0 tx=0x7fb3b800f4e0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3ca610740 1 -- 172.21.15.97:0/3309680899 shutdown_connections 2024-03-26T19:01:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3ca610740 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5581c02abd10 0x5581c02ae100 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3ca610740 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581c0234ac0 0x5581c023f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3ca610740 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581c022f950 0x5581c023e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3ca610740 1 --2- 172.21.15.97:0/3309680899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581c022ef60 0x5581c023df30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3ca610740 1 -- 172.21.15.97:0/3309680899 >> 172.21.15.97:0/3309680899 conn(0x5581c022a4d0 msgr2=0x5581c02303f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.383+0000 7fb3a6a00640 0 client.25426 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.384+0000 7fb3ca610740 1 -- 172.21.15.97:0/3309680899 shutdown_connections 2024-03-26T19:01:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.384+0000 7fb3ca610740 1 -- 172.21.15.97:0/3309680899 wait complete. 2024-03-26T19:01:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.384+0000 7fb3ca610740 20 client.25426 trim_cache size 0 max 16384 2024-03-26T19:01:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:01:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:01:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:01:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:01:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 -- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3a80a2fd0 msgr2=0x7fb3a80a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3a80a2fd0 0x7fb3a80a5490 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40cd520 tx=0x7fb3b40d5550 comp rx=0 tx=0).stop 2024-03-26T19:01:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 -- 172.21.15.97:0/3423320511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c0101740 msgr2=0x7fb3c006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c0101740 0x7fb3c006c360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400e9c0 tx=0x7fb3b400ee80 comp rx=0 tx=0).stop 2024-03-26T19:01:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 -- 172.21.15.97:0/3423320511 shutdown_connections 2024-03-26T19:01:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb3a80a2fd0 0x7fb3a80a5490 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3c01032f0 0x7fb3c01ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c0102930 0x7fb3c006c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 --2- 172.21.15.97:0/3423320511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c0101740 0x7fb3c006c360 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 -- 172.21.15.97:0/3423320511 >> 172.21.15.97:0/3423320511 conn(0x7fb3c00fccd0 msgr2=0x7fb3c010dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 -- 172.21.15.97:0/3423320511 shutdown_connections 2024-03-26T19:01:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:23.385+0000 7fb3c6400640 1 -- 172.21.15.97:0/3423320511 wait complete. 2024-03-26T19:01:24.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:24 smithi105 ceph-mon[36270]: pgmap v464: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 687 op/s 2024-03-26T19:01:24.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:24 smithi097 ceph-mon[31955]: pgmap v464: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 687 op/s 2024-03-26T19:01:24.626 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:24 smithi144 ceph-mon[36053]: pgmap v464: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 687 op/s 2024-03-26T19:01:25.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 120.0 s, 2361.9 tps, lat 42.254 ms stddev 49.003 2024-03-26T19:01:26.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:26 smithi105 ceph-mon[36270]: pgmap v465: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 765 op/s 2024-03-26T19:01:26.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:26 smithi097 ceph-mon[31955]: pgmap v465: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 765 op/s 2024-03-26T19:01:26.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:26 smithi144 ceph-mon[36053]: pgmap v465: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 765 op/s 2024-03-26T19:01:28.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:28 smithi105 ceph-mon[36270]: pgmap v466: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 660 op/s 2024-03-26T19:01:28.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:28 smithi097 ceph-mon[31955]: pgmap v466: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 660 op/s 2024-03-26T19:01:28.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:28 smithi144 ceph-mon[36053]: pgmap v466: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 660 op/s 2024-03-26T19:01:30.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 125.0 s, 3300.0 tps, lat 30.306 ms stddev 32.138 2024-03-26T19:01:30.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:30 smithi105 ceph-mon[36270]: pgmap v467: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 827 op/s 2024-03-26T19:01:30.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:30 smithi097 ceph-mon[31955]: pgmap v467: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 827 op/s 2024-03-26T19:01:30.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:30 smithi144 ceph-mon[36053]: pgmap v467: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 827 op/s 2024-03-26T19:01:32.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:32 smithi097 ceph-mon[31955]: pgmap v468: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 706 op/s 2024-03-26T19:01:32.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:32 smithi144 ceph-mon[36053]: pgmap v468: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 706 op/s 2024-03-26T19:01:32.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:32 smithi105 ceph-mon[36270]: pgmap v468: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 706 op/s 2024-03-26T19:01:34.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:34 smithi097 ceph-mon[31955]: pgmap v469: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.5 MiB/s wr, 792 op/s 2024-03-26T19:01:34.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:34 smithi144 ceph-mon[36053]: pgmap v469: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.5 MiB/s wr, 792 op/s 2024-03-26T19:01:34.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:34 smithi105 ceph-mon[36270]: pgmap v469: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.5 MiB/s wr, 792 op/s 2024-03-26T19:01:35.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 130.0 s, 2911.4 tps, lat 34.021 ms stddev 37.356 2024-03-26T19:01:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.570+0000 7fb51de00640 1 -- 172.21.15.97:0/4159686330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5180774c0 msgr2=0x7fb518077920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.570+0000 7fb51de00640 1 --2- 172.21.15.97:0/4159686330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5180774c0 0x7fb518077920 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb504009860 tx=0x7fb50402bb70 comp rx=0 tx=0).stop 2024-03-26T19:01:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.572+0000 7fb51de00640 1 -- 172.21.15.97:0/4159686330 shutdown_connections 2024-03-26T19:01:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.572+0000 7fb51de00640 1 --2- 172.21.15.97:0/4159686330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb518077e60 0x7fb518078700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.572+0000 7fb51de00640 1 --2- 172.21.15.97:0/4159686330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5180774c0 0x7fb518077920 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.572+0000 7fb51de00640 1 --2- 172.21.15.97:0/4159686330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb518076280 0x7fb5180766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.572+0000 7fb51de00640 1 -- 172.21.15.97:0/4159686330 >> 172.21.15.97:0/4159686330 conn(0x7fb5180ff0b0 msgr2=0x7fb5181014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.572+0000 7fb51de00640 1 -- 172.21.15.97:0/4159686330 shutdown_connections 2024-03-26T19:01:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.572+0000 7fb51de00640 1 -- 172.21.15.97:0/4159686330 wait complete. 2024-03-26T19:01:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.573+0000 7fb51de00640 1 Processor -- start 2024-03-26T19:01:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.573+0000 7fb51de00640 1 -- start start 2024-03-26T19:01:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb51de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb518076280 0x7fb51819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb51de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5180774c0 0x7fb51819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb51de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518077e60 0x7fb5181a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb51de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb518113be0 con 0x7fb518076280 2024-03-26T19:01:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb51de00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb518113a60 con 0x7fb5180774c0 2024-03-26T19:01:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb51de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb518113d60 con 0x7fb518077e60 2024-03-26T19:01:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb516000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5180774c0 0x7fb51819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb516000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5180774c0 0x7fb51819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:56152/0 (socket says 172.21.15.97:56152) 2024-03-26T19:01:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb516000640 1 -- 172.21.15.97:0/1165356577 learned_addr learned my addr 172.21.15.97:0/1165356577 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb517400640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518077e60 0x7fb5181a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.574+0000 7fb516a00640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb518076280 0x7fb51819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.575+0000 7fb516000640 1 -- 172.21.15.97:0/1165356577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518077e60 msgr2=0x7fb5181a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.575+0000 7fb516000640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518077e60 0x7fb5181a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.575+0000 7fb516000640 1 -- 172.21.15.97:0/1165356577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb518076280 msgr2=0x7fb51819e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.575+0000 7fb516000640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb518076280 0x7fb51819e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.575+0000 7fb516000640 1 -- 172.21.15.97:0/1165356577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5181a5db0 con 0x7fb5180774c0 2024-03-26T19:01:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.575+0000 7fb516000640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5180774c0 0x7fb51819e6d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb5040349e0 tx=0x7fb5040042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.576+0000 7fb51ca00640 1 -- 172.21.15.97:0/1165356577 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504041030 con 0x7fb5180774c0 2024-03-26T19:01:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.576+0000 7fb51ca00640 1 -- 172.21.15.97:0/1165356577 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5040347b0 con 0x7fb5180774c0 2024-03-26T19:01:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.576+0000 7fb51ca00640 1 -- 172.21.15.97:0/1165356577 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504039040 con 0x7fb5180774c0 2024-03-26T19:01:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.576+0000 7fb51de00640 1 -- 172.21.15.97:0/1165356577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5181a6040 con 0x7fb5180774c0 2024-03-26T19:01:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.577+0000 7fb51de00640 1 -- 172.21.15.97:0/1165356577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181a64e0 con 0x7fb5180774c0 2024-03-26T19:01:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.583+0000 7fb520bae740 1 Processor -- start 2024-03-26T19:01:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.583+0000 7fb520bae740 1 -- start start 2024-03-26T19:01:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.584+0000 7fb520bae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb520bae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dc790 0x55cf7e3e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb520bae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e3e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb51ca00640 1 -- 172.21.15.97:0/1165356577 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fb504044070 con 0x7fb5180774c0 2024-03-26T19:01:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb517400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb517400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3dc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32804/0 (socket says 172.21.15.97:32804) 2024-03-26T19:01:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb517400640 1 -- 172.21.15.97:0/2123486887 learned_addr learned my addr 172.21.15.97:0/2123486887 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb520bae740 1 -- 172.21.15.97:0/2123486887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cf7e340e80 con 0x55cf7e3dbda0 2024-03-26T19:01:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb520bae740 1 -- 172.21.15.97:0/2123486887 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cf7e2e9660 con 0x55cf7e3dc790 2024-03-26T19:01:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb520bae740 1 -- 172.21.15.97:0/2123486887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cf7e33ce30 con 0x55cf7e3e1900 2024-03-26T19:01:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb516000640 1 --2- 172.21.15.97:0/2123486887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e3e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb516a00640 1 --2- 172.21.15.97:0/2123486887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dc790 0x55cf7e3e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb517400640 1 -- 172.21.15.97:0/2123486887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 msgr2=0x55cf7e3e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb517400640 1 --2- 172.21.15.97:0/2123486887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e3e3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb517400640 1 -- 172.21.15.97:0/2123486887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dc790 msgr2=0x55cf7e3e1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.585+0000 7fb517400640 1 --2- 172.21.15.97:0/2123486887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dc790 0x55cf7e3e1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.586+0000 7fb517400640 1 -- 172.21.15.97:0/2123486887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7e055210 con 0x55cf7e3dbda0 2024-03-26T19:01:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.586+0000 7fb516000640 1 --2- 172.21.15.97:0/2123486887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e3e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:01:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.586+0000 7fb517400640 1 --2- 172.21.15.97:0/2123486887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3dc1c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb51000a430 tx=0x7fb510009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1abcab0c4868c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.587+0000 7fb503e00640 1 -- 172.21.15.97:0/2123486887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510013790 con 0x55cf7e3dbda0 2024-03-26T19:01:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.587+0000 7fb503e00640 1 -- 172.21.15.97:0/2123486887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb510019070 con 0x55cf7e3dbda0 2024-03-26T19:01:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.587+0000 7fb503e00640 1 -- 172.21.15.97:0/2123486887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510013790 con 0x55cf7e3dbda0 2024-03-26T19:01:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.588+0000 7fb520bae740 1 -- 172.21.15.97:0/2123486887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dbda0 msgr2=0x55cf7e3dc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.588+0000 7fb520bae740 1 --2- 172.21.15.97:0/2123486887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3dc1c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb51000a430 tx=0x7fb510009650 comp rx=0 tx=0).stop 2024-03-26T19:01:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.588+0000 7fb520bae740 1 -- 172.21.15.97:0/2123486887 shutdown_connections 2024-03-26T19:01:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.588+0000 7fb520bae740 1 --2- 172.21.15.97:0/2123486887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e3e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.588+0000 7fb520bae740 1 --2- 172.21.15.97:0/2123486887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dc790 0x55cf7e3e1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.588+0000 7fb520bae740 1 --2- 172.21.15.97:0/2123486887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3dc1c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.588+0000 7fb520bae740 1 -- 172.21.15.97:0/2123486887 >> 172.21.15.97:0/2123486887 conn(0x55cf7e3d7310 msgr2=0x55cf7e3d7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.588+0000 7fb520bae740 1 -- 172.21.15.97:0/2123486887 shutdown_connections 2024-03-26T19:01:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.588+0000 7fb520bae740 1 -- 172.21.15.97:0/2123486887 wait complete. 2024-03-26T19:01:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.589+0000 7fb520bae740 1 -- start start 2024-03-26T19:01:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.590+0000 7fb520bae740 1 Processor -- start 2024-03-26T19:01:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.590+0000 7fb520bae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.590+0000 7fb520bae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dc790 0x55cf7e3e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.590+0000 7fb520bae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3e1900 0x55cf7e3ed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.590+0000 7fb520bae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cf7e3e4900 con 0x55cf7e3e1900 2024-03-26T19:01:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.590+0000 7fb520bae740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cf7e2fcac0 con 0x55cf7e3dc790 2024-03-26T19:01:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.590+0000 7fb520bae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cf7e3e4a80 con 0x55cf7e3dbda0 2024-03-26T19:01:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.590+0000 7fb520bae740 10 client.0 _ll_register_callbacks cb 0x55cf7e2fc490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:01:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.591+0000 7fb51ca00640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4fc0a3040 0x7fb4fc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.591+0000 7fb517400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.591+0000 7fb517400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:32908/0 (socket says 172.21.15.97:32908) 2024-03-26T19:01:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.591+0000 7fb517400640 1 -- 172.21.15.97:0/3011679782 learned_addr learned my addr 172.21.15.97:0/3011679782 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.591+0000 7fb517400640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4fc0a3040 0x7fb4fc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.591+0000 7fb516a00640 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dc790 0x55cf7e3e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.592+0000 7fb517400640 1 -- 172.21.15.97:0/3011679782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dc790 msgr2=0x55cf7e3e63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.592+0000 7fb517400640 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dc790 0x55cf7e3e63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.592+0000 7fb517400640 1 -- 172.21.15.97:0/3011679782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3e1900 msgr2=0x55cf7e3ed320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.592+0000 7fb516000640 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3e1900 0x55cf7e3ed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.592+0000 7fb51ca00640 1 -- 172.21.15.97:0/1165356577 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb5040f0a80 con 0x7fb5180774c0 2024-03-26T19:01:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.593+0000 7fb517400640 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3e1900 0x55cf7e3ed320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.593+0000 7fb517400640 1 -- 172.21.15.97:0/3011679782 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7e121620 con 0x55cf7e3dbda0 2024-03-26T19:01:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.594+0000 7fb517400640 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e5e80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb510009900 tx=0x7fb5100138f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b567df65175a0976 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.594+0000 7fb500c00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:01:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.594+0000 7fb517400640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4fc0a3040 0x7fb4fc0a5500 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb510009dc0 tx=0x7fb51000c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.595+0000 7fb500c00640 1 -- 172.21.15.97:0/3011679782 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510032070 con 0x55cf7e3dbda0 2024-03-26T19:01:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.595+0000 7fb500c00640 1 -- 172.21.15.97:0/3011679782 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb5100206f0 con 0x55cf7e3dbda0 2024-03-26T19:01:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.595+0000 7fb520bae740 1 -- 172.21.15.97:0/3011679782 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb510019070 con 0x55cf7e3dbda0 2024-03-26T19:01:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.596+0000 7fb500c00640 1 -- 172.21.15.97:0/3011679782 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb51003e050 con 0x55cf7e3dbda0 2024-03-26T19:01:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.596+0000 7fb520bae740 10 client.25438 fetch_fsmap learned FSMap version 27 2024-03-26T19:01:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.596+0000 7fb520bae740 1 -- 172.21.15.97:0/3011679782 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf7e3f0690 con 0x55cf7e3dbda0 2024-03-26T19:01:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.596+0000 7fb500c00640 1 -- 172.21.15.97:0/3011679782 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb51002e430 con 0x55cf7e3dbda0 2024-03-26T19:01:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.597+0000 7fb500c00640 1 -- 172.21.15.97:0/3011679782 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb51002d070 con 0x55cf7e3dbda0 2024-03-26T19:01:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.598+0000 7fb520bae740 10 client.25438 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:01:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.598+0000 7fb520bae740 10 client.25438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:01:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.598+0000 7fb520bae740 20 client.25438 populate_metadata read hostname 'smithi097' 2024-03-26T19:01:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.598+0000 7fb520bae740 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55cf7e3f1aa0 0x55cf7e3f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.598+0000 7fb520bae740 4 client.25438 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:01:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.598+0000 7fb520bae740 1 -- 172.21.15.97:0/3011679782 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb510019070 con 0x55cf7e3f1aa0 2024-03-26T19:01:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.598+0000 7fb516a00640 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55cf7e3f1aa0 0x55cf7e3f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.600+0000 7fb516a00640 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55cf7e3f1aa0 0x55cf7e3f3e90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d395e9a6e002071 server_cookie=23286dfad9f7a145 in_seq=0 out_seq=0 2024-03-26T19:01:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.600+0000 7fb500c00640 10 client.25438 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.601+0000 7fb500c00640 1 -- 172.21.15.97:0/3011679782 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fb510019070 con 0x55cf7e3f1aa0 2024-03-26T19:01:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.601+0000 7fb500c00640 10 client.25438 handle_command_reply: tid=0 2024-03-26T19:01:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.601+0000 7fb520bae740 1 client.25438 shutdown 2024-03-26T19:01:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.603+0000 7fb520bae740 1 -- 172.21.15.97:0/3011679782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 msgr2=0x55cf7e3e5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.603+0000 7fb520bae740 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e5e80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb510009900 tx=0x7fb5100138f0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.603+0000 7fb520bae740 1 -- 172.21.15.97:0/3011679782 shutdown_connections 2024-03-26T19:01:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.603+0000 7fb520bae740 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55cf7e3f1aa0 0x55cf7e3f3e90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.603+0000 7fb520bae740 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3e1900 0x55cf7e3ed320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.603+0000 7fb520bae740 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dc790 0x55cf7e3e63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.603+0000 7fb520bae740 1 --2- 172.21.15.97:0/3011679782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e5e80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.603+0000 7fb500c00640 0 client.25438 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.603+0000 7fb520bae740 1 -- 172.21.15.97:0/3011679782 >> 172.21.15.97:0/3011679782 conn(0x55cf7e3d7310 msgr2=0x55cf7e3d6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.604+0000 7fb520bae740 1 -- 172.21.15.97:0/3011679782 shutdown_connections 2024-03-26T19:01:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.604+0000 7fb520bae740 1 -- 172.21.15.97:0/3011679782 wait complete. 2024-03-26T19:01:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.604+0000 7fb520bae740 20 client.25438 trim_cache size 0 max 16384 2024-03-26T19:01:36.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:36 smithi097 ceph-mon[31955]: pgmap v470: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 MiB/s wr, 781 op/s 2024-03-26T19:01:36.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:36 smithi144 ceph-mon[36053]: pgmap v470: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 MiB/s wr, 781 op/s 2024-03-26T19:01:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.633+0000 7fb520bae740 1 Processor -- start 2024-03-26T19:01:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.634+0000 7fb520bae740 1 -- start start 2024-03-26T19:01:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.634+0000 7fb520bae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.634+0000 7fb520bae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dc790 0x55cf7e3ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.634+0000 7fb520bae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3e1900 0x55cf7e3dac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.634+0000 7fb520bae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181af360 con 0x55cf7e3dc790 2024-03-26T19:01:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.634+0000 7fb520bae740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181b0ea0 con 0x55cf7e3e1900 2024-03-26T19:01:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.634+0000 7fb520bae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181afa30 con 0x55cf7e3dbda0 2024-03-26T19:01:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.634+0000 7fb516a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.634+0000 7fb516a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:32924/0 (socket says 172.21.15.97:32924) 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.635+0000 7fb516a00640 1 -- 172.21.15.97:0/4167367793 learned_addr learned my addr 172.21.15.97:0/4167367793 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.635+0000 7fb517400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3e1900 0x55cf7e3dac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.635+0000 7fb516000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dc790 0x55cf7e3ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.635+0000 7fb517400640 1 -- 172.21.15.97:0/4167367793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 msgr2=0x55cf7e3e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.635+0000 7fb517400640 1 --2- 172.21.15.97:0/4167367793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.635+0000 7fb517400640 1 -- 172.21.15.97:0/4167367793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dc790 msgr2=0x55cf7e3ed2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.635+0000 7fb517400640 1 --2- 172.21.15.97:0/4167367793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dc790 0x55cf7e3ed2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.635+0000 7fb517400640 1 -- 172.21.15.97:0/4167367793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7e3e16b0 con 0x55cf7e3e1900 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.635+0000 7fb516000640 1 --2- 172.21.15.97:0/4167367793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dc790 0x55cf7e3ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:01:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.636+0000 7fb517400640 1 --2- 172.21.15.97:0/4167367793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3e1900 0x55cf7e3dac80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb5100098d0 tx=0x7fb510009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=878edb74499a221f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.636+0000 7fb503400640 1 -- 172.21.15.97:0/4167367793 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510032040 con 0x55cf7e3e1900 2024-03-26T19:01:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.636+0000 7fb503400640 1 -- 172.21.15.97:0/4167367793 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb510019070 con 0x55cf7e3e1900 2024-03-26T19:01:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.636+0000 7fb503400640 1 -- 172.21.15.97:0/4167367793 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510020a70 con 0x55cf7e3e1900 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.637+0000 7fb520bae740 1 -- 172.21.15.97:0/4167367793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3e1900 msgr2=0x55cf7e3dac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.637+0000 7fb520bae740 1 --2- 172.21.15.97:0/4167367793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3e1900 0x55cf7e3dac80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb5100098d0 tx=0x7fb510009650 comp rx=0 tx=0).stop 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.637+0000 7fb520bae740 1 -- 172.21.15.97:0/4167367793 shutdown_connections 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.637+0000 7fb520bae740 1 --2- 172.21.15.97:0/4167367793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3e1900 0x55cf7e3dac80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.637+0000 7fb520bae740 1 --2- 172.21.15.97:0/4167367793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dc790 0x55cf7e3ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.637+0000 7fb520bae740 1 --2- 172.21.15.97:0/4167367793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.637+0000 7fb520bae740 1 -- 172.21.15.97:0/4167367793 >> 172.21.15.97:0/4167367793 conn(0x55cf7e3d7310 msgr2=0x55cf7e3da570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.637+0000 7fb520bae740 1 -- 172.21.15.97:0/4167367793 shutdown_connections 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.637+0000 7fb520bae740 1 -- 172.21.15.97:0/4167367793 wait complete. 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.638+0000 7fb520bae740 1 -- start start 2024-03-26T19:01:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.638+0000 7fb520bae740 1 Processor -- start 2024-03-26T19:01:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.639+0000 7fb520bae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3ea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.639+0000 7fb520bae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dc790 0x55cf7e3ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.639+0000 7fb520bae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e455cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.639+0000 7fb520bae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181b07d0 con 0x55cf7e3dc790 2024-03-26T19:01:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.639+0000 7fb520bae740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181b0100 con 0x55cf7e3dbda0 2024-03-26T19:01:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.639+0000 7fb520bae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181ad340 con 0x55cf7e3e1900 2024-03-26T19:01:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.639+0000 7fb520bae740 10 client.0 _ll_register_callbacks cb 0x7fb510015db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:01:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.640+0000 7fb517400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e455cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.640+0000 7fb517400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e455cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:32936/0 (socket says 172.21.15.97:32936) 2024-03-26T19:01:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.640+0000 7fb517400640 1 -- 172.21.15.97:0/4010867385 learned_addr learned my addr 172.21.15.97:0/4010867385 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.640+0000 7fb516a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.640+0000 7fb517400640 1 -- 172.21.15.97:0/4010867385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dbda0 msgr2=0x55cf7e3ea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.640+0000 7fb517400640 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3ea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.640+0000 7fb517400640 1 -- 172.21.15.97:0/4010867385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dc790 msgr2=0x55cf7e3ea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.641+0000 7fb517400640 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dc790 0x55cf7e3ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.641+0000 7fb517400640 1 -- 172.21.15.97:0/4010867385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7e456200 con 0x55cf7e3e1900 2024-03-26T19:01:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.641+0000 7fb517400640 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e455cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb510007000 tx=0x7fb510009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25286491e9242ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.641+0000 7fb500c00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:01:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.641+0000 7fb500c00640 1 -- 172.21.15.97:0/4010867385 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510046020 con 0x55cf7e3e1900 2024-03-26T19:01:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.642+0000 7fb500c00640 1 -- 172.21.15.97:0/4010867385 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb510014c80 con 0x55cf7e3e1900 2024-03-26T19:01:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.642+0000 7fb520bae740 1 -- 172.21.15.97:0/4010867385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb510019070 con 0x55cf7e3e1900 2024-03-26T19:01:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.642+0000 7fb500c00640 1 -- 172.21.15.97:0/4010867385 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb510032040 con 0x55cf7e3e1900 2024-03-26T19:01:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.642+0000 7fb520bae740 10 client.25444 fetch_fsmap learned FSMap version 27 2024-03-26T19:01:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.642+0000 7fb520bae740 1 -- 172.21.15.97:0/4010867385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf7e457460 con 0x55cf7e3e1900 2024-03-26T19:01:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.643+0000 7fb500c00640 1 -- 172.21.15.97:0/4010867385 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb510019070 con 0x55cf7e3e1900 2024-03-26T19:01:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.644+0000 7fb500c00640 1 -- 172.21.15.97:0/4010867385 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb510053020 con 0x55cf7e3e1900 2024-03-26T19:01:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.644+0000 7fb520bae740 10 client.25444 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:01:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.644+0000 7fb520bae740 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:01:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.644+0000 7fb520bae740 20 client.25444 populate_metadata read hostname 'smithi097' 2024-03-26T19:01:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.644+0000 7fb520bae740 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55cf7e457b40 0x55cf7e459f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.644+0000 7fb520bae740 4 client.25444 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:01:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.644+0000 7fb520bae740 1 -- 172.21.15.97:0/4010867385 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf7e34e3c0 con 0x55cf7e457b40 2024-03-26T19:01:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.644+0000 7fb516a00640 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55cf7e457b40 0x55cf7e459f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.645+0000 7fb516a00640 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55cf7e457b40 0x55cf7e459f60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2afc02b9e23ea728 server_cookie=c08379ed6d68db77 in_seq=0 out_seq=0 2024-03-26T19:01:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.645+0000 7fb500c00640 10 client.25444 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.663+0000 7fb500c00640 1 -- 172.21.15.97:0/4010867385 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf7e34e3c0 con 0x55cf7e457b40 2024-03-26T19:01:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.663+0000 7fb500c00640 10 client.25444 handle_command_reply: tid=0 2024-03-26T19:01:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.663+0000 7fb520bae740 1 client.25444 shutdown 2024-03-26T19:01:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.664+0000 7fb520bae740 1 -- 172.21.15.97:0/4010867385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 msgr2=0x55cf7e455cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.664+0000 7fb520bae740 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e455cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb510007000 tx=0x7fb510009d20 comp rx=0 tx=0).stop 2024-03-26T19:01:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.664+0000 7fb520bae740 1 -- 172.21.15.97:0/4010867385 shutdown_connections 2024-03-26T19:01:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.664+0000 7fb520bae740 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55cf7e457b40 0x55cf7e459f60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.664+0000 7fb520bae740 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf7e3e1900 0x55cf7e455cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.664+0000 7fb520bae740 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf7e3dc790 0x55cf7e3ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.664+0000 7fb520bae740 1 --2- 172.21.15.97:0/4010867385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cf7e3dbda0 0x55cf7e3ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.664+0000 7fb520bae740 1 -- 172.21.15.97:0/4010867385 >> 172.21.15.97:0/4010867385 conn(0x55cf7e3d7310 msgr2=0x55cf7e3e6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.664+0000 7fb500c00640 0 client.25444 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.665+0000 7fb520bae740 1 -- 172.21.15.97:0/4010867385 shutdown_connections 2024-03-26T19:01:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.665+0000 7fb520bae740 1 -- 172.21.15.97:0/4010867385 wait complete. 2024-03-26T19:01:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.665+0000 7fb520bae740 20 client.25444 trim_cache size 0 max 16384 2024-03-26T19:01:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:01:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:01:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:01:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:01:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.666+0000 7fb51de00640 1 -- 172.21.15.97:0/1165356577 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4fc0a3040 msgr2=0x7fb4fc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.666+0000 7fb51de00640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4fc0a3040 0x7fb4fc0a5500 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb510009dc0 tx=0x7fb51000c000 comp rx=0 tx=0).stop 2024-03-26T19:01:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.666+0000 7fb51de00640 1 -- 172.21.15.97:0/1165356577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5180774c0 msgr2=0x7fb51819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.666+0000 7fb51de00640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5180774c0 0x7fb51819e6d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb5040349e0 tx=0x7fb5040042f0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.666+0000 7fb51de00640 1 -- 172.21.15.97:0/1165356577 shutdown_connections 2024-03-26T19:01:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.666+0000 7fb51de00640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb4fc0a3040 0x7fb4fc0a5500 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.666+0000 7fb51de00640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518077e60 0x7fb5181a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.666+0000 7fb51de00640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5180774c0 0x7fb51819e6d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.666+0000 7fb51de00640 1 --2- 172.21.15.97:0/1165356577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb518076280 0x7fb51819e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.666+0000 7fb51de00640 1 -- 172.21.15.97:0/1165356577 >> 172.21.15.97:0/1165356577 conn(0x7fb5180ff0b0 msgr2=0x7fb518100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.667+0000 7fb51de00640 1 -- 172.21.15.97:0/1165356577 shutdown_connections 2024-03-26T19:01:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:36.667+0000 7fb51de00640 1 -- 172.21.15.97:0/1165356577 wait complete. 2024-03-26T19:01:36.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:36 smithi105 ceph-mon[36270]: pgmap v470: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 MiB/s wr, 781 op/s 2024-03-26T19:01:38.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:38 smithi097 ceph-mon[31955]: pgmap v471: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 MiB/s wr, 652 op/s 2024-03-26T19:01:38.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:38 smithi144 ceph-mon[36053]: pgmap v471: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 MiB/s wr, 652 op/s 2024-03-26T19:01:38.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:38 smithi105 ceph-mon[36270]: pgmap v471: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 MiB/s wr, 652 op/s 2024-03-26T19:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 135.0 s, 2442.0 tps, lat 41.296 ms stddev 47.497 2024-03-26T19:01:40.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:40 smithi097 ceph-mon[31955]: pgmap v472: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 811 op/s 2024-03-26T19:01:40.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:40 smithi144 ceph-mon[36053]: pgmap v472: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 811 op/s 2024-03-26T19:01:40.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:40 smithi105 ceph-mon[36270]: pgmap v472: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 811 op/s 2024-03-26T19:01:42.603 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:42 smithi097 ceph-mon[31955]: pgmap v473: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 MiB/s wr, 750 op/s 2024-03-26T19:01:42.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:42 smithi144 ceph-mon[36053]: pgmap v473: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 MiB/s wr, 750 op/s 2024-03-26T19:01:42.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:42 smithi105 ceph-mon[36270]: pgmap v473: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 MiB/s wr, 750 op/s 2024-03-26T19:01:44.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:44 smithi097 ceph-mon[31955]: pgmap v474: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 721 op/s 2024-03-26T19:01:44.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:44 smithi144 ceph-mon[36053]: pgmap v474: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 721 op/s 2024-03-26T19:01:44.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:44 smithi105 ceph-mon[36270]: pgmap v474: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 721 op/s 2024-03-26T19:01:45.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 140.0 s, 2359.6 tps, lat 42.196 ms stddev 45.428 2024-03-26T19:01:45.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:45 smithi097 ceph-mon[31955]: pgmap v475: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 716 op/s 2024-03-26T19:01:45.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:45 smithi144 ceph-mon[36053]: pgmap v475: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 716 op/s 2024-03-26T19:01:45.711 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:45 smithi105 ceph-mon[36270]: pgmap v475: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 716 op/s 2024-03-26T19:01:48.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:48 smithi097 ceph-mon[31955]: pgmap v476: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 634 op/s 2024-03-26T19:01:48.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:48 smithi144 ceph-mon[36053]: pgmap v476: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 634 op/s 2024-03-26T19:01:48.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:48 smithi105 ceph-mon[36270]: pgmap v476: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 634 op/s 2024-03-26T19:01:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.858+0000 7f54a0a00640 1 -- 172.21.15.97:0/379888821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c102390 msgr2=0x7f549c108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.858+0000 7f54a0a00640 1 --2- 172.21.15.97:0/379888821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c102390 0x7f549c108c20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5494009ca0 tx=0x7f549402c0f0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.858+0000 7f54a0a00640 1 -- 172.21.15.97:0/379888821 shutdown_connections 2024-03-26T19:01:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.858+0000 7f54a0a00640 1 --2- 172.21.15.97:0/379888821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c102390 0x7f549c108c20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.858+0000 7f54a0a00640 1 --2- 172.21.15.97:0/379888821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549c1019d0 0x7f549c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.858+0000 7f54a0a00640 1 --2- 172.21.15.97:0/379888821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549c1007e0 0x7f549c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.858+0000 7f54a0a00640 1 -- 172.21.15.97:0/379888821 >> 172.21.15.97:0/379888821 conn(0x7f549c0fbd50 msgr2=0x7f549c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.858+0000 7f54a0a00640 1 -- 172.21.15.97:0/379888821 shutdown_connections 2024-03-26T19:01:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.859+0000 7f54a0a00640 1 -- 172.21.15.97:0/379888821 wait complete. 2024-03-26T19:01:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.860+0000 7f54a0a00640 1 Processor -- start 2024-03-26T19:01:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.861+0000 7f54a0a00640 1 -- start start 2024-03-26T19:01:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f54a0a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549c1007e0 0x7f549c1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f54a0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549c1019d0 0x7f549c1a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f54a0a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c102390 0x7f549c1a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f54a0a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f549c10c460 con 0x7f549c1019d0 2024-03-26T19:01:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f54a0a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f549c10c2e0 con 0x7f549c102390 2024-03-26T19:01:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f54a0a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f549c10c5e0 con 0x7f549c1007e0 2024-03-26T19:01:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f5499e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549c1007e0 0x7f549c1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f5499e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549c1007e0 0x7f549c1a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50282/0 (socket says 172.21.15.97:50282) 2024-03-26T19:01:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f5499e00640 1 -- 172.21.15.97:0/3079369331 learned_addr learned my addr 172.21.15.97:0/3079369331 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f549a800640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c102390 0x7f549c1a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.863+0000 7f5499400640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549c1019d0 0x7f549c1a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.864+0000 7f549a800640 1 -- 172.21.15.97:0/3079369331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549c1007e0 msgr2=0x7f549c1a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.864+0000 7f549a800640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549c1007e0 0x7f549c1a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.864+0000 7f549a800640 1 -- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549c1019d0 msgr2=0x7f549c1a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.864+0000 7f549a800640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549c1019d0 0x7f549c1a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.864+0000 7f549a800640 1 -- 172.21.15.97:0/3079369331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c1a5dd0 con 0x7f549c102390 2024-03-26T19:01:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.864+0000 7f5499400640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549c1019d0 0x7f549c1a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:01:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.864+0000 7f549a800640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c102390 0x7f549c1a54c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5494037040 tx=0x7f5494032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.865+0000 7f5492a00640 1 -- 172.21.15.97:0/3079369331 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494041020 con 0x7f549c102390 2024-03-26T19:01:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.865+0000 7f5492a00640 1 -- 172.21.15.97:0/3079369331 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494034cd0 con 0x7f549c102390 2024-03-26T19:01:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.865+0000 7f5492a00640 1 -- 172.21.15.97:0/3079369331 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494039040 con 0x7f549c102390 2024-03-26T19:01:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.865+0000 7f54a0a00640 1 -- 172.21.15.97:0/3079369331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c1a6000 con 0x7f549c102390 2024-03-26T19:01:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.865+0000 7f54a0a00640 1 -- 172.21.15.97:0/3079369331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c1a6470 con 0x7f549c102390 2024-03-26T19:01:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.870+0000 7f54a3771740 1 Processor -- start 2024-03-26T19:01:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.872+0000 7f5492a00640 1 -- 172.21.15.97:0/3079369331 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f5494044020 con 0x7f549c102390 2024-03-26T19:01:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.872+0000 7f54a3771740 1 -- start start 2024-03-26T19:01:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.872+0000 7f54a3771740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.872+0000 7f54a3771740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.872+0000 7f54a3771740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b9900 0x5631755bbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.872+0000 7f54a3771740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563175518e80 con 0x5631755b9900 2024-03-26T19:01:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.872+0000 7f54a3771740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5631754c1660 con 0x5631755b4790 2024-03-26T19:01:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.872+0000 7f54a3771740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563175514e30 con 0x5631755b3da0 2024-03-26T19:01:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.873+0000 7f5499400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.873+0000 7f5499400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50290/0 (socket says 172.21.15.97:50290) 2024-03-26T19:01:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.873+0000 7f5499400640 1 -- 172.21.15.97:0/474402986 learned_addr learned my addr 172.21.15.97:0/474402986 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.873+0000 7f5499400640 1 --2- 172.21.15.97:0/474402986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b9900 0x5631755bbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.873+0000 7f5499e00640 1 --2- 172.21.15.97:0/474402986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.873+0000 7f5499400640 1 -- 172.21.15.97:0/474402986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 msgr2=0x5631755b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.873+0000 7f5499400640 1 --2- 172.21.15.97:0/474402986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.873+0000 7f5499400640 1 -- 172.21.15.97:0/474402986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b9900 msgr2=0x5631755bbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.874+0000 7f5499400640 1 --2- 172.21.15.97:0/474402986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b9900 0x5631755bbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.874+0000 7f5499400640 1 -- 172.21.15.97:0/474402986 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56317522d210 con 0x5631755b3da0 2024-03-26T19:01:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.874+0000 7f5499400640 1 --2- 172.21.15.97:0/474402986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755b41c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f548800bf60 tx=0x7f548800b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4357fbf77d8b1e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.874+0000 7f5487000640 1 -- 172.21.15.97:0/474402986 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54880140f0 con 0x5631755b3da0 2024-03-26T19:01:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.875+0000 7f5492a00640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54800a3090 0x7f54800a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.876+0000 7f5492a00640 1 -- 172.21.15.97:0/3079369331 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f54940f1310 con 0x7f549c102390 2024-03-26T19:01:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.876+0000 7f5487000640 1 -- 172.21.15.97:0/474402986 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5488018070 con 0x5631755b3da0 2024-03-26T19:01:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.876+0000 7f5487000640 1 -- 172.21.15.97:0/474402986 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54880047a0 con 0x5631755b3da0 2024-03-26T19:01:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.877+0000 7f5499e00640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54800a3090 0x7f54800a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.878+0000 7f54a3771740 1 -- 172.21.15.97:0/474402986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 msgr2=0x5631755b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.878+0000 7f54a3771740 1 --2- 172.21.15.97:0/474402986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755b41c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f548800bf60 tx=0x7f548800b540 comp rx=0 tx=0).stop 2024-03-26T19:01:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.878+0000 7f54a3771740 1 -- 172.21.15.97:0/474402986 shutdown_connections 2024-03-26T19:01:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.878+0000 7f54a3771740 1 --2- 172.21.15.97:0/474402986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b9900 0x5631755bbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.878+0000 7f54a3771740 1 --2- 172.21.15.97:0/474402986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.878+0000 7f54a3771740 1 --2- 172.21.15.97:0/474402986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755b41c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.878+0000 7f54a3771740 1 -- 172.21.15.97:0/474402986 >> 172.21.15.97:0/474402986 conn(0x5631755af310 msgr2=0x5631755af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.878+0000 7f54a3771740 1 -- 172.21.15.97:0/474402986 shutdown_connections 2024-03-26T19:01:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.879+0000 7f5499e00640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54800a3090 0x7f54800a5550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5631755b4b90 tx=0x7f548c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.879+0000 7f54a3771740 1 -- 172.21.15.97:0/474402986 wait complete. 2024-03-26T19:01:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.879+0000 7f54a3771740 1 -- start start 2024-03-26T19:01:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.880+0000 7f54a3771740 1 Processor -- start 2024-03-26T19:01:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.881+0000 7f54a3771740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b3da0 0x5631755bde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.881+0000 7f54a3771740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b4790 0x5631755be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.881+0000 7f54a3771740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b9900 0x5631755c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.881+0000 7f54a3771740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5631755bc8b0 con 0x5631755b3da0 2024-03-26T19:01:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f54a3771740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5631754d4ac0 con 0x5631755b9900 2024-03-26T19:01:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f54a3771740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5631755bca30 con 0x5631755b4790 2024-03-26T19:01:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f54a3771740 10 client.0 _ll_register_callbacks cb 0x5631754d4490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:01:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f549a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b9900 0x5631755c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f549a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b9900 0x5631755c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34144/0 (socket says 172.21.15.97:34144) 2024-03-26T19:01:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f549a800640 1 -- 172.21.15.97:0/663119820 learned_addr learned my addr 172.21.15.97:0/663119820 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f549a800640 1 -- 172.21.15.97:0/663119820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b4790 msgr2=0x5631755be380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f549a800640 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b4790 0x5631755be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f549a800640 1 -- 172.21.15.97:0/663119820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b3da0 msgr2=0x5631755bde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f549a800640 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b3da0 0x5631755bde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.882+0000 7f549a800640 1 -- 172.21.15.97:0/663119820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631752f9620 con 0x5631755b9900 2024-03-26T19:01:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.883+0000 7f549a800640 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b9900 0x5631755c5350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f54940ba440 tx=0x7f54940bd490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbcdeea12aa7739c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.883+0000 7f547b400640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:01:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.883+0000 7f547b400640 1 -- 172.21.15.97:0/663119820 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940c1bc0 con 0x5631755b9900 2024-03-26T19:01:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.883+0000 7f547b400640 1 -- 172.21.15.97:0/663119820 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f54940c5ff0 con 0x5631755b9900 2024-03-26T19:01:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.883+0000 7f54a3771740 1 -- 172.21.15.97:0/663119820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5488018070 con 0x5631755b9900 2024-03-26T19:01:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f547b400640 1 -- 172.21.15.97:0/663119820 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54940c6270 con 0x5631755b9900 2024-03-26T19:01:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f54a3771740 10 client.25493 fetch_fsmap learned FSMap version 27 2024-03-26T19:01:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f54a3771740 1 -- 172.21.15.97:0/663119820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631755c86c0 con 0x5631755b9900 2024-03-26T19:01:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f547b400640 1 -- 172.21.15.97:0/663119820 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5488018070 con 0x5631755b9900 2024-03-26T19:01:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f547b400640 1 -- 172.21.15.97:0/663119820 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f549403c030 con 0x5631755b9900 2024-03-26T19:01:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f54a3771740 10 client.25493 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:01:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f54a3771740 10 client.25493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:01:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f54a3771740 20 client.25493 populate_metadata read hostname 'smithi097' 2024-03-26T19:01:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f54a3771740 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5631755c9e40 0x5631755cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f54a3771740 4 client.25493 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:01:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.884+0000 7f54a3771740 1 -- 172.21.15.97:0/663119820 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631755263c0 con 0x5631755c9e40 2024-03-26T19:01:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.885+0000 7f5499e00640 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5631755c9e40 0x5631755cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.886+0000 7f5499e00640 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5631755c9e40 0x5631755cc2e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e104c7d7fda90563 server_cookie=c2d5d181bc072c46 in_seq=0 out_seq=0 2024-03-26T19:01:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.886+0000 7f547b400640 10 client.25493 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.888+0000 7f547b400640 1 -- 172.21.15.97:0/663119820 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5631755263c0 con 0x5631755c9e40 2024-03-26T19:01:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.888+0000 7f547b400640 10 client.25493 handle_command_reply: tid=0 2024-03-26T19:01:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.888+0000 7f54a3771740 1 client.25493 shutdown 2024-03-26T19:01:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.889+0000 7f54a3771740 1 -- 172.21.15.97:0/663119820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b9900 msgr2=0x5631755c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.889+0000 7f54a3771740 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b9900 0x5631755c5350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f54940ba440 tx=0x7f54940bd490 comp rx=0 tx=0).stop 2024-03-26T19:01:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.889+0000 7f54a3771740 1 -- 172.21.15.97:0/663119820 shutdown_connections 2024-03-26T19:01:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.889+0000 7f54a3771740 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5631755c9e40 0x5631755cc2e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.889+0000 7f54a3771740 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b9900 0x5631755c5350 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.889+0000 7f54a3771740 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b4790 0x5631755be380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.889+0000 7f54a3771740 1 --2- 172.21.15.97:0/663119820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b3da0 0x5631755bde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.889+0000 7f547b400640 0 client.25493 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.889+0000 7f54a3771740 1 -- 172.21.15.97:0/663119820 >> 172.21.15.97:0/663119820 conn(0x5631755af310 msgr2=0x5631755ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.890+0000 7f54a3771740 1 -- 172.21.15.97:0/663119820 shutdown_connections 2024-03-26T19:01:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.890+0000 7f54a3771740 1 -- 172.21.15.97:0/663119820 wait complete. 2024-03-26T19:01:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.890+0000 7f54a3771740 20 client.25493 trim_cache size 0 max 16384 2024-03-26T19:01:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.914+0000 7f54a3771740 1 Processor -- start 2024-03-26T19:01:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.914+0000 7f54a3771740 1 -- start start 2024-03-26T19:01:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f54a3771740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b3da0 0x5631755b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f54a3771740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f54a3771740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b9900 0x5631755d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f54a3771740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1af1b0 con 0x5631755b3da0 2024-03-26T19:01:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f54a3771740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b0cf0 con 0x5631755b4790 2024-03-26T19:01:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f54a3771740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1af880 con 0x5631755b9900 2024-03-26T19:01:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f549a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b9900 0x5631755d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f549a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b9900 0x5631755d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50304/0 (socket says 172.21.15.97:50304) 2024-03-26T19:01:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f549a800640 1 -- 172.21.15.97:0/3675161665 learned_addr learned my addr 172.21.15.97:0/3675161665 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f5499400640 1 --2- 172.21.15.97:0/3675161665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f549a800640 1 -- 172.21.15.97:0/3675161665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 msgr2=0x5631755c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f549a800640 1 --2- 172.21.15.97:0/3675161665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f549a800640 1 -- 172.21.15.97:0/3675161665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b3da0 msgr2=0x5631755b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f549a800640 1 --2- 172.21.15.97:0/3675161665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b3da0 0x5631755b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.915+0000 7f549a800640 1 -- 172.21.15.97:0/3675161665 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631755cc960 con 0x5631755b9900 2024-03-26T19:01:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.916+0000 7f549a800640 1 --2- 172.21.15.97:0/3675161665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b9900 0x5631755d0be0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f54940324c0 tx=0x7f54940bbf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a92930b2f9d66773 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.916+0000 7f5486600640 1 -- 172.21.15.97:0/3675161665 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940c6480 con 0x5631755b9900 2024-03-26T19:01:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.916+0000 7f5486600640 1 -- 172.21.15.97:0/3675161665 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54940da020 con 0x5631755b9900 2024-03-26T19:01:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.916+0000 7f5486600640 1 -- 172.21.15.97:0/3675161665 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940c6250 con 0x5631755b9900 2024-03-26T19:01:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.917+0000 7f54a3771740 1 -- 172.21.15.97:0/3675161665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b9900 msgr2=0x5631755d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.917+0000 7f54a3771740 1 --2- 172.21.15.97:0/3675161665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b9900 0x5631755d0be0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f54940324c0 tx=0x7f54940bbf80 comp rx=0 tx=0).stop 2024-03-26T19:01:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.917+0000 7f54a3771740 1 -- 172.21.15.97:0/3675161665 shutdown_connections 2024-03-26T19:01:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.917+0000 7f54a3771740 1 --2- 172.21.15.97:0/3675161665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b9900 0x5631755d0be0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.917+0000 7f54a3771740 1 --2- 172.21.15.97:0/3675161665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.917+0000 7f54a3771740 1 --2- 172.21.15.97:0/3675161665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b3da0 0x5631755b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.917+0000 7f54a3771740 1 -- 172.21.15.97:0/3675161665 >> 172.21.15.97:0/3675161665 conn(0x5631755af310 msgr2=0x5631755b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.918+0000 7f54a3771740 1 -- 172.21.15.97:0/3675161665 shutdown_connections 2024-03-26T19:01:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.918+0000 7f54a3771740 1 -- 172.21.15.97:0/3675161665 wait complete. 2024-03-26T19:01:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.918+0000 7f54a3771740 1 -- start start 2024-03-26T19:01:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.919+0000 7f54a3771740 1 Processor -- start 2024-03-26T19:01:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.919+0000 7f54a3771740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.919+0000 7f54a3771740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.919+0000 7f54a3771740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b9900 0x5631755c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.919+0000 7f54a3771740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b0620 con 0x5631755b9900 2024-03-26T19:01:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.919+0000 7f54a3771740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1aff50 con 0x5631755b4790 2024-03-26T19:01:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.919+0000 7f54a3771740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1ad190 con 0x5631755b3da0 2024-03-26T19:01:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.920+0000 7f54a3771740 10 client.0 _ll_register_callbacks cb 0x7f54940bd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:01:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.920+0000 7f5499400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.920+0000 7f5499e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.920+0000 7f5499e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50318/0 (socket says 172.21.15.97:50318) 2024-03-26T19:01:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.921+0000 7f5499e00640 1 -- 172.21.15.97:0/1957376833 learned_addr learned my addr 172.21.15.97:0/1957376833 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:01:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.921+0000 7f5499e00640 1 -- 172.21.15.97:0/1957376833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 msgr2=0x5631755c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.921+0000 7f5499e00640 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.921+0000 7f5499e00640 1 -- 172.21.15.97:0/1957376833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b9900 msgr2=0x5631755c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:01:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.921+0000 7f5499e00640 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b9900 0x5631755c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.921+0000 7f5499e00640 1 -- 172.21.15.97:0/1957376833 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631755afd80 con 0x5631755b3da0 2024-03-26T19:01:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.922+0000 7f5499e00640 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755c4c40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f548c009460 tx=0x7f548c029c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8aa71a819709d2c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:01:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.922+0000 7f547b400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:01:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.922+0000 7f547b400640 1 -- 172.21.15.97:0/1957376833 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c029420 con 0x5631755b3da0 2024-03-26T19:01:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.922+0000 7f547b400640 1 -- 172.21.15.97:0/1957376833 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f548c0295c0 con 0x5631755b3da0 2024-03-26T19:01:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.922+0000 7f54a3771740 1 -- 172.21.15.97:0/1957376833 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54940da020 con 0x5631755b3da0 2024-03-26T19:01:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.923+0000 7f547b400640 1 -- 172.21.15.97:0/1957376833 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f548c039050 con 0x5631755b3da0 2024-03-26T19:01:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.923+0000 7f54a3771740 10 client.25459 fetch_fsmap learned FSMap version 27 2024-03-26T19:01:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.923+0000 7f54a3771740 1 -- 172.21.15.97:0/1957376833 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631755aa730 con 0x5631755b3da0 2024-03-26T19:01:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.923+0000 7f547b400640 1 -- 172.21.15.97:0/1957376833 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54940da020 con 0x5631755b3da0 2024-03-26T19:01:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.923+0000 7f547b400640 1 -- 172.21.15.97:0/1957376833 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f548c031800 con 0x5631755b3da0 2024-03-26T19:01:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.924+0000 7f54a3771740 10 client.25459 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:01:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.924+0000 7f54a3771740 10 client.25459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:01:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.924+0000 7f54a3771740 20 client.25459 populate_metadata read hostname 'smithi097' 2024-03-26T19:01:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.924+0000 7f54a3771740 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5631756423e0 0x563175644800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:01:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.924+0000 7f54a3771740 4 client.25459 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:01:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.924+0000 7f54a3771740 1 -- 172.21.15.97:0/1957376833 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563175520710 con 0x5631756423e0 2024-03-26T19:01:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.924+0000 7f5499400640 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5631756423e0 0x563175644800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:01:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.925+0000 7f5499400640 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5631756423e0 0x563175644800 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a01bde5c0ada0c04 server_cookie=1a626af2fd2d9ff5 in_seq=0 out_seq=0 2024-03-26T19:01:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.925+0000 7f547b400640 10 client.25459 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f547b400640 1 -- 172.21.15.97:0/1957376833 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563175520710 con 0x5631756423e0 2024-03-26T19:01:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f547b400640 10 client.25459 handle_command_reply: tid=0 2024-03-26T19:01:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f54a3771740 1 client.25459 shutdown 2024-03-26T19:01:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f54a3771740 1 -- 172.21.15.97:0/1957376833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 msgr2=0x5631755c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f54a3771740 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755c4c40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f548c009460 tx=0x7f548c029c20 comp rx=0 tx=0).stop 2024-03-26T19:01:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f54a3771740 1 -- 172.21.15.97:0/1957376833 shutdown_connections 2024-03-26T19:01:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f54a3771740 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5631756423e0 0x563175644800 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f54a3771740 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5631755b9900 0x5631755c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f54a3771740 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5631755b4790 0x5631755c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f54a3771740 1 --2- 172.21.15.97:0/1957376833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5631755b3da0 0x5631755c4c40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f54a3771740 1 -- 172.21.15.97:0/1957376833 >> 172.21.15.97:0/1957376833 conn(0x5631755af310 msgr2=0x5631755b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f547b400640 0 client.25459 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:01:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.943+0000 7f54a3771740 1 -- 172.21.15.97:0/1957376833 shutdown_connections 2024-03-26T19:01:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.944+0000 7f54a3771740 1 -- 172.21.15.97:0/1957376833 wait complete. 2024-03-26T19:01:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.944+0000 7f54a3771740 20 client.25459 trim_cache size 0 max 16384 2024-03-26T19:01:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:01:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:01:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:01:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.944+0000 7f54a0a00640 1 -- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54800a3090 msgr2=0x7f54800a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.944+0000 7f54a0a00640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54800a3090 0x7f54800a5550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5631755b4b90 tx=0x7f548c005f90 comp rx=0 tx=0).stop 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.944+0000 7f54a0a00640 1 -- 172.21.15.97:0/3079369331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c102390 msgr2=0x7f549c1a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.944+0000 7f54a0a00640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c102390 0x7f549c1a54c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5494037040 tx=0x7f5494032ec0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.945+0000 7f54a0a00640 1 -- 172.21.15.97:0/3079369331 shutdown_connections 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.945+0000 7f54a0a00640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f54800a3090 0x7f54800a5550 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.945+0000 7f54a0a00640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c102390 0x7f549c1a54c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.945+0000 7f54a0a00640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549c1019d0 0x7f549c1a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.945+0000 7f54a0a00640 1 --2- 172.21.15.97:0/3079369331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549c1007e0 0x7f549c1a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.945+0000 7f54a0a00640 1 -- 172.21.15.97:0/3079369331 >> 172.21.15.97:0/3079369331 conn(0x7f549c0fbd50 msgr2=0x7f549c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:01:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.945+0000 7f54a0a00640 1 -- 172.21.15.97:0/3079369331 shutdown_connections 2024-03-26T19:01:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:01:49.945+0000 7f54a0a00640 1 -- 172.21.15.97:0/3079369331 wait complete. 2024-03-26T19:01:50.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 145.0 s, 2243.6 tps, lat 44.005 ms stddev 54.903 2024-03-26T19:01:50.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:50 smithi097 ceph-mon[31955]: pgmap v477: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 820 op/s 2024-03-26T19:01:50.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:50 smithi144 ceph-mon[36053]: pgmap v477: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 820 op/s 2024-03-26T19:01:50.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:50 smithi105 ceph-mon[36270]: pgmap v477: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 820 op/s 2024-03-26T19:01:52.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:52 smithi097 ceph-mon[31955]: pgmap v478: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 700 op/s 2024-03-26T19:01:52.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:52 smithi144 ceph-mon[36053]: pgmap v478: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 700 op/s 2024-03-26T19:01:52.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:52 smithi105 ceph-mon[36270]: pgmap v478: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 700 op/s 2024-03-26T19:01:54.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:54 smithi097 ceph-mon[31955]: pgmap v479: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 624 op/s 2024-03-26T19:01:54.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:54 smithi144 ceph-mon[36053]: pgmap v479: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 624 op/s 2024-03-26T19:01:54.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:54 smithi105 ceph-mon[36270]: pgmap v479: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 624 op/s 2024-03-26T19:01:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 150.0 s, 2150.4 tps, lat 47.169 ms stddev 58.351 2024-03-26T19:01:56.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:56 smithi097 ceph-mon[31955]: pgmap v480: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 728 op/s 2024-03-26T19:01:56.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:56 smithi144 ceph-mon[36053]: pgmap v480: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 728 op/s 2024-03-26T19:01:56.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:56 smithi105 ceph-mon[36270]: pgmap v480: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 728 op/s 2024-03-26T19:01:58.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:01:58 smithi144 ceph-mon[36053]: pgmap v481: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 595 op/s 2024-03-26T19:01:58.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:01:58 smithi105 ceph-mon[36270]: pgmap v481: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 595 op/s 2024-03-26T19:01:58.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:01:58 smithi097 ceph-mon[31955]: pgmap v481: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 595 op/s 2024-03-26T19:02:00.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 155.0 s, 2359.5 tps, lat 41.987 ms stddev 46.547 2024-03-26T19:02:00.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:00 smithi144 ceph-mon[36053]: pgmap v482: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 797 op/s 2024-03-26T19:02:00.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:00 smithi105 ceph-mon[36270]: pgmap v482: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 797 op/s 2024-03-26T19:02:00.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:00 smithi097 ceph-mon[31955]: pgmap v482: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 797 op/s 2024-03-26T19:02:02.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:02 smithi144 ceph-mon[36053]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 652 op/s 2024-03-26T19:02:02.407 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:02 smithi097 ceph-mon[31955]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 652 op/s 2024-03-26T19:02:02.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:02 smithi105 ceph-mon[36270]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 652 op/s 2024-03-26T19:02:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.127+0000 7f7989a00640 1 -- 172.21.15.97:0/2148933201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984107f20 msgr2=0x7f7984108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.127+0000 7f7989a00640 1 --2- 172.21.15.97:0/2148933201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984107f20 0x7f7984108340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7974009900 tx=0x7f797402bc40 comp rx=0 tx=0).stop 2024-03-26T19:02:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.128+0000 7f7989a00640 1 -- 172.21.15.97:0/2148933201 shutdown_connections 2024-03-26T19:02:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.128+0000 7f7989a00640 1 --2- 172.21.15.97:0/2148933201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7984109a30 0x7f7984110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.128+0000 7f7989a00640 1 --2- 172.21.15.97:0/2148933201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984109070 0x7f79841094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.128+0000 7f7989a00640 1 --2- 172.21.15.97:0/2148933201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984107f20 0x7f7984108340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.128+0000 7f7989a00640 1 -- 172.21.15.97:0/2148933201 >> 172.21.15.97:0/2148933201 conn(0x7f7984076ed0 msgr2=0x7f79840792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.128+0000 7f7989a00640 1 -- 172.21.15.97:0/2148933201 shutdown_connections 2024-03-26T19:02:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.128+0000 7f7989a00640 1 -- 172.21.15.97:0/2148933201 wait complete. 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.129+0000 7f7989a00640 1 Processor -- start 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.129+0000 7f7989a00640 1 -- start start 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.129+0000 7f7989a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7984107f20 0x7f798419e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.129+0000 7f7989a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984109070 0x7f798419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7989a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984109a30 0x7f79841a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7989a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7984113c00 con 0x7f7984109a30 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7989a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7984113a80 con 0x7f7984107f20 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7989a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7984113d80 con 0x7f7984109070 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7982a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7984107f20 0x7f798419e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7982a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7984107f20 0x7f798419e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:56656/0 (socket says 172.21.15.97:56656) 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7982a00640 1 -- 172.21.15.97:0/3066546186 learned_addr learned my addr 172.21.15.97:0/3066546186 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7982000640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984109070 0x7f798419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7983400640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984109a30 0x7f79841a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7982a00640 1 -- 172.21.15.97:0/3066546186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984109070 msgr2=0x7f798419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7982a00640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984109070 0x7f798419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7982a00640 1 -- 172.21.15.97:0/3066546186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984109a30 msgr2=0x7f79841a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7982a00640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984109a30 0x7f79841a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.130+0000 7f7982a00640 1 -- 172.21.15.97:0/3066546186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79841a5da0 con 0x7f7984107f20 2024-03-26T19:02:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.131+0000 7f7982000640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984109070 0x7f798419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:02:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.131+0000 7f7982a00640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7984107f20 0x7f798419e180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f797402c250 tx=0x7f7974004e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.132+0000 7f797b400640 1 -- 172.21.15.97:0/3066546186 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974041070 con 0x7f7984107f20 2024-03-26T19:02:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.132+0000 7f797b400640 1 -- 172.21.15.97:0/3066546186 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79740054e0 con 0x7f7984107f20 2024-03-26T19:02:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.132+0000 7f797b400640 1 -- 172.21.15.97:0/3066546186 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974034440 con 0x7f7984107f20 2024-03-26T19:02:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.132+0000 7f7989a00640 1 -- 172.21.15.97:0/3066546186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79841a6030 con 0x7f7984107f20 2024-03-26T19:02:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.132+0000 7f7989a00640 1 -- 172.21.15.97:0/3066546186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79841a64d0 con 0x7f7984107f20 2024-03-26T19:02:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.136+0000 7f798c827740 1 Processor -- start 2024-03-26T19:02:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f798c827740 1 -- start start 2024-03-26T19:02:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f798c827740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a47da0 0x555d52a481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f798c827740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a48790 0x555d52a4d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f798c827740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a4d900 0x555d52a4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f798c827740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555d529ace80 con 0x555d52a47da0 2024-03-26T19:02:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f798c827740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555d52955660 con 0x555d52a4d900 2024-03-26T19:02:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f798c827740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d529a8e30 con 0x555d52a48790 2024-03-26T19:02:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f7982000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a4d900 0x555d52a4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f7983400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a47da0 0x555d52a481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f7983400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a47da0 0x555d52a481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49654/0 (socket says 172.21.15.97:49654) 2024-03-26T19:02:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.137+0000 7f7983400640 1 -- 172.21.15.97:0/428649807 learned_addr learned my addr 172.21.15.97:0/428649807 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.138+0000 7f7982000640 1 -- 172.21.15.97:0/428649807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a48790 msgr2=0x555d52a4d290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:02:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.138+0000 7f7982000640 1 --2- 172.21.15.97:0/428649807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a48790 0x555d52a4d290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.138+0000 7f7982000640 1 -- 172.21.15.97:0/428649807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a47da0 msgr2=0x555d52a481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.138+0000 7f7982000640 1 --2- 172.21.15.97:0/428649807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a47da0 0x555d52a481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.138+0000 7f7982000640 1 -- 172.21.15.97:0/428649807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d526c1210 con 0x555d52a4d900 2024-03-26T19:02:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.139+0000 7f7982000640 1 --2- 172.21.15.97:0/428649807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a4d900 0x555d52a4fd90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7984107c40 tx=0x7f79700118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=558035c8529683c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.139+0000 7f796fe00640 1 -- 172.21.15.97:0/428649807 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970004310 con 0x555d52a4d900 2024-03-26T19:02:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.139+0000 7f796fe00640 1 -- 172.21.15.97:0/428649807 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79700044b0 con 0x555d52a4d900 2024-03-26T19:02:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.139+0000 7f796fe00640 1 -- 172.21.15.97:0/428649807 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970003850 con 0x555d52a4d900 2024-03-26T19:02:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.140+0000 7f797b400640 1 -- 172.21.15.97:0/3066546186 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f7974044070 con 0x7f7984107f20 2024-03-26T19:02:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.140+0000 7f798c827740 1 -- 172.21.15.97:0/428649807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a4d900 msgr2=0x555d52a4fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.140+0000 7f798c827740 1 --2- 172.21.15.97:0/428649807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a4d900 0x555d52a4fd90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7984107c40 tx=0x7f79700118d0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.141+0000 7f798c827740 1 -- 172.21.15.97:0/428649807 shutdown_connections 2024-03-26T19:02:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.141+0000 7f798c827740 1 --2- 172.21.15.97:0/428649807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a4d900 0x555d52a4fd90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.141+0000 7f798c827740 1 --2- 172.21.15.97:0/428649807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a48790 0x555d52a4d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.141+0000 7f798c827740 1 --2- 172.21.15.97:0/428649807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a47da0 0x555d52a481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.141+0000 7f798c827740 1 -- 172.21.15.97:0/428649807 >> 172.21.15.97:0/428649807 conn(0x555d52a43310 msgr2=0x555d52a43710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.142+0000 7f798c827740 1 -- 172.21.15.97:0/428649807 shutdown_connections 2024-03-26T19:02:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.142+0000 7f798c827740 1 -- 172.21.15.97:0/428649807 wait complete. 2024-03-26T19:02:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.142+0000 7f798c827740 1 -- start start 2024-03-26T19:02:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.142+0000 7f798c827740 1 Processor -- start 2024-03-26T19:02:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f797b400640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79680a2fa0 0x7f79680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f798c827740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 0x555d52a51f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f797b400640 1 -- 172.21.15.97:0/3066546186 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f79740f0d90 con 0x7f7984107f20 2024-03-26T19:02:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f798c827740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f798c827740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a57490 0x555d52a59880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f798c827740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555d52a508b0 con 0x555d52a57490 2024-03-26T19:02:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f798c827740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555d52968ac0 con 0x555d52a47da0 2024-03-26T19:02:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f798c827740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d52a50a30 con 0x555d52a4d900 2024-03-26T19:02:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f798c827740 10 client.0 _ll_register_callbacks cb 0x555d52968490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:02:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f7982a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 0x555d52a51f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f7982a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 0x555d52a51f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:56670/0 (socket says 172.21.15.97:56670) 2024-03-26T19:02:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.143+0000 7f7982a00640 1 -- 172.21.15.97:0/1656715967 learned_addr learned my addr 172.21.15.97:0/1656715967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.144+0000 7f7982000640 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.144+0000 7f7983400640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79680a2fa0 0x7f79680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.144+0000 7f7982000640 1 -- 172.21.15.97:0/1656715967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 msgr2=0x555d52a51f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.144+0000 7f7982000640 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 0x555d52a51f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.144+0000 7f7982000640 1 -- 172.21.15.97:0/1656715967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a57490 msgr2=0x555d52a59880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.144+0000 7f7982000640 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a57490 0x555d52a59880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.144+0000 7f7982000640 1 -- 172.21.15.97:0/1656715967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d5278d620 con 0x555d52a4d900 2024-03-26T19:02:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.145+0000 7f7982000640 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a524d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7970000c00 tx=0x7f79700048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76491f991c502ecd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.145+0000 7f796cc00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:02:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.145+0000 7f796cc00640 1 -- 172.21.15.97:0/1656715967 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970027030 con 0x555d52a4d900 2024-03-26T19:02:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.145+0000 7f796cc00640 1 -- 172.21.15.97:0/1656715967 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7970004fc0 con 0x555d52a4d900 2024-03-26T19:02:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.146+0000 7f798c827740 1 -- 172.21.15.97:0/1656715967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79700044b0 con 0x555d52a4d900 2024-03-26T19:02:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.146+0000 7f796cc00640 1 -- 172.21.15.97:0/1656715967 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7970032050 con 0x555d52a4d900 2024-03-26T19:02:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.146+0000 7f798c827740 10 client.25468 fetch_fsmap learned FSMap version 27 2024-03-26T19:02:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.146+0000 7f798c827740 1 -- 172.21.15.97:0/1656715967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d52a5cbf0 con 0x555d52a4d900 2024-03-26T19:02:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.147+0000 7f796cc00640 1 -- 172.21.15.97:0/1656715967 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79700044b0 con 0x555d52a4d900 2024-03-26T19:02:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.147+0000 7f796cc00640 1 -- 172.21.15.97:0/1656715967 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f797000f480 con 0x555d52a4d900 2024-03-26T19:02:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.147+0000 7f798c827740 10 client.25468 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:02:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.147+0000 7f798c827740 10 client.25468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:02:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.147+0000 7f798c827740 20 client.25468 populate_metadata read hostname 'smithi097' 2024-03-26T19:02:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.147+0000 7f798c827740 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x555d52a5e110 0x555d52a60500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.147+0000 7f798c827740 4 client.25468 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:02:03.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.147+0000 7f798c827740 1 -- 172.21.15.97:0/1656715967 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d529ba3c0 con 0x555d52a5e110 2024-03-26T19:02:03.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.148+0000 7f7982a00640 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x555d52a5e110 0x555d52a60500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.149+0000 7f7983400640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79680a2fa0 0x7f79680a5460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f797c005fd0 tx=0x7f797c009d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:03.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.149+0000 7f7982a00640 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x555d52a5e110 0x555d52a60500 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e075ea1a38e4b1e4 server_cookie=15486ef337b7045f in_seq=0 out_seq=0 2024-03-26T19:02:03.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.149+0000 7f796cc00640 10 client.25468 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.150+0000 7f796cc00640 1 -- 172.21.15.97:0/1656715967 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555d529ba3c0 con 0x555d52a5e110 2024-03-26T19:02:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.150+0000 7f796cc00640 10 client.25468 handle_command_reply: tid=0 2024-03-26T19:02:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 client.25468 shutdown 2024-03-26T19:02:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 -- 172.21.15.97:0/1656715967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 msgr2=0x555d52a524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a524d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7970000c00 tx=0x7f79700048f0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 -- 172.21.15.97:0/1656715967 shutdown_connections 2024-03-26T19:02:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x555d52a5e110 0x555d52a60500 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a57490 0x555d52a59880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a524d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 --2- 172.21.15.97:0/1656715967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 0x555d52a51f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 -- 172.21.15.97:0/1656715967 >> 172.21.15.97:0/1656715967 conn(0x555d52a43310 msgr2=0x555d52a42540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f796cc00640 0 client.25468 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 -- 172.21.15.97:0/1656715967 shutdown_connections 2024-03-26T19:02:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.151+0000 7f798c827740 1 -- 172.21.15.97:0/1656715967 wait complete. 2024-03-26T19:02:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.152+0000 7f798c827740 20 client.25468 trim_cache size 0 max 16384 2024-03-26T19:02:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.175+0000 7f798c827740 1 Processor -- start 2024-03-26T19:02:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.175+0000 7f798c827740 1 -- start start 2024-03-26T19:02:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.175+0000 7f798c827740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 0x555d52a46c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.175+0000 7f798c827740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a60120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.175+0000 7f798c827740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a57490 0x555d52a49200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.175+0000 7f798c827740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f79841af360 con 0x555d52a57490 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.175+0000 7f798c827740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f79841b0ea0 con 0x555d52a47da0 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.175+0000 7f798c827740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79841afa30 con 0x555d52a4d900 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 0x555d52a46c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a60120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 0x555d52a46c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:56672/0 (socket says 172.21.15.97:56672) 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982a00640 1 -- 172.21.15.97:0/2546593155 learned_addr learned my addr 172.21.15.97:0/2546593155 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a60120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:48034/0 (socket says 172.21.15.97:48034) 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982000640 1 -- 172.21.15.97:0/2546593155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 msgr2=0x555d52a46c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982000640 1 --2- 172.21.15.97:0/2546593155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 0x555d52a46c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982000640 1 -- 172.21.15.97:0/2546593155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a57490 msgr2=0x555d52a49200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982000640 1 --2- 172.21.15.97:0/2546593155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a57490 0x555d52a49200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982000640 1 -- 172.21.15.97:0/2546593155 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d52a473a0 con 0x555d52a4d900 2024-03-26T19:02:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.176+0000 7f7982000640 1 --2- 172.21.15.97:0/2546593155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a60120 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7970011dd0 tx=0x7f7970004640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c79670faad503520 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.177+0000 7f796f400640 1 -- 172.21.15.97:0/2546593155 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970027040 con 0x555d52a4d900 2024-03-26T19:02:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.177+0000 7f796f400640 1 -- 172.21.15.97:0/2546593155 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79700054a0 con 0x555d52a4d900 2024-03-26T19:02:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.177+0000 7f796f400640 1 -- 172.21.15.97:0/2546593155 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79700035f0 con 0x555d52a4d900 2024-03-26T19:02:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.178+0000 7f798c827740 1 -- 172.21.15.97:0/2546593155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 msgr2=0x555d52a60120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.178+0000 7f798c827740 1 --2- 172.21.15.97:0/2546593155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a60120 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7970011dd0 tx=0x7f7970004640 comp rx=0 tx=0).stop 2024-03-26T19:02:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.178+0000 7f798c827740 1 -- 172.21.15.97:0/2546593155 shutdown_connections 2024-03-26T19:02:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.178+0000 7f798c827740 1 --2- 172.21.15.97:0/2546593155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a57490 0x555d52a49200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.178+0000 7f798c827740 1 --2- 172.21.15.97:0/2546593155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a4d900 0x555d52a60120 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.178+0000 7f798c827740 1 --2- 172.21.15.97:0/2546593155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a47da0 0x555d52a46c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.178+0000 7f798c827740 1 -- 172.21.15.97:0/2546593155 >> 172.21.15.97:0/2546593155 conn(0x555d52a43310 msgr2=0x555d52a46570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.178+0000 7f798c827740 1 -- 172.21.15.97:0/2546593155 shutdown_connections 2024-03-26T19:02:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.178+0000 7f798c827740 1 -- 172.21.15.97:0/2546593155 wait complete. 2024-03-26T19:02:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.179+0000 7f798c827740 1 -- start start 2024-03-26T19:02:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.179+0000 7f798c827740 1 Processor -- start 2024-03-26T19:02:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f798c827740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a47da0 0x555d52a6ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f798c827740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a4d900 0x555d52a6f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f798c827740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a57490 0x555d52a70370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f798c827740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f79841b07d0 con 0x555d52a4d900 2024-03-26T19:02:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f798c827740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f79841b0100 con 0x555d52a57490 2024-03-26T19:02:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f798c827740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79841ad340 con 0x555d52a47da0 2024-03-26T19:02:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f798c827740 10 client.0 _ll_register_callbacks cb 0x7f7970005c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f7982a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a47da0 0x555d52a6ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f7983400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a57490 0x555d52a70370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f7982a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a47da0 0x555d52a6ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:48038/0 (socket says 172.21.15.97:48038) 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f7982a00640 1 -- 172.21.15.97:0/2397934421 learned_addr learned my addr 172.21.15.97:0/2397934421 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f7982a00640 1 -- 172.21.15.97:0/2397934421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a57490 msgr2=0x555d52a70370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f7982a00640 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a57490 0x555d52a70370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f7982a00640 1 -- 172.21.15.97:0/2397934421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a4d900 msgr2=0x555d52a6f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.180+0000 7f7982a00640 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a4d900 0x555d52a6f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.181+0000 7f7982a00640 1 -- 172.21.15.97:0/2397934421 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d52a70960 con 0x555d52a47da0 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.181+0000 7f7982a00640 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a47da0 0x555d52a6ee20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79740b9f70 tx=0x7f79740a8040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=454f7a2d347fa5b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.181+0000 7f796cc00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.181+0000 7f796cc00640 1 -- 172.21.15.97:0/2397934421 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797404b040 con 0x555d52a47da0 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.181+0000 7f796cc00640 1 -- 172.21.15.97:0/2397934421 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f79740b9c10 con 0x555d52a47da0 2024-03-26T19:02:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.181+0000 7f798c827740 1 -- 172.21.15.97:0/2397934421 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79700054a0 con 0x555d52a47da0 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.181+0000 7f796cc00640 1 -- 172.21.15.97:0/2397934421 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f79740dc020 con 0x555d52a47da0 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.181+0000 7f798c827740 10 client.25474 fetch_fsmap learned FSMap version 27 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.181+0000 7f798c827740 1 -- 172.21.15.97:0/2397934421 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d52ac3f70 con 0x555d52a47da0 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.182+0000 7f796cc00640 1 -- 172.21.15.97:0/2397934421 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79700054a0 con 0x555d52a47da0 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.182+0000 7f796cc00640 1 -- 172.21.15.97:0/2397934421 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f79740035f0 con 0x555d52a47da0 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.182+0000 7f798c827740 10 client.25474 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.182+0000 7f798c827740 10 client.25474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.182+0000 7f798c827740 20 client.25474 populate_metadata read hostname 'smithi097' 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.182+0000 7f798c827740 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x555d52ac48d0 0x555d52ac6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.182+0000 7f798c827740 4 client.25474 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.182+0000 7f798c827740 1 -- 172.21.15.97:0/2397934421 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d529b4710 con 0x555d52ac48d0 2024-03-26T19:02:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.182+0000 7f7982000640 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x555d52ac48d0 0x555d52ac6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.183+0000 7f7982000640 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x555d52ac48d0 0x555d52ac6cf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe52ded1ec041ef server_cookie=ee2383100258d908 in_seq=0 out_seq=0 2024-03-26T19:02:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.183+0000 7f796cc00640 10 client.25474 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.197+0000 7f796cc00640 1 -- 172.21.15.97:0/2397934421 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d529b4710 con 0x555d52ac48d0 2024-03-26T19:02:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.197+0000 7f796cc00640 10 client.25474 handle_command_reply: tid=0 2024-03-26T19:02:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.197+0000 7f798c827740 1 client.25474 shutdown 2024-03-26T19:02:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.197+0000 7f798c827740 1 -- 172.21.15.97:0/2397934421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a47da0 msgr2=0x555d52a6ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.197+0000 7f798c827740 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a47da0 0x555d52a6ee20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79740b9f70 tx=0x7f79740a8040 comp rx=0 tx=0).stop 2024-03-26T19:02:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.198+0000 7f798c827740 1 -- 172.21.15.97:0/2397934421 shutdown_connections 2024-03-26T19:02:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.198+0000 7f798c827740 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x555d52ac48d0 0x555d52ac6cf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.198+0000 7f798c827740 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d52a57490 0x555d52a70370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.198+0000 7f798c827740 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d52a4d900 0x555d52a6f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.198+0000 7f798c827740 1 --2- 172.21.15.97:0/2397934421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d52a47da0 0x555d52a6ee20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.198+0000 7f798c827740 1 -- 172.21.15.97:0/2397934421 >> 172.21.15.97:0/2397934421 conn(0x555d52a43310 msgr2=0x555d52a41cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.198+0000 7f796cc00640 0 client.25474 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.198+0000 7f798c827740 1 -- 172.21.15.97:0/2397934421 shutdown_connections 2024-03-26T19:02:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.199+0000 7f798c827740 1 -- 172.21.15.97:0/2397934421 wait complete. 2024-03-26T19:02:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.199+0000 7f798c827740 20 client.25474 trim_cache size 0 max 16384 2024-03-26T19:02:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:02:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:02:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:02:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:02:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.202+0000 7f7989a00640 1 -- 172.21.15.97:0/3066546186 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79680a2fa0 msgr2=0x7f79680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.202+0000 7f7989a00640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79680a2fa0 0x7f79680a5460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f797c005fd0 tx=0x7f797c009d80 comp rx=0 tx=0).stop 2024-03-26T19:02:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.202+0000 7f7989a00640 1 -- 172.21.15.97:0/3066546186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7984107f20 msgr2=0x7f798419e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.202+0000 7f7989a00640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7984107f20 0x7f798419e180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f797402c250 tx=0x7f7974004e30 comp rx=0 tx=0).stop 2024-03-26T19:02:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.202+0000 7f7989a00640 1 -- 172.21.15.97:0/3066546186 shutdown_connections 2024-03-26T19:02:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.202+0000 7f7989a00640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f79680a2fa0 0x7f79680a5460 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.202+0000 7f7989a00640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984109a30 0x7f79841a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.202+0000 7f7989a00640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7984109070 0x7f798419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.202+0000 7f7989a00640 1 --2- 172.21.15.97:0/3066546186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7984107f20 0x7f798419e180 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.202+0000 7f7989a00640 1 -- 172.21.15.97:0/3066546186 >> 172.21.15.97:0/3066546186 conn(0x7f7984076ed0 msgr2=0x7f798410c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.203+0000 7f7989a00640 1 -- 172.21.15.97:0/3066546186 shutdown_connections 2024-03-26T19:02:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:03.203+0000 7f7989a00640 1 -- 172.21.15.97:0/3066546186 wait complete. 2024-03-26T19:02:04.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:04 smithi144 ceph-mon[36053]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.0 MiB/s wr, 630 op/s 2024-03-26T19:02:04.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:04 smithi105 ceph-mon[36270]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.0 MiB/s wr, 630 op/s 2024-03-26T19:02:04.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:04 smithi097 ceph-mon[31955]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.0 MiB/s wr, 630 op/s 2024-03-26T19:02:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 160.0 s, 2187.8 tps, lat 46.058 ms stddev 56.697 2024-03-26T19:02:06.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:06 smithi105 ceph-mon[36270]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 755 op/s 2024-03-26T19:02:06.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:06 smithi097 ceph-mon[31955]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 755 op/s 2024-03-26T19:02:06.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:06 smithi144 ceph-mon[36053]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 755 op/s 2024-03-26T19:02:07.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:07 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:02:07.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:07 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:02:07.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:07 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:02:08.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:08 smithi105 ceph-mon[36270]: pgmap v486: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 MiB/s wr, 630 op/s 2024-03-26T19:02:08.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:08 smithi097 ceph-mon[31955]: pgmap v486: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 MiB/s wr, 630 op/s 2024-03-26T19:02:08.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:08 smithi144 ceph-mon[36053]: pgmap v486: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 MiB/s wr, 630 op/s 2024-03-26T19:02:09.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:02:09.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:02:09.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:02:09.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:02:09.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:02:09.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:02:09.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:02:09.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:02:09.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:02:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 165.0 s, 2482.0 tps, lat 40.446 ms stddev 48.581 2024-03-26T19:02:10.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:10 smithi105 ceph-mon[36270]: pgmap v487: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 796 op/s 2024-03-26T19:02:10.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:10 smithi097 ceph-mon[31955]: pgmap v487: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 796 op/s 2024-03-26T19:02:10.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:10 smithi144 ceph-mon[36053]: pgmap v487: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 796 op/s 2024-03-26T19:02:12.429 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:12 smithi144 ceph-mon[36053]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 633 op/s 2024-03-26T19:02:12.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:12 smithi105 ceph-mon[36270]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 633 op/s 2024-03-26T19:02:12.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:12 smithi097 ceph-mon[31955]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 633 op/s 2024-03-26T19:02:14.461 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:14 smithi105 ceph-mon[36270]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.3 MiB/s wr, 623 op/s 2024-03-26T19:02:14.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:14 smithi097 ceph-mon[31955]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.3 MiB/s wr, 623 op/s 2024-03-26T19:02:14.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:14 smithi144 ceph-mon[36053]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.3 MiB/s wr, 623 op/s 2024-03-26T19:02:15.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 170.0 s, 3124.6 tps, lat 31.970 ms stddev 32.435 2024-03-26T19:02:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.389+0000 7fdd18c00640 1 -- 172.21.15.97:0/574367462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141032e0 msgr2=0x7fdd14109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.389+0000 7fdd18c00640 1 --2- 172.21.15.97:0/574367462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141032e0 0x7fdd14109b70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009900 tx=0x7fdd0002bc30 comp rx=0 tx=0).stop 2024-03-26T19:02:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.391+0000 7fdd18c00640 1 -- 172.21.15.97:0/574367462 shutdown_connections 2024-03-26T19:02:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.391+0000 7fdd18c00640 1 --2- 172.21.15.97:0/574367462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141032e0 0x7fdd14109b70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.391+0000 7fdd18c00640 1 --2- 172.21.15.97:0/574367462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd14102920 0x7fdd14102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.391+0000 7fdd18c00640 1 --2- 172.21.15.97:0/574367462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd14101730 0x7fdd14101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.391+0000 7fdd18c00640 1 -- 172.21.15.97:0/574367462 >> 172.21.15.97:0/574367462 conn(0x7fdd140fcca0 msgr2=0x7fdd140ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.392+0000 7fdd18c00640 1 -- 172.21.15.97:0/574367462 shutdown_connections 2024-03-26T19:02:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.392+0000 7fdd18c00640 1 -- 172.21.15.97:0/574367462 wait complete. 2024-03-26T19:02:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.393+0000 7fdd18c00640 1 Processor -- start 2024-03-26T19:02:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.395+0000 7fdd18c00640 1 -- start start 2024-03-26T19:02:16.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.396+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd14101730 0x7fdd141a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.397+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd14102920 0x7fdd141a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.397+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141032e0 0x7fdd141adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.397+0000 7fdd18c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1406bd30 con 0x7fdd14101730 2024-03-26T19:02:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.397+0000 7fdd18c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1406bbb0 con 0x7fdd14102920 2024-03-26T19:02:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.397+0000 7fdd18c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1406beb0 con 0x7fdd141032e0 2024-03-26T19:02:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.397+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141032e0 0x7fdd141adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.398+0000 7fdd12a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd14102920 0x7fdd141a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.398+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141032e0 0x7fdd141adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44272/0 (socket says 172.21.15.97:44272) 2024-03-26T19:02:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.398+0000 7fdd13e00640 1 -- 172.21.15.97:0/1297894014 learned_addr learned my addr 172.21.15.97:0/1297894014 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.398+0000 7fdd13400640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd14101730 0x7fdd141a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.398+0000 7fdd13e00640 1 -- 172.21.15.97:0/1297894014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd14102920 msgr2=0x7fdd141a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.398+0000 7fdd13e00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd14102920 0x7fdd141a6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.398+0000 7fdd13e00640 1 -- 172.21.15.97:0/1297894014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd14101730 msgr2=0x7fdd141a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.398+0000 7fdd13e00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd14101730 0x7fdd141a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.399+0000 7fdd13e00640 1 -- 172.21.15.97:0/1297894014 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd141ae340 con 0x7fdd141032e0 2024-03-26T19:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.399+0000 7fdd12a00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd14102920 0x7fdd141a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.399+0000 7fdd13e00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141032e0 0x7fdd141adcd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdd0002c7b0 tx=0x7fdd00005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.400+0000 7fdd0be00640 1 -- 172.21.15.97:0/1297894014 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00041070 con 0x7fdd141032e0 2024-03-26T19:02:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.401+0000 7fdd0be00640 1 -- 172.21.15.97:0/1297894014 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd000054e0 con 0x7fdd141032e0 2024-03-26T19:02:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.401+0000 7fdd0be00640 1 -- 172.21.15.97:0/1297894014 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00003e60 con 0x7fdd141032e0 2024-03-26T19:02:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.401+0000 7fdd18c00640 1 -- 172.21.15.97:0/1297894014 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd141ae5d0 con 0x7fdd141032e0 2024-03-26T19:02:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.402+0000 7fdd18c00640 1 -- 172.21.15.97:0/1297894014 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd141aea70 con 0x7fdd141032e0 2024-03-26T19:02:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.408+0000 7fdd0be00640 1 -- 172.21.15.97:0/1297894014 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fdd00044070 con 0x7fdd141032e0 2024-03-26T19:02:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.411+0000 7fdd1cf94740 1 Processor -- start 2024-03-26T19:02:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.411+0000 7fdd0be00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcf80a2f90 0x7fdcf80a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.411+0000 7fdd1cf94740 1 -- start start 2024-03-26T19:02:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.412+0000 7fdd0be00640 1 -- 172.21.15.97:0/1297894014 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fdd000f0980 con 0x7fdd141032e0 2024-03-26T19:02:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.412+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af23380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.412+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 0x55a82af28450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.412+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af2af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.412+0000 7fdd12a00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcf80a2f90 0x7fdcf80a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.413+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a82ae87e80 con 0x55a82af23950 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.413+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a82ae30660 con 0x55a82af28ac0 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.413+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a82ae83e30 con 0x55a82af22f60 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.413+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af2af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.413+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af2af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:55610/0 (socket says 172.21.15.97:55610) 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.413+0000 7fdd13e00640 1 -- 172.21.15.97:0/960358274 learned_addr learned my addr 172.21.15.97:0/960358274 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.413+0000 7fdd13400640 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af23380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.413+0000 7fdd12a00640 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 0x55a82af28450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.413+0000 7fdd13400640 1 -- 172.21.15.97:0/960358274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 msgr2=0x55a82af2af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.413+0000 7fdd13400640 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af2af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.414+0000 7fdd13400640 1 -- 172.21.15.97:0/960358274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 msgr2=0x55a82af28450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.414+0000 7fdd13400640 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 0x55a82af28450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.414+0000 7fdd13400640 1 -- 172.21.15.97:0/960358274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82ab9c210 con 0x55a82af22f60 2024-03-26T19:02:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.414+0000 7fdd13e00640 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af2af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:02:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.414+0000 7fdd12a00640 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 0x55a82af28450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:02:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.415+0000 7fdd13400640 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af23380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005fd0 tx=0x7fdd0c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2058ff22e4490bf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.415+0000 7fdd12a00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcf80a2f90 0x7fdcf80a5450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdd141a7c30 tx=0x7fdd04013a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.416+0000 7fdcffe00640 1 -- 172.21.15.97:0/960358274 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c004b20 con 0x55a82af22f60 2024-03-26T19:02:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.416+0000 7fdcffe00640 1 -- 172.21.15.97:0/960358274 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0c004cc0 con 0x55a82af22f60 2024-03-26T19:02:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.416+0000 7fdcffe00640 1 -- 172.21.15.97:0/960358274 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c004b20 con 0x55a82af22f60 2024-03-26T19:02:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.417+0000 7fdd1cf94740 1 -- 172.21.15.97:0/960358274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 msgr2=0x55a82af23380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.417+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af23380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005fd0 tx=0x7fdd0c00f730 comp rx=0 tx=0).stop 2024-03-26T19:02:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.418+0000 7fdd1cf94740 1 -- 172.21.15.97:0/960358274 shutdown_connections 2024-03-26T19:02:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.418+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af2af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.418+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 0x55a82af28450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.418+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/960358274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af23380 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.418+0000 7fdd1cf94740 1 -- 172.21.15.97:0/960358274 >> 172.21.15.97:0/960358274 conn(0x55a82af1e4d0 msgr2=0x55a82af1e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.418+0000 7fdd1cf94740 1 -- 172.21.15.97:0/960358274 shutdown_connections 2024-03-26T19:02:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.418+0000 7fdd1cf94740 1 -- 172.21.15.97:0/960358274 wait complete. 2024-03-26T19:02:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.419+0000 7fdd1cf94740 1 -- start start 2024-03-26T19:02:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.419+0000 7fdd1cf94740 1 Processor -- start 2024-03-26T19:02:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.420+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af22f60 0x55a82af2cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.420+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af23950 0x55a82af2d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.420+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.420+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a82af2bac0 con 0x55a82af22f60 2024-03-26T19:02:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.420+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a82ae43ac0 con 0x55a82af28ac0 2024-03-26T19:02:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.420+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a82af2bc40 con 0x55a82af23950 2024-03-26T19:02:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.420+0000 7fdd1cf94740 10 client.0 _ll_register_callbacks cb 0x55a82ae43490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:02:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.421+0000 7fdd12a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af23950 0x55a82af2d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.421+0000 7fdd12a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af23950 0x55a82af2d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44282/0 (socket says 172.21.15.97:44282) 2024-03-26T19:02:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.421+0000 7fdd12a00640 1 -- 172.21.15.97:0/3301166323 learned_addr learned my addr 172.21.15.97:0/3301166323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.421+0000 7fdd13400640 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af22f60 0x55a82af2cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.421+0000 7fdd13e00640 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.422+0000 7fdd12a00640 1 -- 172.21.15.97:0/3301166323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 msgr2=0x55a82af344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.422+0000 7fdd12a00640 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.422+0000 7fdd12a00640 1 -- 172.21.15.97:0/3301166323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af22f60 msgr2=0x55a82af2cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.422+0000 7fdd12a00640 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af22f60 0x55a82af2cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.422+0000 7fdd12a00640 1 -- 172.21.15.97:0/3301166323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a82ac68620 con 0x55a82af23950 2024-03-26T19:02:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.423+0000 7fdd12a00640 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af23950 0x55a82af2d510 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdd0401b470 tx=0x7fdd04004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=967156d3fc33fb24 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.423+0000 7fdcfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:02:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.423+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3301166323 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04005020 con 0x55a82af23950 2024-03-26T19:02:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.423+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3301166323 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fdd04018070 con 0x55a82af23950 2024-03-26T19:02:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.424+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3301166323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd0c004cc0 con 0x55a82af23950 2024-03-26T19:02:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.424+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3301166323 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd040051c0 con 0x55a82af23950 2024-03-26T19:02:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.424+0000 7fdd1cf94740 10 client.25486 fetch_fsmap learned FSMap version 27 2024-03-26T19:02:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.424+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3301166323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a82af37880 con 0x55a82af23950 2024-03-26T19:02:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.425+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3301166323 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0c004cc0 con 0x55a82af23950 2024-03-26T19:02:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.426+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3301166323 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd04026420 con 0x55a82af23950 2024-03-26T19:02:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.426+0000 7fdd1cf94740 10 client.25486 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:02:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.426+0000 7fdd1cf94740 10 client.25486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:02:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.426+0000 7fdd1cf94740 20 client.25486 populate_metadata read hostname 'smithi097' 2024-03-26T19:02:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.426+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a82af38fc0 0x55a82af3b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.426+0000 7fdd1cf94740 4 client.25486 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:02:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.426+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3301166323 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a82ae953c0 con 0x55a82af38fc0 2024-03-26T19:02:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.426+0000 7fdd13400640 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a82af38fc0 0x55a82af3b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.427+0000 7fdd13400640 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a82af38fc0 0x55a82af3b400 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81adc9596a470f49 server_cookie=307b69e2aadc3800 in_seq=0 out_seq=0 2024-03-26T19:02:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.427+0000 7fdcfcc00640 10 client.25486 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.428+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3301166323 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a82ae953c0 con 0x55a82af38fc0 2024-03-26T19:02:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.428+0000 7fdcfcc00640 10 client.25486 handle_command_reply: tid=0 2024-03-26T19:02:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.428+0000 7fdd1cf94740 1 client.25486 shutdown 2024-03-26T19:02:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.429+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3301166323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af23950 msgr2=0x55a82af2d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.429+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af23950 0x55a82af2d510 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdd0401b470 tx=0x7fdd04004730 comp rx=0 tx=0).stop 2024-03-26T19:02:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.430+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3301166323 shutdown_connections 2024-03-26T19:02:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.430+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a82af38fc0 0x55a82af3b400 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.430+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.430+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af23950 0x55a82af2d510 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.430+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3301166323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af22f60 0x55a82af2cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.430+0000 7fdcfcc00640 0 client.25486 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.430+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3301166323 >> 172.21.15.97:0/3301166323 conn(0x55a82af1e4d0 msgr2=0x55a82af1d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.431+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3301166323 shutdown_connections 2024-03-26T19:02:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.431+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3301166323 wait complete. 2024-03-26T19:02:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.432+0000 7fdd1cf94740 20 client.25486 trim_cache size 0 max 16384 2024-03-26T19:02:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.463+0000 7fdd1cf94740 1 Processor -- start 2024-03-26T19:02:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.463+0000 7fdd1cf94740 1 -- start start 2024-03-26T19:02:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.464+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.464+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 0x55a82af34400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.464+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af24720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.464+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141b7900 con 0x55a82af23950 2024-03-26T19:02:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.464+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141b9440 con 0x55a82af28ac0 2024-03-26T19:02:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.464+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141b7fd0 con 0x55a82af22f60 2024-03-26T19:02:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.464+0000 7fdd13400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.465+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af24720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.465+0000 7fdd13400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af241e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44298/0 (socket says 172.21.15.97:44298) 2024-03-26T19:02:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.465+0000 7fdd13400640 1 -- 172.21.15.97:0/3909646192 learned_addr learned my addr 172.21.15.97:0/3909646192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.465+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af24720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:55626/0 (socket says 172.21.15.97:55626) 2024-03-26T19:02:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.465+0000 7fdd12a00640 1 --2- 172.21.15.97:0/3909646192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 0x55a82af34400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.465+0000 7fdd13e00640 1 -- 172.21.15.97:0/3909646192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 msgr2=0x55a82af241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.465+0000 7fdd13e00640 1 --2- 172.21.15.97:0/3909646192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.465+0000 7fdd13e00640 1 -- 172.21.15.97:0/3909646192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 msgr2=0x55a82af34400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.465+0000 7fdd13e00640 1 --2- 172.21.15.97:0/3909646192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 0x55a82af34400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.465+0000 7fdd13e00640 1 -- 172.21.15.97:0/3909646192 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82af19ae0 con 0x55a82af28ac0 2024-03-26T19:02:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.466+0000 7fdd12a00640 1 --2- 172.21.15.97:0/3909646192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 0x55a82af34400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:02:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.466+0000 7fdd13e00640 1 --2- 172.21.15.97:0/3909646192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af24720 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdd000c5560 tx=0x7fdd0002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bd4f48a64b2e3eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.466+0000 7fdcff400640 1 -- 172.21.15.97:0/3909646192 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0004b040 con 0x55a82af28ac0 2024-03-26T19:02:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.466+0000 7fdcff400640 1 -- 172.21.15.97:0/3909646192 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd000c11d0 con 0x55a82af28ac0 2024-03-26T19:02:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.466+0000 7fdcff400640 1 -- 172.21.15.97:0/3909646192 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd000035f0 con 0x55a82af28ac0 2024-03-26T19:02:16.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:16 smithi105 ceph-mon[36270]: pgmap v490: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 729 op/s 2024-03-26T19:02:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.467+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3909646192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 msgr2=0x55a82af24720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.467+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3909646192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af24720 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdd000c5560 tx=0x7fdd0002bc30 comp rx=0 tx=0).stop 2024-03-26T19:02:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.467+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3909646192 shutdown_connections 2024-03-26T19:02:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.467+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3909646192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af28ac0 0x55a82af24720 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.467+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3909646192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af23950 0x55a82af34400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.467+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3909646192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.467+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3909646192 >> 172.21.15.97:0/3909646192 conn(0x55a82af1e4d0 msgr2=0x55a82af21fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.468+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3909646192 shutdown_connections 2024-03-26T19:02:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.468+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3909646192 wait complete. 2024-03-26T19:02:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.468+0000 7fdd1cf94740 1 -- start start 2024-03-26T19:02:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.468+0000 7fdd1cf94740 1 Processor -- start 2024-03-26T19:02:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.469+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.469+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af23950 0x55a82af32920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.469+0000 7fdd1cf94740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af28ac0 0x55a82af33930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.469+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141b8d70 con 0x55a82af28ac0 2024-03-26T19:02:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.469+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141b86a0 con 0x55a82af23950 2024-03-26T19:02:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.469+0000 7fdd1cf94740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141b58a0 con 0x55a82af22f60 2024-03-26T19:02:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.469+0000 7fdd1cf94740 10 client.0 _ll_register_callbacks cb 0x7fdd04011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:02:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd13400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd13400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af323e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44314/0 (socket says 172.21.15.97:44314) 2024-03-26T19:02:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd12a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af23950 0x55a82af32920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd13400640 1 -- 172.21.15.97:0/3189141177 learned_addr learned my addr 172.21.15.97:0/3189141177 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd13e00640 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af28ac0 0x55a82af33930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd13400640 1 -- 172.21.15.97:0/3189141177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af23950 msgr2=0x55a82af32920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd13400640 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af23950 0x55a82af32920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd13400640 1 -- 172.21.15.97:0/3189141177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af28ac0 msgr2=0x55a82af33930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd13400640 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af28ac0 0x55a82af33930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd13400640 1 -- 172.21.15.97:0/3189141177 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a82af1ef40 con 0x55a82af22f60 2024-03-26T19:02:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.470+0000 7fdd13e00640 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af28ac0 0x55a82af33930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:02:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.471+0000 7fdd13400640 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af323e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009540 tx=0x7fdd0c0059a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1495895078b413c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.471+0000 7fdcfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:02:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.471+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3189141177 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c02d060 con 0x55a82af22f60 2024-03-26T19:02:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.471+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3189141177 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fdd0c028d90 con 0x55a82af22f60 2024-03-26T19:02:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.472+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3189141177 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd000c11d0 con 0x55a82af22f60 2024-03-26T19:02:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.472+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3189141177 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd0c028420 con 0x55a82af22f60 2024-03-26T19:02:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.472+0000 7fdd1cf94740 10 client.25492 fetch_fsmap learned FSMap version 27 2024-03-26T19:02:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.472+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3189141177 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a82af9fae0 con 0x55a82af22f60 2024-03-26T19:02:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.473+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3189141177 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0c003e50 con 0x55a82af22f60 2024-03-26T19:02:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.473+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3189141177 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd0c005440 con 0x55a82af22f60 2024-03-26T19:02:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.474+0000 7fdd1cf94740 10 client.25492 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:02:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.474+0000 7fdd1cf94740 10 client.25492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:02:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.474+0000 7fdd1cf94740 20 client.25492 populate_metadata read hostname 'smithi097' 2024-03-26T19:02:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.474+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a82afa01b0 0x55a82afa25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.474+0000 7fdd1cf94740 4 client.25492 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:02:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.474+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3189141177 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdd000c11d0 con 0x55a82afa01b0 2024-03-26T19:02:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.474+0000 7fdd12a00640 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a82afa01b0 0x55a82afa25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.476+0000 7fdd12a00640 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a82afa01b0 0x55a82afa25a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e927602a8bbc202 server_cookie=aa043d26d0871c36 in_seq=0 out_seq=0 2024-03-26T19:02:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.476+0000 7fdcfcc00640 10 client.25492 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.498+0000 7fdcfcc00640 1 -- 172.21.15.97:0/3189141177 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fdd000c11d0 con 0x55a82afa01b0 2024-03-26T19:02:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.498+0000 7fdcfcc00640 10 client.25492 handle_command_reply: tid=0 2024-03-26T19:02:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.498+0000 7fdd1cf94740 1 client.25492 shutdown 2024-03-26T19:02:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.498+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3189141177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 msgr2=0x55a82af323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.498+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af323e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009540 tx=0x7fdd0c0059a0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.499+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3189141177 shutdown_connections 2024-03-26T19:02:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.499+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a82afa01b0 0x55a82afa25a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.499+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a82af28ac0 0x55a82af33930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.499+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a82af23950 0x55a82af32920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.499+0000 7fdd1cf94740 1 --2- 172.21.15.97:0/3189141177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a82af22f60 0x55a82af323e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.499+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3189141177 >> 172.21.15.97:0/3189141177 conn(0x55a82af1e4d0 msgr2=0x55a82af1ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.499+0000 7fdcfcc00640 0 client.25492 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.499+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3189141177 shutdown_connections 2024-03-26T19:02:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.499+0000 7fdd1cf94740 1 -- 172.21.15.97:0/3189141177 wait complete. 2024-03-26T19:02:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.499+0000 7fdd1cf94740 20 client.25492 trim_cache size 0 max 16384 2024-03-26T19:02:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:02:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:02:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:02:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:02:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.501+0000 7fdd18c00640 1 -- 172.21.15.97:0/1297894014 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcf80a2f90 msgr2=0x7fdcf80a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.501+0000 7fdd18c00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcf80a2f90 0x7fdcf80a5450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdd141a7c30 tx=0x7fdd04013a20 comp rx=0 tx=0).stop 2024-03-26T19:02:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.501+0000 7fdd18c00640 1 -- 172.21.15.97:0/1297894014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141032e0 msgr2=0x7fdd141adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.501+0000 7fdd18c00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141032e0 0x7fdd141adcd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdd0002c7b0 tx=0x7fdd00005dd0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.501+0000 7fdd18c00640 1 -- 172.21.15.97:0/1297894014 shutdown_connections 2024-03-26T19:02:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.501+0000 7fdd18c00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcf80a2f90 0x7fdcf80a5450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.501+0000 7fdd18c00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141032e0 0x7fdd141adcd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.501+0000 7fdd18c00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd14102920 0x7fdd141a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.501+0000 7fdd18c00640 1 --2- 172.21.15.97:0/1297894014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd14101730 0x7fdd141a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.501+0000 7fdd18c00640 1 -- 172.21.15.97:0/1297894014 >> 172.21.15.97:0/1297894014 conn(0x7fdd140fcca0 msgr2=0x7fdd140fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.502+0000 7fdd18c00640 1 -- 172.21.15.97:0/1297894014 shutdown_connections 2024-03-26T19:02:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:16.502+0000 7fdd18c00640 1 -- 172.21.15.97:0/1297894014 wait complete. 2024-03-26T19:02:16.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:16 smithi097 ceph-mon[31955]: pgmap v490: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 729 op/s 2024-03-26T19:02:16.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:16 smithi144 ceph-mon[36053]: pgmap v490: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 729 op/s 2024-03-26T19:02:18.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:18 smithi105 ceph-mon[36270]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 671 op/s 2024-03-26T19:02:18.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:18 smithi097 ceph-mon[31955]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 671 op/s 2024-03-26T19:02:18.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:18 smithi144 ceph-mon[36053]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 671 op/s 2024-03-26T19:02:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 175.0 s, 2450.8 tps, lat 40.668 ms stddev 47.880 2024-03-26T19:02:20.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:20 smithi105 ceph-mon[36270]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.8 MiB/s wr, 774 op/s 2024-03-26T19:02:20.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:20 smithi097 ceph-mon[31955]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.8 MiB/s wr, 774 op/s 2024-03-26T19:02:20.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:20 smithi144 ceph-mon[36053]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.8 MiB/s wr, 774 op/s 2024-03-26T19:02:22.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:22 smithi105 ceph-mon[36270]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 671 op/s 2024-03-26T19:02:22.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:22 smithi097 ceph-mon[31955]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 671 op/s 2024-03-26T19:02:22.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:22 smithi144 ceph-mon[36053]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 671 op/s 2024-03-26T19:02:24.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:24 smithi105 ceph-mon[36270]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 731 op/s 2024-03-26T19:02:24.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:24 smithi097 ceph-mon[31955]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 731 op/s 2024-03-26T19:02:24.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:24 smithi144 ceph-mon[36053]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 731 op/s 2024-03-26T19:02:25.024 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 180.0 s, 2226.8 tps, lat 44.947 ms stddev 55.650 2024-03-26T19:02:26.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:26 smithi105 ceph-mon[36270]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.0 MiB/s wr, 781 op/s 2024-03-26T19:02:26.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:26 smithi097 ceph-mon[31955]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.0 MiB/s wr, 781 op/s 2024-03-26T19:02:26.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:26 smithi144 ceph-mon[36053]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.0 MiB/s wr, 781 op/s 2024-03-26T19:02:28.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:28 smithi105 ceph-mon[36270]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 653 op/s 2024-03-26T19:02:28.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:28 smithi097 ceph-mon[31955]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 653 op/s 2024-03-26T19:02:28.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:28 smithi144 ceph-mon[36053]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 653 op/s 2024-03-26T19:02:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.714+0000 7f4542400640 1 -- 172.21.15.97:0/2152908462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f453c107f20 msgr2=0x7f453c108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.714+0000 7f4542400640 1 --2- 172.21.15.97:0/2152908462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f453c107f20 0x7f453c108340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f452c009900 tx=0x7f452c02bc30 comp rx=0 tx=0).stop 2024-03-26T19:02:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.715+0000 7f4542400640 1 -- 172.21.15.97:0/2152908462 shutdown_connections 2024-03-26T19:02:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.715+0000 7f4542400640 1 --2- 172.21.15.97:0/2152908462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f453c109a30 0x7f453c110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.715+0000 7f4542400640 1 --2- 172.21.15.97:0/2152908462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f453c109070 0x7f453c1094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.715+0000 7f4542400640 1 --2- 172.21.15.97:0/2152908462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f453c107f20 0x7f453c108340 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.715+0000 7f4542400640 1 -- 172.21.15.97:0/2152908462 >> 172.21.15.97:0/2152908462 conn(0x7f453c076ed0 msgr2=0x7f453c0792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.716+0000 7f4542400640 1 -- 172.21.15.97:0/2152908462 shutdown_connections 2024-03-26T19:02:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.716+0000 7f4542400640 1 -- 172.21.15.97:0/2152908462 wait complete. 2024-03-26T19:02:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.718+0000 7f4542400640 1 Processor -- start 2024-03-26T19:02:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.718+0000 7f4542400640 1 -- start start 2024-03-26T19:02:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.719+0000 7f4542400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f453c107f20 0x7f453c19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.719+0000 7f4542400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f453c109070 0x7f453c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.719+0000 7f4542400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f453c109a30 0x7f453c1a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.719+0000 7f4542400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f453c113c00 con 0x7f453c109a30 2024-03-26T19:02:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.719+0000 7f4542400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f453c113a80 con 0x7f453c107f20 2024-03-26T19:02:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.719+0000 7f4542400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f453c113d80 con 0x7f453c109070 2024-03-26T19:02:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.719+0000 7f453b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f453c107f20 0x7f453c19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.720+0000 7f453aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f453c109070 0x7f453c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.720+0000 7f453aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f453c109070 0x7f453c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50062/0 (socket says 172.21.15.97:50062) 2024-03-26T19:02:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.720+0000 7f453aa00640 1 -- 172.21.15.97:0/1614571880 learned_addr learned my addr 172.21.15.97:0/1614571880 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.720+0000 7f453be00640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f453c109a30 0x7f453c1a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.720+0000 7f453aa00640 1 -- 172.21.15.97:0/1614571880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f453c107f20 msgr2=0x7f453c19e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.720+0000 7f453aa00640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f453c107f20 0x7f453c19e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.720+0000 7f453aa00640 1 -- 172.21.15.97:0/1614571880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f453c109a30 msgr2=0x7f453c1a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.720+0000 7f453aa00640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f453c109a30 0x7f453c1a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.720+0000 7f453aa00640 1 -- 172.21.15.97:0/1614571880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453c1a5d70 con 0x7f453c109070 2024-03-26T19:02:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.721+0000 7f453aa00640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f453c109070 0x7f453c19e690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f452800d770 tx=0x7f452800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.721+0000 7f4533e00640 1 -- 172.21.15.97:0/1614571880 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528017070 con 0x7f453c109070 2024-03-26T19:02:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.721+0000 7f4533e00640 1 -- 172.21.15.97:0/1614571880 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4528009c30 con 0x7f453c109070 2024-03-26T19:02:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.722+0000 7f4533e00640 1 -- 172.21.15.97:0/1614571880 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528003dd0 con 0x7f453c109070 2024-03-26T19:02:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.722+0000 7f4542400640 1 -- 172.21.15.97:0/1614571880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453c1a6060 con 0x7f453c109070 2024-03-26T19:02:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.722+0000 7f4542400640 1 -- 172.21.15.97:0/1614571880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453c1a6560 con 0x7f453c109070 2024-03-26T19:02:29.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.729+0000 7f4533e00640 1 -- 172.21.15.97:0/1614571880 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f4528003f70 con 0x7f453c109070 2024-03-26T19:02:29.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.731+0000 7f45451ba740 1 Processor -- start 2024-03-26T19:02:29.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.731+0000 7f45451ba740 1 -- start start 2024-03-26T19:02:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.732+0000 7f45451ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a888c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.732+0000 7f45451ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888c790 0x5568a8891290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.732+0000 7f45451ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a8891900 0x5568a8893d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.732+0000 7f45451ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5568a87f0e80 con 0x5568a8891900 2024-03-26T19:02:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.732+0000 7f45451ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5568a8799660 con 0x5568a888c790 2024-03-26T19:02:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.732+0000 7f45451ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5568a87ece30 con 0x5568a888bda0 2024-03-26T19:02:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.732+0000 7f453b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888c790 0x5568a8891290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.733+0000 7f453be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a888c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.733+0000 7f453aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a8891900 0x5568a8893d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.733+0000 7f453b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888c790 0x5568a8891290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44348/0 (socket says 172.21.15.97:44348) 2024-03-26T19:02:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.733+0000 7f453b400640 1 -- 172.21.15.97:0/3032926515 learned_addr learned my addr 172.21.15.97:0/3032926515 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.733+0000 7f453b400640 1 -- 172.21.15.97:0/3032926515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 msgr2=0x5568a888c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.733+0000 7f453b400640 1 --2- 172.21.15.97:0/3032926515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a888c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.733+0000 7f453b400640 1 -- 172.21.15.97:0/3032926515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a8891900 msgr2=0x5568a8893d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.733+0000 7f453b400640 1 --2- 172.21.15.97:0/3032926515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a8891900 0x5568a8893d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.733+0000 7f4533e00640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45200a3040 0x7f45200a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.733+0000 7f453b400640 1 -- 172.21.15.97:0/3032926515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a8505210 con 0x5568a888c790 2024-03-26T19:02:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.734+0000 7f4533e00640 1 -- 172.21.15.97:0/1614571880 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f45280ced20 con 0x7f453c109070 2024-03-26T19:02:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.734+0000 7f453b400640 1 --2- 172.21.15.97:0/3032926515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888c790 0x5568a8891290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f452c000c00 tx=0x7f452c0095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7391abf3731b0dad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.734+0000 7f453be00640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45200a3040 0x7f45200a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.734+0000 7f4530c00640 1 -- 172.21.15.97:0/3032926515 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c045070 con 0x5568a888c790 2024-03-26T19:02:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.734+0000 7f4530c00640 1 -- 172.21.15.97:0/3032926515 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f452c004040 con 0x5568a888c790 2024-03-26T19:02:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.734+0000 7f4530c00640 1 -- 172.21.15.97:0/3032926515 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c004eb0 con 0x5568a888c790 2024-03-26T19:02:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.735+0000 7f45451ba740 1 -- 172.21.15.97:0/3032926515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888c790 msgr2=0x5568a8891290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.735+0000 7f45451ba740 1 --2- 172.21.15.97:0/3032926515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888c790 0x5568a8891290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f452c000c00 tx=0x7f452c0095e0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.736+0000 7f45451ba740 1 -- 172.21.15.97:0/3032926515 shutdown_connections 2024-03-26T19:02:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.736+0000 7f45451ba740 1 --2- 172.21.15.97:0/3032926515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a8891900 0x5568a8893d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.736+0000 7f45451ba740 1 --2- 172.21.15.97:0/3032926515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888c790 0x5568a8891290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.736+0000 7f45451ba740 1 --2- 172.21.15.97:0/3032926515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a888c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.736+0000 7f45451ba740 1 -- 172.21.15.97:0/3032926515 >> 172.21.15.97:0/3032926515 conn(0x5568a8887310 msgr2=0x5568a8887710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.737+0000 7f453be00640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45200a3040 0x7f45200a5500 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5568a888cff0 tx=0x7f4534009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.737+0000 7f45451ba740 1 -- 172.21.15.97:0/3032926515 shutdown_connections 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.737+0000 7f45451ba740 1 -- 172.21.15.97:0/3032926515 wait complete. 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.738+0000 7f45451ba740 1 -- start start 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.738+0000 7f45451ba740 1 Processor -- start 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.739+0000 7f45451ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a8895d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.739+0000 7f45451ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 0x5568a88962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.739+0000 7f45451ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.739+0000 7f45451ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5568a8894850 con 0x5568a888c790 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.739+0000 7f45451ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5568a87acac0 con 0x5568a8891900 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.739+0000 7f45451ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5568a88949d0 con 0x5568a888bda0 2024-03-26T19:02:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.739+0000 7f45451ba740 10 client.0 _ll_register_callbacks cb 0x5568a87ac490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 0x5568a88962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44360/0 (socket says 172.21.15.97:44360) 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453be00640 1 -- 172.21.15.97:0/2681725478 learned_addr learned my addr 172.21.15.97:0/2681725478 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453b400640 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a8895d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453be00640 1 -- 172.21.15.97:0/2681725478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 msgr2=0x5568a8895d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453be00640 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a8895d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453be00640 1 -- 172.21.15.97:0/2681725478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 msgr2=0x5568a88962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453be00640 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 0x5568a88962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453be00640 1 -- 172.21.15.97:0/2681725478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a85d1620 con 0x5568a8891900 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453aa00640 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 0x5568a88962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:02:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.740+0000 7f453be00640 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f45340127b0 tx=0x7f4534004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95d694701e32290 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.741+0000 7f4525600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.741+0000 7f4525600640 1 -- 172.21.15.97:0/2681725478 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453400fb50 con 0x5568a8891900 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.741+0000 7f4525600640 1 -- 172.21.15.97:0/2681725478 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f453400fcf0 con 0x5568a8891900 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.741+0000 7f45451ba740 1 -- 172.21.15.97:0/2681725478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f452c004040 con 0x5568a8891900 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.741+0000 7f4525600640 1 -- 172.21.15.97:0/2681725478 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f453402e070 con 0x5568a8891900 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.741+0000 7f45451ba740 10 client.25547 fetch_fsmap learned FSMap version 27 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.741+0000 7f45451ba740 1 -- 172.21.15.97:0/2681725478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568a88a06e0 con 0x5568a8891900 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.742+0000 7f4525600640 1 -- 172.21.15.97:0/2681725478 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f452c004040 con 0x5568a8891900 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.742+0000 7f4525600640 1 -- 172.21.15.97:0/2681725478 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f453401d4c0 con 0x5568a8891900 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.742+0000 7f45451ba740 10 client.25547 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.742+0000 7f45451ba740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.742+0000 7f45451ba740 20 client.25547 populate_metadata read hostname 'smithi097' 2024-03-26T19:02:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.742+0000 7f45451ba740 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5568a88a1e20 0x5568a88a4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.742+0000 7f45451ba740 4 client.25547 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:02:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.742+0000 7f45451ba740 1 -- 172.21.15.97:0/2681725478 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568a87fe3c0 con 0x5568a88a1e20 2024-03-26T19:02:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.742+0000 7f453b400640 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5568a88a1e20 0x5568a88a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.743+0000 7f453b400640 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5568a88a1e20 0x5568a88a4210 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91e155c67a453e8 server_cookie=a0890c878909a787 in_seq=0 out_seq=0 2024-03-26T19:02:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.743+0000 7f4525600640 10 client.25547 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.744+0000 7f4525600640 1 -- 172.21.15.97:0/2681725478 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5568a87fe3c0 con 0x5568a88a1e20 2024-03-26T19:02:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.744+0000 7f4525600640 10 client.25547 handle_command_reply: tid=0 2024-03-26T19:02:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.744+0000 7f45451ba740 1 client.25547 shutdown 2024-03-26T19:02:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.745+0000 7f45451ba740 1 -- 172.21.15.97:0/2681725478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 msgr2=0x5568a889d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.745+0000 7f45451ba740 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f45340127b0 tx=0x7f4534004a40 comp rx=0 tx=0).stop 2024-03-26T19:02:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.745+0000 7f45451ba740 1 -- 172.21.15.97:0/2681725478 shutdown_connections 2024-03-26T19:02:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.745+0000 7f45451ba740 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5568a88a1e20 0x5568a88a4210 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.746+0000 7f45451ba740 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889d300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.746+0000 7f4525600640 0 client.25547 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.746+0000 7f45451ba740 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 0x5568a88962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.746+0000 7f45451ba740 1 --2- 172.21.15.97:0/2681725478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a8895d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.746+0000 7f45451ba740 1 -- 172.21.15.97:0/2681725478 >> 172.21.15.97:0/2681725478 conn(0x5568a8887310 msgr2=0x5568a8886540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.746+0000 7f45451ba740 1 -- 172.21.15.97:0/2681725478 shutdown_connections 2024-03-26T19:02:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.746+0000 7f45451ba740 1 -- 172.21.15.97:0/2681725478 wait complete. 2024-03-26T19:02:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.747+0000 7f45451ba740 20 client.25547 trim_cache size 0 max 16384 2024-03-26T19:02:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.780+0000 7f45451ba740 1 Processor -- start 2024-03-26T19:02:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.780+0000 7f45451ba740 1 -- start start 2024-03-26T19:02:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.780+0000 7f45451ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888bda0 0x5568a888d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.780+0000 7f45451ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888c790 0x5568a889d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.780+0000 7f45451ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a8891900 0x5568a88a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.781+0000 7f45451ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f453c1af320 con 0x5568a8891900 2024-03-26T19:02:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.781+0000 7f45451ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f453c1b0e60 con 0x5568a888bda0 2024-03-26T19:02:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.781+0000 7f45451ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f453c1af9f0 con 0x5568a888c790 2024-03-26T19:02:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.781+0000 7f453b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888bda0 0x5568a888d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.782+0000 7f453b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888bda0 0x5568a888d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44368/0 (socket says 172.21.15.97:44368) 2024-03-26T19:02:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.782+0000 7f453b400640 1 -- 172.21.15.97:0/3239993753 learned_addr learned my addr 172.21.15.97:0/3239993753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.781+0000 7f453aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888c790 0x5568a889d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.782+0000 7f453be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a8891900 0x5568a88a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.782+0000 7f453aa00640 1 -- 172.21.15.97:0/3239993753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888bda0 msgr2=0x5568a888d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.782+0000 7f453aa00640 1 --2- 172.21.15.97:0/3239993753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888bda0 0x5568a888d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.782+0000 7f453aa00640 1 -- 172.21.15.97:0/3239993753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a8891900 msgr2=0x5568a88a8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.782+0000 7f453aa00640 1 --2- 172.21.15.97:0/3239993753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a8891900 0x5568a88a8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.782+0000 7f453aa00640 1 -- 172.21.15.97:0/3239993753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a88a4580 con 0x5568a888c790 2024-03-26T19:02:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.783+0000 7f453b400640 1 --2- 172.21.15.97:0/3239993753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888bda0 0x5568a888d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:02:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.783+0000 7f453aa00640 1 --2- 172.21.15.97:0/3239993753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888c790 0x5568a889d2c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4528009610 tx=0x7f4528000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70019255ae57c859 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.783+0000 7f4527e00640 1 -- 172.21.15.97:0/3239993753 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528020040 con 0x5568a888c790 2024-03-26T19:02:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.783+0000 7f4527e00640 1 -- 172.21.15.97:0/3239993753 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4528009de0 con 0x5568a888c790 2024-03-26T19:02:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.783+0000 7f4527e00640 1 -- 172.21.15.97:0/3239993753 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528097a90 con 0x5568a888c790 2024-03-26T19:02:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.784+0000 7f45451ba740 1 -- 172.21.15.97:0/3239993753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888c790 msgr2=0x5568a889d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.784+0000 7f45451ba740 1 --2- 172.21.15.97:0/3239993753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888c790 0x5568a889d2c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4528009610 tx=0x7f4528000bc0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.785+0000 7f45451ba740 1 -- 172.21.15.97:0/3239993753 shutdown_connections 2024-03-26T19:02:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.785+0000 7f45451ba740 1 --2- 172.21.15.97:0/3239993753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a8891900 0x5568a88a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.785+0000 7f45451ba740 1 --2- 172.21.15.97:0/3239993753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888c790 0x5568a889d2c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.785+0000 7f45451ba740 1 --2- 172.21.15.97:0/3239993753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a888bda0 0x5568a888d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.785+0000 7f45451ba740 1 -- 172.21.15.97:0/3239993753 >> 172.21.15.97:0/3239993753 conn(0x5568a8887310 msgr2=0x5568a888d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.785+0000 7f45451ba740 1 -- 172.21.15.97:0/3239993753 shutdown_connections 2024-03-26T19:02:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.785+0000 7f45451ba740 1 -- 172.21.15.97:0/3239993753 wait complete. 2024-03-26T19:02:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.786+0000 7f45451ba740 1 -- start start 2024-03-26T19:02:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.786+0000 7f45451ba740 1 Processor -- start 2024-03-26T19:02:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.787+0000 7f45451ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a889aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.787+0000 7f45451ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 0x5568a889b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.787+0000 7f45451ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.787+0000 7f45451ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f453c1b0790 con 0x5568a888c790 2024-03-26T19:02:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.787+0000 7f45451ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f453c1b00c0 con 0x5568a8891900 2024-03-26T19:02:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.787+0000 7f45451ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f453c1ad300 con 0x5568a888bda0 2024-03-26T19:02:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.787+0000 7f45451ba740 10 client.0 _ll_register_callbacks cb 0x7f4534012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:02:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.788+0000 7f453be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.788+0000 7f453be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44372/0 (socket says 172.21.15.97:44372) 2024-03-26T19:02:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.788+0000 7f453be00640 1 -- 172.21.15.97:0/2426069784 learned_addr learned my addr 172.21.15.97:0/2426069784 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.788+0000 7f453b400640 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a889aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.788+0000 7f453aa00640 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 0x5568a889b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.788+0000 7f453be00640 1 -- 172.21.15.97:0/2426069784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 msgr2=0x5568a889aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.788+0000 7f453be00640 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a889aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.788+0000 7f453be00640 1 -- 172.21.15.97:0/2426069784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 msgr2=0x5568a889b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.788+0000 7f453be00640 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 0x5568a889b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.788+0000 7f453be00640 1 -- 172.21.15.97:0/2426069784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a889ca20 con 0x5568a8891900 2024-03-26T19:02:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.789+0000 7f453aa00640 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 0x5568a889b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:02:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.790+0000 7f453be00640 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889c430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4534000f80 tx=0x7f453401dc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108432f6ec4b72fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.790+0000 7f4525600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:02:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.791+0000 7f4525600640 1 -- 172.21.15.97:0/2426069784 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453401d5b0 con 0x5568a8891900 2024-03-26T19:02:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.791+0000 7f4525600640 1 -- 172.21.15.97:0/2426069784 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f453400f850 con 0x5568a8891900 2024-03-26T19:02:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.791+0000 7f45451ba740 1 -- 172.21.15.97:0/2426069784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4528009de0 con 0x5568a8891900 2024-03-26T19:02:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.792+0000 7f4525600640 1 -- 172.21.15.97:0/2426069784 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f453400fa50 con 0x5568a8891900 2024-03-26T19:02:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.792+0000 7f45451ba740 10 client.25553 fetch_fsmap learned FSMap version 27 2024-03-26T19:02:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.792+0000 7f45451ba740 1 -- 172.21.15.97:0/2426069784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568a8919650 con 0x5568a8891900 2024-03-26T19:02:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.793+0000 7f4525600640 1 -- 172.21.15.97:0/2426069784 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4528009de0 con 0x5568a8891900 2024-03-26T19:02:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.794+0000 7f4525600640 1 -- 172.21.15.97:0/2426069784 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f453403c070 con 0x5568a8891900 2024-03-26T19:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.794+0000 7f45451ba740 10 client.25553 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.794+0000 7f45451ba740 10 client.25553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.794+0000 7f45451ba740 20 client.25553 populate_metadata read hostname 'smithi097' 2024-03-26T19:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.794+0000 7f45451ba740 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5568a88a1e20 0x5568a891bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.794+0000 7f45451ba740 4 client.25553 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.794+0000 7f45451ba740 1 -- 172.21.15.97:0/2426069784 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568a87f8710 con 0x5568a88a1e20 2024-03-26T19:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.794+0000 7f453b400640 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5568a88a1e20 0x5568a891bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.795+0000 7f4525600640 10 client.25553 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.796+0000 7f453b400640 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5568a88a1e20 0x5568a891bf80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b061a2da7291a4c6 server_cookie=7cac09f012d549b in_seq=0 out_seq=0 2024-03-26T19:02:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f4525600640 1 -- 172.21.15.97:0/2426069784 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568a87f8710 con 0x5568a88a1e20 2024-03-26T19:02:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f4525600640 10 client.25553 handle_command_reply: tid=0 2024-03-26T19:02:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f45451ba740 1 client.25553 shutdown 2024-03-26T19:02:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f45451ba740 1 -- 172.21.15.97:0/2426069784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 msgr2=0x5568a889c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f45451ba740 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889c430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4534000f80 tx=0x7f453401dc20 comp rx=0 tx=0).stop 2024-03-26T19:02:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f45451ba740 1 -- 172.21.15.97:0/2426069784 shutdown_connections 2024-03-26T19:02:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f45451ba740 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5568a88a1e20 0x5568a891bf80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f45451ba740 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568a8891900 0x5568a889c430 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f45451ba740 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568a888c790 0x5568a889b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f45451ba740 1 --2- 172.21.15.97:0/2426069784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5568a888bda0 0x5568a889aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f45451ba740 1 -- 172.21.15.97:0/2426069784 >> 172.21.15.97:0/2426069784 conn(0x5568a8887310 msgr2=0x5568a888dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f4525600640 0 client.25553 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.822+0000 7f45451ba740 1 -- 172.21.15.97:0/2426069784 shutdown_connections 2024-03-26T19:02:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.823+0000 7f45451ba740 1 -- 172.21.15.97:0/2426069784 wait complete. 2024-03-26T19:02:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.823+0000 7f45451ba740 20 client.25553 trim_cache size 0 max 16384 2024-03-26T19:02:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:02:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:02:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:02:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.824+0000 7f4542400640 1 -- 172.21.15.97:0/1614571880 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45200a3040 msgr2=0x7f45200a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.824+0000 7f4542400640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45200a3040 0x7f45200a5500 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5568a888cff0 tx=0x7f4534009500 comp rx=0 tx=0).stop 2024-03-26T19:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.824+0000 7f4542400640 1 -- 172.21.15.97:0/1614571880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f453c109070 msgr2=0x7f453c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.824+0000 7f4542400640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f453c109070 0x7f453c19e690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f452800d770 tx=0x7f452800dc30 comp rx=0 tx=0).stop 2024-03-26T19:02:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.824+0000 7f4542400640 1 -- 172.21.15.97:0/1614571880 shutdown_connections 2024-03-26T19:02:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.824+0000 7f4542400640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f45200a3040 0x7f45200a5500 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.824+0000 7f4542400640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f453c109a30 0x7f453c1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.824+0000 7f4542400640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f453c109070 0x7f453c19e690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.824+0000 7f4542400640 1 --2- 172.21.15.97:0/1614571880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f453c107f20 0x7f453c19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.824+0000 7f4542400640 1 -- 172.21.15.97:0/1614571880 >> 172.21.15.97:0/1614571880 conn(0x7f453c076ed0 msgr2=0x7f453c10c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.825+0000 7f4542400640 1 -- 172.21.15.97:0/1614571880 shutdown_connections 2024-03-26T19:02:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:29.825+0000 7f4542400640 1 -- 172.21.15.97:0/1614571880 wait complete. 2024-03-26T19:02:30.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 185.0 s, 2208.0 tps, lat 45.259 ms stddev 48.592 2024-03-26T19:02:30.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:30 smithi097 ceph-mon[31955]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 731 op/s 2024-03-26T19:02:30.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:30 smithi144 ceph-mon[36053]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 731 op/s 2024-03-26T19:02:30.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:30 smithi105 ceph-mon[36270]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 731 op/s 2024-03-26T19:02:32.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:32 smithi097 ceph-mon[31955]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 677 op/s 2024-03-26T19:02:32.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:32 smithi144 ceph-mon[36053]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 677 op/s 2024-03-26T19:02:32.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:32 smithi105 ceph-mon[36270]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 677 op/s 2024-03-26T19:02:34.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:34 smithi097 ceph-mon[31955]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 639 op/s 2024-03-26T19:02:34.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:34 smithi144 ceph-mon[36053]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 639 op/s 2024-03-26T19:02:34.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:34 smithi105 ceph-mon[36270]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 639 op/s 2024-03-26T19:02:35.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 190.0 s, 2787.5 tps, lat 35.868 ms stddev 40.570 2024-03-26T19:02:36.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:36 smithi097 ceph-mon[31955]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.8 MiB/s wr, 709 op/s 2024-03-26T19:02:36.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:36 smithi144 ceph-mon[36053]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.8 MiB/s wr, 709 op/s 2024-03-26T19:02:36.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:36 smithi105 ceph-mon[36270]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.8 MiB/s wr, 709 op/s 2024-03-26T19:02:38.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:38 smithi097 ceph-mon[31955]: pgmap v501: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 646 op/s 2024-03-26T19:02:38.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:38 smithi144 ceph-mon[36053]: pgmap v501: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 646 op/s 2024-03-26T19:02:38.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:38 smithi105 ceph-mon[36270]: pgmap v501: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 646 op/s 2024-03-26T19:02:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 195.0 s, 2170.8 tps, lat 46.043 ms stddev 50.195 2024-03-26T19:02:40.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:40 smithi097 ceph-mon[31955]: pgmap v502: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 751 op/s 2024-03-26T19:02:40.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:40 smithi144 ceph-mon[36053]: pgmap v502: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 751 op/s 2024-03-26T19:02:40.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:40 smithi105 ceph-mon[36270]: pgmap v502: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 751 op/s 2024-03-26T19:02:42.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:42 smithi097 ceph-mon[31955]: pgmap v503: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 697 op/s 2024-03-26T19:02:42.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:42 smithi144 ceph-mon[36053]: pgmap v503: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 697 op/s 2024-03-26T19:02:42.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:42 smithi105 ceph-mon[36270]: pgmap v503: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 697 op/s 2024-03-26T19:02:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.030+0000 7f7a52400640 1 -- 172.21.15.97:0/951934348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c1034a0 msgr2=0x7f7a4c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.030+0000 7f7a52400640 1 --2- 172.21.15.97:0/951934348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c1034a0 0x7f7a4c109d30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009be0 tx=0x7f7a4402bef0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.031+0000 7f7a52400640 1 -- 172.21.15.97:0/951934348 shutdown_connections 2024-03-26T19:02:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.031+0000 7f7a52400640 1 --2- 172.21.15.97:0/951934348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c1034a0 0x7f7a4c109d30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.031+0000 7f7a52400640 1 --2- 172.21.15.97:0/951934348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a4c102ae0 0x7f7a4c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.031+0000 7f7a52400640 1 --2- 172.21.15.97:0/951934348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4c1018f0 0x7f7a4c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.031+0000 7f7a52400640 1 -- 172.21.15.97:0/951934348 >> 172.21.15.97:0/951934348 conn(0x7f7a4c0fce80 msgr2=0x7f7a4c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.031+0000 7f7a52400640 1 -- 172.21.15.97:0/951934348 shutdown_connections 2024-03-26T19:02:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.031+0000 7f7a52400640 1 -- 172.21.15.97:0/951934348 wait complete. 2024-03-26T19:02:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.032+0000 7f7a52400640 1 Processor -- start 2024-03-26T19:02:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.032+0000 7f7a52400640 1 -- start start 2024-03-26T19:02:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.033+0000 7f7a52400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c1018f0 0x7f7a4c19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.033+0000 7f7a52400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a4c102ae0 0x7f7a4c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.033+0000 7f7a52400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4c1034a0 0x7f7a4c1a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.033+0000 7f7a52400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c075850 con 0x7f7a4c1034a0 2024-03-26T19:02:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.033+0000 7f7a52400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c0756d0 con 0x7f7a4c102ae0 2024-03-26T19:02:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.033+0000 7f7a52400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c0759d0 con 0x7f7a4c1018f0 2024-03-26T19:02:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.034+0000 7f7a4b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c1018f0 0x7f7a4c19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.034+0000 7f7a4b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c1018f0 0x7f7a4c19e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44208/0 (socket says 172.21.15.97:44208) 2024-03-26T19:02:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.034+0000 7f7a4b400640 1 -- 172.21.15.97:0/2470861137 learned_addr learned my addr 172.21.15.97:0/2470861137 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.034+0000 7f7a4aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a4c102ae0 0x7f7a4c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.035+0000 7f7a4b400640 1 -- 172.21.15.97:0/2470861137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a4c102ae0 msgr2=0x7f7a4c19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.035+0000 7f7a4be00640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4c1034a0 0x7f7a4c1a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.035+0000 7f7a4b400640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a4c102ae0 0x7f7a4c19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.035+0000 7f7a4b400640 1 -- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4c1034a0 msgr2=0x7f7a4c1a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.035+0000 7f7a4b400640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4c1034a0 0x7f7a4c1a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.035+0000 7f7a4b400640 1 -- 172.21.15.97:0/2470861137 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c1a5ce0 con 0x7f7a4c1018f0 2024-03-26T19:02:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.036+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a4c102ae0 0x7f7a4c19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:02:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.036+0000 7f7a4be00640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4c1034a0 0x7f7a4c1a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:02:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.036+0000 7f7a4b400640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c1018f0 0x7f7a4c19e0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00d770 tx=0x7f7a3c00db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.037+0000 7f7a43e00640 1 -- 172.21.15.97:0/2470861137 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c017070 con 0x7f7a4c1018f0 2024-03-26T19:02:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.037+0000 7f7a43e00640 1 -- 172.21.15.97:0/2470861137 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3c009d00 con 0x7f7a4c1018f0 2024-03-26T19:02:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.037+0000 7f7a43e00640 1 -- 172.21.15.97:0/2470861137 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c003cc0 con 0x7f7a4c1018f0 2024-03-26T19:02:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.037+0000 7f7a52400640 1 -- 172.21.15.97:0/2470861137 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c1a5fd0 con 0x7f7a4c1018f0 2024-03-26T19:02:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.037+0000 7f7a52400640 1 -- 172.21.15.97:0/2470861137 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c1a6380 con 0x7f7a4c1018f0 2024-03-26T19:02:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.042+0000 7f7a43e00640 1 -- 172.21.15.97:0/2470861137 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f7a3c009820 con 0x7f7a4c1018f0 2024-03-26T19:02:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.042+0000 7f7a55030740 1 Processor -- start 2024-03-26T19:02:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.042+0000 7f7a55030740 1 -- start start 2024-03-26T19:02:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.043+0000 7f7a55030740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c53da0 0x559028c541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.043+0000 7f7a55030740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c59290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.043+0000 7f7a55030740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c59900 0x559028c5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.043+0000 7f7a55030740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559028bb8e80 con 0x559028c59900 2024-03-26T19:02:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.043+0000 7f7a55030740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559028b61660 con 0x559028c53da0 2024-03-26T19:02:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.043+0000 7f7a55030740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559028bb4e30 con 0x559028c54790 2024-03-26T19:02:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.044+0000 7f7a4b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c59290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.044+0000 7f7a4b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c59290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44216/0 (socket says 172.21.15.97:44216) 2024-03-26T19:02:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.044+0000 7f7a4b400640 1 -- 172.21.15.97:0/1206239415 learned_addr learned my addr 172.21.15.97:0/1206239415 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.044+0000 7f7a4be00640 1 --2- 172.21.15.97:0/1206239415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c53da0 0x559028c541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.044+0000 7f7a43e00640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7a300a3110 0x7f7a300a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.044+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/1206239415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c59900 0x559028c5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.045+0000 7f7a43e00640 1 -- 172.21.15.97:0/2470861137 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7a3c0ced10 con 0x7f7a4c1018f0 2024-03-26T19:02:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.046+0000 7f7a4aa00640 1 -- 172.21.15.97:0/1206239415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 msgr2=0x559028c59290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.046+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/1206239415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c59290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.046+0000 7f7a4aa00640 1 -- 172.21.15.97:0/1206239415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c53da0 msgr2=0x559028c541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.046+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/1206239415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c53da0 0x559028c541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.046+0000 7f7a4aa00640 1 -- 172.21.15.97:0/1206239415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590288cd210 con 0x559028c59900 2024-03-26T19:02:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.046+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/1206239415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c59900 0x559028c5bd90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a38005fd0 tx=0x7f7a38011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6a9454ecb71eca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.047+0000 7f7a4be00640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7a300a3110 0x7f7a300a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.048+0000 7f7a40c00640 1 -- 172.21.15.97:0/1206239415 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3801e070 con 0x559028c59900 2024-03-26T19:02:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.048+0000 7f7a40c00640 1 -- 172.21.15.97:0/1206239415 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a380035f0 con 0x559028c59900 2024-03-26T19:02:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.048+0000 7f7a40c00640 1 -- 172.21.15.97:0/1206239415 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800f860 con 0x559028c59900 2024-03-26T19:02:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.049+0000 7f7a55030740 1 -- 172.21.15.97:0/1206239415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c59900 msgr2=0x559028c5bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.049+0000 7f7a4be00640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7a300a3110 0x7f7a300a55d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a44034fd0 tx=0x7f7a4402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.049+0000 7f7a55030740 1 --2- 172.21.15.97:0/1206239415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c59900 0x559028c5bd90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a38005fd0 tx=0x7f7a38011950 comp rx=0 tx=0).stop 2024-03-26T19:02:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.049+0000 7f7a55030740 1 -- 172.21.15.97:0/1206239415 shutdown_connections 2024-03-26T19:02:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.049+0000 7f7a55030740 1 --2- 172.21.15.97:0/1206239415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c59900 0x559028c5bd90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.049+0000 7f7a55030740 1 --2- 172.21.15.97:0/1206239415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c59290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.049+0000 7f7a55030740 1 --2- 172.21.15.97:0/1206239415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c53da0 0x559028c541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.049+0000 7f7a55030740 1 -- 172.21.15.97:0/1206239415 >> 172.21.15.97:0/1206239415 conn(0x559028c4f310 msgr2=0x559028c4f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.049+0000 7f7a55030740 1 -- 172.21.15.97:0/1206239415 shutdown_connections 2024-03-26T19:02:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.049+0000 7f7a55030740 1 -- 172.21.15.97:0/1206239415 wait complete. 2024-03-26T19:02:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.050+0000 7f7a55030740 1 -- start start 2024-03-26T19:02:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.050+0000 7f7a55030740 1 Processor -- start 2024-03-26T19:02:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.051+0000 7f7a55030740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 0x559028c6ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.051+0000 7f7a55030740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c6f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.051+0000 7f7a55030740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a55030740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559028c6d8c0 con 0x559028c53da0 2024-03-26T19:02:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a55030740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559028b74ac0 con 0x559028c59900 2024-03-26T19:02:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a55030740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559028c6da40 con 0x559028c54790 2024-03-26T19:02:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a55030740 10 client.0 _ll_register_callbacks cb 0x559028b74490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:02:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a4aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c6f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a4aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c6f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44224/0 (socket says 172.21.15.97:44224) 2024-03-26T19:02:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a4aa00640 1 -- 172.21.15.97:0/2800775559 learned_addr learned my addr 172.21.15.97:0/2800775559 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a4b400640 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 0x559028c6ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a4be00640 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a4aa00640 1 -- 172.21.15.97:0/2800775559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 msgr2=0x559028c763c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.052+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c763c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.053+0000 7f7a4aa00640 1 -- 172.21.15.97:0/2800775559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 msgr2=0x559028c6ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.053+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 0x559028c6ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.053+0000 7f7a4aa00640 1 -- 172.21.15.97:0/2800775559 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559028999620 con 0x559028c54790 2024-03-26T19:02:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.053+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c6f380 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a38000c00 tx=0x7f7a38004230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=718c5637a2b10f42 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.053+0000 7f7a35600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:02:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.053+0000 7f7a35600640 1 -- 172.21.15.97:0/2800775559 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3801e040 con 0x559028c54790 2024-03-26T19:02:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.053+0000 7f7a35600640 1 -- 172.21.15.97:0/2800775559 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7a38004320 con 0x559028c54790 2024-03-26T19:02:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.053+0000 7f7a55030740 1 -- 172.21.15.97:0/2800775559 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a380035f0 con 0x559028c54790 2024-03-26T19:02:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.054+0000 7f7a35600640 1 -- 172.21.15.97:0/2800775559 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a38005040 con 0x559028c54790 2024-03-26T19:02:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.054+0000 7f7a55030740 10 client.25522 fetch_fsmap learned FSMap version 27 2024-03-26T19:02:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.054+0000 7f7a55030740 1 -- 172.21.15.97:0/2800775559 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559028c79730 con 0x559028c54790 2024-03-26T19:02:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.055+0000 7f7a35600640 1 -- 172.21.15.97:0/2800775559 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a380035f0 con 0x559028c54790 2024-03-26T19:02:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.055+0000 7f7a35600640 1 -- 172.21.15.97:0/2800775559 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7a3800f600 con 0x559028c54790 2024-03-26T19:02:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.055+0000 7f7a55030740 10 client.25522 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:02:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.055+0000 7f7a55030740 10 client.25522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:02:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.055+0000 7f7a55030740 20 client.25522 populate_metadata read hostname 'smithi097' 2024-03-26T19:02:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.055+0000 7f7a55030740 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559028c7aae0 0x559028c7cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.055+0000 7f7a55030740 4 client.25522 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:02:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.055+0000 7f7a55030740 1 -- 172.21.15.97:0/2800775559 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559028bc63c0 con 0x559028c7aae0 2024-03-26T19:02:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.056+0000 7f7a4b400640 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559028c7aae0 0x559028c7cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.057+0000 7f7a4b400640 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559028c7aae0 0x559028c7cf80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ff4e29a2cf3d6a server_cookie=1c30300f4e86318c in_seq=0 out_seq=0 2024-03-26T19:02:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.057+0000 7f7a35600640 10 client.25522 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.058+0000 7f7a35600640 1 -- 172.21.15.97:0/2800775559 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559028bc63c0 con 0x559028c7aae0 2024-03-26T19:02:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.058+0000 7f7a35600640 10 client.25522 handle_command_reply: tid=0 2024-03-26T19:02:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.058+0000 7f7a55030740 1 client.25522 shutdown 2024-03-26T19:02:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.059+0000 7f7a55030740 1 -- 172.21.15.97:0/2800775559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 msgr2=0x559028c6f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.059+0000 7f7a55030740 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c6f380 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a38000c00 tx=0x7f7a38004230 comp rx=0 tx=0).stop 2024-03-26T19:02:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.060+0000 7f7a55030740 1 -- 172.21.15.97:0/2800775559 shutdown_connections 2024-03-26T19:02:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.060+0000 7f7a55030740 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559028c7aae0 0x559028c7cf80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.060+0000 7f7a55030740 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c763c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.060+0000 7f7a55030740 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c6f380 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.060+0000 7f7a55030740 1 --2- 172.21.15.97:0/2800775559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 0x559028c6ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.060+0000 7f7a55030740 1 -- 172.21.15.97:0/2800775559 >> 172.21.15.97:0/2800775559 conn(0x559028c4f310 msgr2=0x559028c4e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.060+0000 7f7a35600640 0 client.25522 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.061+0000 7f7a55030740 1 -- 172.21.15.97:0/2800775559 shutdown_connections 2024-03-26T19:02:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.061+0000 7f7a55030740 1 -- 172.21.15.97:0/2800775559 wait complete. 2024-03-26T19:02:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.061+0000 7f7a55030740 20 client.25522 trim_cache size 0 max 16384 2024-03-26T19:02:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.095+0000 7f7a55030740 1 Processor -- start 2024-03-26T19:02:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.095+0000 7f7a55030740 1 -- start start 2024-03-26T19:02:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.095+0000 7f7a55030740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c53da0 0x559028c592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.095+0000 7f7a55030740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c54790 0x559028c76380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.095+0000 7f7a55030740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c55940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.095+0000 7f7a55030740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c1a6b00 con 0x559028c54790 2024-03-26T19:02:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.095+0000 7f7a55030740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c1a8640 con 0x559028c59900 2024-03-26T19:02:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.095+0000 7f7a55030740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c1a71d0 con 0x559028c53da0 2024-03-26T19:02:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.095+0000 7f7a4b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c53da0 0x559028c592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.095+0000 7f7a4b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c53da0 0x559028c592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44238/0 (socket says 172.21.15.97:44238) 2024-03-26T19:02:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.096+0000 7f7a4b400640 1 -- 172.21.15.97:0/2979318585 learned_addr learned my addr 172.21.15.97:0/2979318585 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.096+0000 7f7a4be00640 1 --2- 172.21.15.97:0/2979318585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c55940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.096+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/2979318585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c54790 0x559028c76380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.096+0000 7f7a4b400640 1 -- 172.21.15.97:0/2979318585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 msgr2=0x559028c55940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.096+0000 7f7a4b400640 1 --2- 172.21.15.97:0/2979318585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c55940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.096+0000 7f7a4b400640 1 -- 172.21.15.97:0/2979318585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c54790 msgr2=0x559028c76380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.096+0000 7f7a4b400640 1 --2- 172.21.15.97:0/2979318585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c54790 0x559028c76380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.096+0000 7f7a4b400640 1 -- 172.21.15.97:0/2979318585 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559028c54b70 con 0x559028c53da0 2024-03-26T19:02:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.097+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/2979318585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c54790 0x559028c76380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:02:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.098+0000 7f7a4b400640 1 --2- 172.21.15.97:0/2979318585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c53da0 0x559028c592c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009fd0 tx=0x7f7a3c0ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d952d303b3ef7a91 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.098+0000 7f7a37e00640 1 -- 172.21.15.97:0/2979318585 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c020040 con 0x559028c53da0 2024-03-26T19:02:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.098+0000 7f7a37e00640 1 -- 172.21.15.97:0/2979318585 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3c0bb040 con 0x559028c53da0 2024-03-26T19:02:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.098+0000 7f7a37e00640 1 -- 172.21.15.97:0/2979318585 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c009b10 con 0x559028c53da0 2024-03-26T19:02:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.099+0000 7f7a55030740 1 -- 172.21.15.97:0/2979318585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c53da0 msgr2=0x559028c592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.099+0000 7f7a55030740 1 --2- 172.21.15.97:0/2979318585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c53da0 0x559028c592c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009fd0 tx=0x7f7a3c0ce040 comp rx=0 tx=0).stop 2024-03-26T19:02:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.099+0000 7f7a55030740 1 -- 172.21.15.97:0/2979318585 shutdown_connections 2024-03-26T19:02:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.099+0000 7f7a55030740 1 --2- 172.21.15.97:0/2979318585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c55940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.099+0000 7f7a55030740 1 --2- 172.21.15.97:0/2979318585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c54790 0x559028c76380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.099+0000 7f7a55030740 1 --2- 172.21.15.97:0/2979318585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c53da0 0x559028c592c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.099+0000 7f7a55030740 1 -- 172.21.15.97:0/2979318585 >> 172.21.15.97:0/2979318585 conn(0x559028c4f310 msgr2=0x559028c55230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.100+0000 7f7a55030740 1 -- 172.21.15.97:0/2979318585 shutdown_connections 2024-03-26T19:02:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.100+0000 7f7a55030740 1 -- 172.21.15.97:0/2979318585 wait complete. 2024-03-26T19:02:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.101+0000 7f7a55030740 1 -- start start 2024-03-26T19:02:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.101+0000 7f7a55030740 1 Processor -- start 2024-03-26T19:02:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.102+0000 7f7a55030740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 0x559028c73680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.102+0000 7f7a55030740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c73bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.102+0000 7f7a55030740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c74bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.102+0000 7f7a55030740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c1a7f70 con 0x559028c53da0 2024-03-26T19:02:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.102+0000 7f7a55030740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c1a78a0 con 0x559028c59900 2024-03-26T19:02:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.102+0000 7f7a55030740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c116850 con 0x559028c54790 2024-03-26T19:02:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.102+0000 7f7a55030740 10 client.0 _ll_register_callbacks cb 0x7f7a38009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:02:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.102+0000 7f7a4aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c73bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.102+0000 7f7a4aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c73bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44254/0 (socket says 172.21.15.97:44254) 2024-03-26T19:02:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.102+0000 7f7a4aa00640 1 -- 172.21.15.97:0/3928531093 learned_addr learned my addr 172.21.15.97:0/3928531093 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.103+0000 7f7a4be00640 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c74bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.103+0000 7f7a4b400640 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 0x559028c73680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.103+0000 7f7a4aa00640 1 -- 172.21.15.97:0/3928531093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 msgr2=0x559028c74bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.103+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c74bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.103+0000 7f7a4aa00640 1 -- 172.21.15.97:0/3928531093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 msgr2=0x559028c73680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.103+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 0x559028c73680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.103+0000 7f7a4aa00640 1 -- 172.21.15.97:0/3928531093 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559028c751c0 con 0x559028c54790 2024-03-26T19:02:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.103+0000 7f7a4b400640 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 0x559028c73680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:02:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.104+0000 7f7a4aa00640 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c73bc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7a38011990 tx=0x7f7a38005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5d8cf3682b790f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.104+0000 7f7a35600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:02:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.104+0000 7f7a35600640 1 -- 172.21.15.97:0/3928531093 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3801e070 con 0x559028c54790 2024-03-26T19:02:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.104+0000 7f7a35600640 1 -- 172.21.15.97:0/3928531093 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7a3800f540 con 0x559028c54790 2024-03-26T19:02:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.104+0000 7f7a55030740 1 -- 172.21.15.97:0/3928531093 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a3c0bb040 con 0x559028c54790 2024-03-26T19:02:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.105+0000 7f7a35600640 1 -- 172.21.15.97:0/3928531093 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a38034080 con 0x559028c54790 2024-03-26T19:02:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.105+0000 7f7a55030740 10 client.25528 fetch_fsmap learned FSMap version 27 2024-03-26T19:02:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.105+0000 7f7a55030740 1 -- 172.21.15.97:0/3928531093 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559028c76470 con 0x559028c54790 2024-03-26T19:02:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.105+0000 7f7a35600640 1 -- 172.21.15.97:0/3928531093 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3c0bb040 con 0x559028c54790 2024-03-26T19:02:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.106+0000 7f7a35600640 1 -- 172.21.15.97:0/3928531093 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7a38004460 con 0x559028c54790 2024-03-26T19:02:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.106+0000 7f7a55030740 10 client.25528 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:02:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.106+0000 7f7a55030740 10 client.25528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:02:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.106+0000 7f7a55030740 20 client.25528 populate_metadata read hostname 'smithi097' 2024-03-26T19:02:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.106+0000 7f7a55030740 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559028ce14d0 0x559028ce38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.106+0000 7f7a55030740 4 client.25528 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:02:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.106+0000 7f7a55030740 1 -- 172.21.15.97:0/3928531093 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559028bc0710 con 0x559028ce14d0 2024-03-26T19:02:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.106+0000 7f7a4b400640 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559028ce14d0 0x559028ce38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.107+0000 7f7a4b400640 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559028ce14d0 0x559028ce38f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca3e6c2e67a52a6 server_cookie=b5e0148bbd4c5c38 in_seq=0 out_seq=0 2024-03-26T19:02:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.108+0000 7f7a35600640 10 client.25528 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.125+0000 7f7a35600640 1 -- 172.21.15.97:0/3928531093 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7a3c0b3040 con 0x559028ce14d0 2024-03-26T19:02:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.125+0000 7f7a35600640 10 client.25528 handle_command_reply: tid=0 2024-03-26T19:02:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.125+0000 7f7a55030740 1 client.25528 shutdown 2024-03-26T19:02:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.126+0000 7f7a55030740 1 -- 172.21.15.97:0/3928531093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 msgr2=0x559028c73bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.126+0000 7f7a55030740 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c73bc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7a38011990 tx=0x7f7a38005970 comp rx=0 tx=0).stop 2024-03-26T19:02:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.127+0000 7f7a55030740 1 -- 172.21.15.97:0/3928531093 shutdown_connections 2024-03-26T19:02:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.127+0000 7f7a55030740 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559028ce14d0 0x559028ce38f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.127+0000 7f7a55030740 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559028c59900 0x559028c74bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.127+0000 7f7a55030740 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559028c54790 0x559028c73bc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.127+0000 7f7a55030740 1 --2- 172.21.15.97:0/3928531093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559028c53da0 0x559028c73680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.127+0000 7f7a35600640 0 client.25528 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.127+0000 7f7a55030740 1 -- 172.21.15.97:0/3928531093 >> 172.21.15.97:0/3928531093 conn(0x559028c4f310 msgr2=0x559028c4e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.128+0000 7f7a55030740 1 -- 172.21.15.97:0/3928531093 shutdown_connections 2024-03-26T19:02:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.128+0000 7f7a55030740 1 -- 172.21.15.97:0/3928531093 wait complete. 2024-03-26T19:02:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.128+0000 7f7a55030740 20 client.25528 trim_cache size 0 max 16384 2024-03-26T19:02:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:02:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:02:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:02:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:02:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.130+0000 7f7a52400640 1 -- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7a300a3110 msgr2=0x7f7a300a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.130+0000 7f7a52400640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7a300a3110 0x7f7a300a55d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a44034fd0 tx=0x7f7a4402e040 comp rx=0 tx=0).stop 2024-03-26T19:02:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.130+0000 7f7a52400640 1 -- 172.21.15.97:0/2470861137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c1018f0 msgr2=0x7f7a4c19e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.130+0000 7f7a52400640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c1018f0 0x7f7a4c19e0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00d770 tx=0x7f7a3c00db70 comp rx=0 tx=0).stop 2024-03-26T19:02:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.130+0000 7f7a52400640 1 -- 172.21.15.97:0/2470861137 shutdown_connections 2024-03-26T19:02:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.130+0000 7f7a52400640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7a300a3110 0x7f7a300a55d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.130+0000 7f7a52400640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a4c1034a0 0x7f7a4c1a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.130+0000 7f7a52400640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a4c102ae0 0x7f7a4c19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.130+0000 7f7a52400640 1 --2- 172.21.15.97:0/2470861137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c1018f0 0x7f7a4c19e0c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.130+0000 7f7a52400640 1 -- 172.21.15.97:0/2470861137 >> 172.21.15.97:0/2470861137 conn(0x7f7a4c0fce80 msgr2=0x7f7a4c10dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.131+0000 7f7a52400640 1 -- 172.21.15.97:0/2470861137 shutdown_connections 2024-03-26T19:02:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:43.131+0000 7f7a52400640 1 -- 172.21.15.97:0/2470861137 wait complete. 2024-03-26T19:02:44.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:44 smithi097 ceph-mon[31955]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.6 MiB/s wr, 664 op/s 2024-03-26T19:02:44.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:44 smithi144 ceph-mon[36053]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.6 MiB/s wr, 664 op/s 2024-03-26T19:02:44.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:44 smithi105 ceph-mon[36270]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.6 MiB/s wr, 664 op/s 2024-03-26T19:02:45.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 200.0 s, 2395.2 tps, lat 41.606 ms stddev 49.539 2024-03-26T19:02:46.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:46 smithi097 ceph-mon[31955]: pgmap v505: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 723 op/s 2024-03-26T19:02:46.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:46 smithi144 ceph-mon[36053]: pgmap v505: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 723 op/s 2024-03-26T19:02:46.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:46 smithi105 ceph-mon[36270]: pgmap v505: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 723 op/s 2024-03-26T19:02:48.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:48 smithi097 ceph-mon[31955]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 MiB/s wr, 617 op/s 2024-03-26T19:02:48.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:48 smithi144 ceph-mon[36053]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 MiB/s wr, 617 op/s 2024-03-26T19:02:48.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:48 smithi105 ceph-mon[36270]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 MiB/s wr, 617 op/s 2024-03-26T19:02:49.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:49 smithi097 ceph-mon[31955]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 730 op/s 2024-03-26T19:02:49.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:49 smithi144 ceph-mon[36053]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 730 op/s 2024-03-26T19:02:49.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:49 smithi105 ceph-mon[36270]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 730 op/s 2024-03-26T19:02:50.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 205.0 s, 2294.2 tps, lat 43.756 ms stddev 50.911 2024-03-26T19:02:52.378 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:52 smithi144 ceph-mon[36053]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 713 op/s 2024-03-26T19:02:52.414 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:52 smithi105 ceph-mon[36270]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 713 op/s 2024-03-26T19:02:52.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:52 smithi097 ceph-mon[31955]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 713 op/s 2024-03-26T19:02:54.378 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:54 smithi144 ceph-mon[36053]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-03-26T19:02:54.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:54 smithi105 ceph-mon[36270]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-03-26T19:02:54.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:54 smithi097 ceph-mon[31955]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-03-26T19:02:55.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 210.0 s, 2233.4 tps, lat 44.600 ms stddev 49.543 2024-03-26T19:02:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.323+0000 7f110ca00640 1 -- 172.21.15.97:0/3572223740 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f802cc90 con 0x7f1110107f20 2024-03-26T19:02:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.323+0000 7f1114a00640 1 -- 172.21.15.97:0/3572223740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1110107f20 msgr2=0x7f1110108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.323+0000 7f1114a00640 1 --2- 172.21.15.97:0/3572223740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1110107f20 0x7f1110108340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009900 tx=0x7f10f802bc40 comp rx=0 tx=0).stop 2024-03-26T19:02:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.324+0000 7f1114a00640 1 -- 172.21.15.97:0/3572223740 shutdown_connections 2024-03-26T19:02:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.324+0000 7f1114a00640 1 --2- 172.21.15.97:0/3572223740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1110109a30 0x7f1110110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.324+0000 7f1114a00640 1 --2- 172.21.15.97:0/3572223740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1110109070 0x7f11101094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.324+0000 7f1114a00640 1 --2- 172.21.15.97:0/3572223740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1110107f20 0x7f1110108340 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.324+0000 7f1114a00640 1 -- 172.21.15.97:0/3572223740 >> 172.21.15.97:0/3572223740 conn(0x7f1110076ed0 msgr2=0x7f11100792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.324+0000 7f1114a00640 1 -- 172.21.15.97:0/3572223740 shutdown_connections 2024-03-26T19:02:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.324+0000 7f1114a00640 1 -- 172.21.15.97:0/3572223740 wait complete. 2024-03-26T19:02:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.326+0000 7f1114a00640 1 Processor -- start 2024-03-26T19:02:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.326+0000 7f1114a00640 1 -- start start 2024-03-26T19:02:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.326+0000 7f1114a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1110107f20 0x7f111019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.326+0000 7f1114a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1110109070 0x7f111019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f1114a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1110109a30 0x7f11101a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f1114a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1110113c20 con 0x7f1110109a30 2024-03-26T19:02:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f1114a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1110113aa0 con 0x7f1110107f20 2024-03-26T19:02:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f1114a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1110113da0 con 0x7f1110109070 2024-03-26T19:02:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f110de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1110107f20 0x7f111019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f110de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1110107f20 0x7f111019e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:41866/0 (socket says 172.21.15.97:41866) 2024-03-26T19:02:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f110de00640 1 -- 172.21.15.97:0/3658857007 learned_addr learned my addr 172.21.15.97:0/3658857007 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f110d400640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1110109070 0x7f111019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f110de00640 1 -- 172.21.15.97:0/3658857007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1110109070 msgr2=0x7f111019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f110de00640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1110109070 0x7f111019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f110de00640 1 -- 172.21.15.97:0/3658857007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1110109a30 msgr2=0x7f11101a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f110de00640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1110109a30 0x7f11101a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.327+0000 7f110de00640 1 -- 172.21.15.97:0/3658857007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11101a5d90 con 0x7f1110107f20 2024-03-26T19:02:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.328+0000 7f110de00640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1110107f20 0x7f111019e170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f10f802c8f0 tx=0x7f10f802c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.328+0000 7f1106a00640 1 -- 172.21.15.97:0/3658857007 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8041070 con 0x7f1110107f20 2024-03-26T19:02:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.328+0000 7f1106a00640 1 -- 172.21.15.97:0/3658857007 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f8004450 con 0x7f1110107f20 2024-03-26T19:02:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.328+0000 7f1114a00640 1 -- 172.21.15.97:0/3658857007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11101a6020 con 0x7f1110107f20 2024-03-26T19:02:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.328+0000 7f1106a00640 1 -- 172.21.15.97:0/3658857007 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8034440 con 0x7f1110107f20 2024-03-26T19:02:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.328+0000 7f1114a00640 1 -- 172.21.15.97:0/3658857007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11101a6560 con 0x7f1110107f20 2024-03-26T19:02:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.334+0000 7f1106a00640 1 -- 172.21.15.97:0/3658857007 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f10f8044070 con 0x7f1110107f20 2024-03-26T19:02:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.335+0000 7f1117657740 1 Processor -- start 2024-03-26T19:02:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.336+0000 7f1117657740 1 -- start start 2024-03-26T19:02:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.336+0000 7f1117657740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 0x562d808121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.337+0000 7f1117657740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 0x562d80817290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.337+0000 7f110e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 0x562d808121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.337+0000 7f110e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 0x562d808121c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:41878/0 (socket says 172.21.15.97:41878) 2024-03-26T19:02:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.337+0000 7f110e800640 1 -- 172.21.15.97:0/38026042 learned_addr learned my addr 172.21.15.97:0/38026042 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.337+0000 7f1117657740 1 --2- 172.21.15.97:0/38026042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80817900 0x562d80819d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.337+0000 7f1117657740 1 -- 172.21.15.97:0/38026042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562d80776e80 con 0x562d80812790 2024-03-26T19:02:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.337+0000 7f1117657740 1 -- 172.21.15.97:0/38026042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562d8071f660 con 0x562d80811da0 2024-03-26T19:02:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.337+0000 7f1117657740 1 -- 172.21.15.97:0/38026042 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562d80772e30 con 0x562d80817900 2024-03-26T19:02:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.337+0000 7f110d400640 1 --2- 172.21.15.97:0/38026042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80817900 0x562d80819d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.338+0000 7f110de00640 1 --2- 172.21.15.97:0/38026042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 0x562d80817290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.338+0000 7f110d400640 1 -- 172.21.15.97:0/38026042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 msgr2=0x562d808121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.338+0000 7f110d400640 1 --2- 172.21.15.97:0/38026042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 0x562d808121c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.338+0000 7f110d400640 1 -- 172.21.15.97:0/38026042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 msgr2=0x562d80817290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.338+0000 7f110d400640 1 --2- 172.21.15.97:0/38026042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 0x562d80817290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.338+0000 7f110d400640 1 -- 172.21.15.97:0/38026042 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8048b210 con 0x562d80817900 2024-03-26T19:02:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.338+0000 7f1106a00640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10f40a30c0 0x7f10f40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.338+0000 7f1106a00640 1 -- 172.21.15.97:0/3658857007 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f10f80f1010 con 0x7f1110107f20 2024-03-26T19:02:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.339+0000 7f110d400640 1 --2- 172.21.15.97:0/38026042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80817900 0x562d80819d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1110107c40 tx=0x7f1100010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf0aade9b039ae7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.339+0000 7f110e800640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10f40a30c0 0x7f10f40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.339+0000 7f10ff000640 1 -- 172.21.15.97:0/38026042 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100003b70 con 0x562d80817900 2024-03-26T19:02:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.340+0000 7f10ff000640 1 -- 172.21.15.97:0/38026042 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1100003d10 con 0x562d80817900 2024-03-26T19:02:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.340+0000 7f10ff000640 1 -- 172.21.15.97:0/38026042 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100011ae0 con 0x562d80817900 2024-03-26T19:02:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.342+0000 7f1117657740 1 -- 172.21.15.97:0/38026042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80817900 msgr2=0x562d80819d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.342+0000 7f110e800640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10f40a30c0 0x7f10f40a5580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x562d80812ff0 tx=0x7f11080096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.342+0000 7f1117657740 1 --2- 172.21.15.97:0/38026042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80817900 0x562d80819d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1110107c40 tx=0x7f1100010860 comp rx=0 tx=0).stop 2024-03-26T19:02:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.342+0000 7f1117657740 1 -- 172.21.15.97:0/38026042 shutdown_connections 2024-03-26T19:02:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.342+0000 7f1117657740 1 --2- 172.21.15.97:0/38026042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80817900 0x562d80819d90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.342+0000 7f1117657740 1 --2- 172.21.15.97:0/38026042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 0x562d80817290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.342+0000 7f1117657740 1 --2- 172.21.15.97:0/38026042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 0x562d808121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.342+0000 7f1117657740 1 -- 172.21.15.97:0/38026042 >> 172.21.15.97:0/38026042 conn(0x562d8080d310 msgr2=0x562d8080d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.343+0000 7f1117657740 1 -- 172.21.15.97:0/38026042 shutdown_connections 2024-03-26T19:02:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.343+0000 7f1117657740 1 -- 172.21.15.97:0/38026042 wait complete. 2024-03-26T19:02:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.343+0000 7f1117657740 1 -- start start 2024-03-26T19:02:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.344+0000 7f1117657740 1 Processor -- start 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.344+0000 7f1117657740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80811da0 0x562d8081bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.344+0000 7f1117657740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 0x562d8081c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.344+0000 7f1117657740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 0x562d808232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.344+0000 7f1117657740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562d8081a8b0 con 0x562d80812790 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.344+0000 7f1117657740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562d80732ac0 con 0x562d80817900 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.344+0000 7f1117657740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562d8081aa30 con 0x562d80811da0 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f1117657740 10 client.0 _ll_register_callbacks cb 0x562d80732490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80811da0 0x562d8081bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80811da0 0x562d8081bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56042/0 (socket says 172.21.15.97:56042) 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110de00640 1 -- 172.21.15.97:0/346085561 learned_addr learned my addr 172.21.15.97:0/346085561 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110d400640 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 0x562d8081c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110de00640 1 -- 172.21.15.97:0/346085561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 msgr2=0x562d808232d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110e800640 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 0x562d808232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110de00640 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 0x562d808232d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110de00640 1 -- 172.21.15.97:0/346085561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 msgr2=0x562d8081c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110de00640 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 0x562d8081c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110de00640 1 -- 172.21.15.97:0/346085561 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d80557620 con 0x562d80811da0 2024-03-26T19:02:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110d400640 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 0x562d8081c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:02:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.345+0000 7f110e800640 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 0x562d808232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:02:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.346+0000 7f110de00640 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80811da0 0x562d8081bd50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f10f8006010 tx=0x7f10f80bd4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3775fab367991e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.346+0000 7f10f3400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:02:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.346+0000 7f10f3400640 1 -- 172.21.15.97:0/346085561 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f804b050 con 0x562d80811da0 2024-03-26T19:02:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.346+0000 7f10f3400640 1 -- 172.21.15.97:0/346085561 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f10f80bd970 con 0x562d80811da0 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.346+0000 7f1117657740 1 -- 172.21.15.97:0/346085561 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1100003d10 con 0x562d80811da0 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.346+0000 7f10f3400640 1 -- 172.21.15.97:0/346085561 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10f8004c60 con 0x562d80811da0 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.346+0000 7f1117657740 10 client.25540 fetch_fsmap learned FSMap version 27 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.346+0000 7f1117657740 1 -- 172.21.15.97:0/346085561 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d8081a920 con 0x562d80811da0 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.347+0000 7f10f3400640 1 -- 172.21.15.97:0/346085561 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1100003d10 con 0x562d80811da0 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.347+0000 7f10f3400640 1 -- 172.21.15.97:0/346085561 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f10f80c5c90 con 0x562d80811da0 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.347+0000 7f1117657740 10 client.25540 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.347+0000 7f1117657740 10 client.25540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.347+0000 7f1117657740 20 client.25540 populate_metadata read hostname 'smithi097' 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.347+0000 7f1117657740 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d80827dc0 0x562d8082a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.347+0000 7f1117657740 4 client.25540 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.347+0000 7f1117657740 1 -- 172.21.15.97:0/346085561 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d807843c0 con 0x562d80827dc0 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.347+0000 7f110d400640 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d80827dc0 0x562d8082a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.348+0000 7f110d400640 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d80827dc0 0x562d8082a230 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631c00c5fa7f3924 server_cookie=351f3972556cec63 in_seq=0 out_seq=0 2024-03-26T19:02:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.348+0000 7f10f3400640 10 client.25540 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.349+0000 7f10f3400640 1 -- 172.21.15.97:0/346085561 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562d807843c0 con 0x562d80827dc0 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.349+0000 7f10f3400640 10 client.25540 handle_command_reply: tid=0 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.350+0000 7f1117657740 1 client.25540 shutdown 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.351+0000 7f1117657740 1 -- 172.21.15.97:0/346085561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80811da0 msgr2=0x562d8081bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.351+0000 7f1117657740 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80811da0 0x562d8081bd50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f10f8006010 tx=0x7f10f80bd4e0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.351+0000 7f1117657740 1 -- 172.21.15.97:0/346085561 shutdown_connections 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.351+0000 7f1117657740 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d80827dc0 0x562d8082a230 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.351+0000 7f1117657740 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 0x562d808232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.351+0000 7f1117657740 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80812790 0x562d8081c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.351+0000 7f1117657740 1 --2- 172.21.15.97:0/346085561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80811da0 0x562d8081bd50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.351+0000 7f1117657740 1 -- 172.21.15.97:0/346085561 >> 172.21.15.97:0/346085561 conn(0x562d8080d310 msgr2=0x562d8080c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.351+0000 7f10f3400640 0 client.25540 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.352+0000 7f1117657740 1 -- 172.21.15.97:0/346085561 shutdown_connections 2024-03-26T19:02:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.352+0000 7f1117657740 1 -- 172.21.15.97:0/346085561 wait complete. 2024-03-26T19:02:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.352+0000 7f1117657740 20 client.25540 trim_cache size 0 max 16384 2024-03-26T19:02:56.378 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:56 smithi144 ceph-mon[36053]: pgmap v510: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 706 op/s 2024-03-26T19:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.390+0000 7f1117657740 1 Processor -- start 2024-03-26T19:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.391+0000 7f1117657740 1 -- start start 2024-03-26T19:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.391+0000 7f1117657740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80811da0 0x562d8081c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.391+0000 7f1117657740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80813020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.391+0000 7f1117657740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 0x562d80813560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.391+0000 7f1117657740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f11101af230 con 0x562d80811da0 2024-03-26T19:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.391+0000 7f1117657740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f11101b0d70 con 0x562d80817900 2024-03-26T19:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.391+0000 7f1117657740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11101af900 con 0x562d80812790 2024-03-26T19:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.392+0000 7f110e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 0x562d80813560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.392+0000 7f110e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 0x562d80813560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:41888/0 (socket says 172.21.15.97:41888) 2024-03-26T19:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.392+0000 7f110e800640 1 -- 172.21.15.97:0/55847314 learned_addr learned my addr 172.21.15.97:0/55847314 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.392+0000 7f110de00640 1 --2- 172.21.15.97:0/55847314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80811da0 0x562d8081c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.394+0000 7f110d400640 1 --2- 172.21.15.97:0/55847314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80813020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.395+0000 7f110d400640 1 -- 172.21.15.97:0/55847314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 msgr2=0x562d80813560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.395+0000 7f110d400640 1 --2- 172.21.15.97:0/55847314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 0x562d80813560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.395+0000 7f110d400640 1 -- 172.21.15.97:0/55847314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80811da0 msgr2=0x562d8081c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.395+0000 7f110d400640 1 --2- 172.21.15.97:0/55847314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80811da0 0x562d8081c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.395+0000 7f110d400640 1 -- 172.21.15.97:0/55847314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8080bab0 con 0x562d80812790 2024-03-26T19:02:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.395+0000 7f110d400640 1 --2- 172.21.15.97:0/55847314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80813020 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1100006010 tx=0x7f1100009720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3727bc75471515e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.396+0000 7f10fe600640 1 -- 172.21.15.97:0/55847314 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100029020 con 0x562d80812790 2024-03-26T19:02:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.396+0000 7f10fe600640 1 -- 172.21.15.97:0/55847314 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1100009960 con 0x562d80812790 2024-03-26T19:02:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.396+0000 7f10fe600640 1 -- 172.21.15.97:0/55847314 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110001f040 con 0x562d80812790 2024-03-26T19:02:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.397+0000 7f1117657740 1 -- 172.21.15.97:0/55847314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 msgr2=0x562d80813020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.397+0000 7f1117657740 1 --2- 172.21.15.97:0/55847314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80813020 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1100006010 tx=0x7f1100009720 comp rx=0 tx=0).stop 2024-03-26T19:02:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.397+0000 7f1117657740 1 -- 172.21.15.97:0/55847314 shutdown_connections 2024-03-26T19:02:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.397+0000 7f1117657740 1 --2- 172.21.15.97:0/55847314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80817900 0x562d80813560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.397+0000 7f1117657740 1 --2- 172.21.15.97:0/55847314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80813020 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.397+0000 7f1117657740 1 --2- 172.21.15.97:0/55847314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80811da0 0x562d8081c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.397+0000 7f1117657740 1 -- 172.21.15.97:0/55847314 >> 172.21.15.97:0/55847314 conn(0x562d8080d310 msgr2=0x562d8081b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.397+0000 7f1117657740 1 -- 172.21.15.97:0/55847314 shutdown_connections 2024-03-26T19:02:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.397+0000 7f1117657740 1 -- 172.21.15.97:0/55847314 wait complete. 2024-03-26T19:02:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.398+0000 7f1117657740 1 -- start start 2024-03-26T19:02:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.398+0000 7f1117657740 1 Processor -- start 2024-03-26T19:02:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.399+0000 7f1117657740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 0x562d80820620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.399+0000 7f1117657740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80820b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.399+0000 7f110de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 0x562d80820620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.399+0000 7f110de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 0x562d80820620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:41892/0 (socket says 172.21.15.97:41892) 2024-03-26T19:02:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.399+0000 7f1117657740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80817900 0x562d80821b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.399+0000 7f1117657740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f11101b06a0 con 0x562d80817900 2024-03-26T19:02:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.399+0000 7f1117657740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f11101affd0 con 0x562d80811da0 2024-03-26T19:02:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.399+0000 7f1117657740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11101ad210 con 0x562d80812790 2024-03-26T19:02:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.399+0000 7f1117657740 10 client.0 _ll_register_callbacks cb 0x7f10f8003eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:02:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.399+0000 7f110d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80820b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.400+0000 7f110d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80820b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56048/0 (socket says 172.21.15.97:56048) 2024-03-26T19:02:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.400+0000 7f110d400640 1 -- 172.21.15.97:0/4006835127 learned_addr learned my addr 172.21.15.97:0/4006835127 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:02:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.400+0000 7f110e800640 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80817900 0x562d80821b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.400+0000 7f110d400640 1 -- 172.21.15.97:0/4006835127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 msgr2=0x562d80820620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.400+0000 7f110d400640 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 0x562d80820620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.400+0000 7f110d400640 1 -- 172.21.15.97:0/4006835127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80817900 msgr2=0x562d80821b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:02:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.400+0000 7f110d400640 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80817900 0x562d80821b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.400+0000 7f110d400640 1 -- 172.21.15.97:0/4006835127 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d80822160 con 0x562d80812790 2024-03-26T19:02:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.401+0000 7f110d400640 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80820b60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1100005cc0 tx=0x7f1100003a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d6760678637864a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:02:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.402+0000 7f10f3400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:02:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.405+0000 7f10f3400640 1 -- 172.21.15.97:0/4006835127 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100003d70 con 0x562d80812790 2024-03-26T19:02:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.405+0000 7f10f3400640 1 -- 172.21.15.97:0/4006835127 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1100011770 con 0x562d80812790 2024-03-26T19:02:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.405+0000 7f1117657740 1 -- 172.21.15.97:0/4006835127 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1100009960 con 0x562d80812790 2024-03-26T19:02:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.405+0000 7f10f3400640 1 -- 172.21.15.97:0/4006835127 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1100039050 con 0x562d80812790 2024-03-26T19:02:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.406+0000 7f1117657740 10 client.25546 fetch_fsmap learned FSMap version 27 2024-03-26T19:02:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.406+0000 7f1117657740 1 -- 172.21.15.97:0/4006835127 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d8088e170 con 0x562d80812790 2024-03-26T19:02:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.406+0000 7f10f3400640 1 -- 172.21.15.97:0/4006835127 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1100009960 con 0x562d80812790 2024-03-26T19:02:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.406+0000 7f10f3400640 1 -- 172.21.15.97:0/4006835127 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1100025450 con 0x562d80812790 2024-03-26T19:02:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.406+0000 7f1117657740 10 client.25546 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:02:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.406+0000 7f1117657740 10 client.25546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:02:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.406+0000 7f1117657740 20 client.25546 populate_metadata read hostname 'smithi097' 2024-03-26T19:02:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.406+0000 7f1117657740 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d8088ea00 0x562d80890e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:02:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.406+0000 7f1117657740 4 client.25546 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:02:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.406+0000 7f1117657740 1 -- 172.21.15.97:0/4006835127 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d8077e710 con 0x562d8088ea00 2024-03-26T19:02:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.408+0000 7f110de00640 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d8088ea00 0x562d80890e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:02:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.408+0000 7f110de00640 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d8088ea00 0x562d80890e20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8892c113133ae020 server_cookie=8bc57259f09976ee in_seq=0 out_seq=0 2024-03-26T19:02:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.408+0000 7f10f3400640 10 client.25546 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.438+0000 7f10f3400640 1 -- 172.21.15.97:0/4006835127 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d8077e710 con 0x562d8088ea00 2024-03-26T19:02:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.438+0000 7f10f3400640 10 client.25546 handle_command_reply: tid=0 2024-03-26T19:02:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.438+0000 7f1117657740 1 client.25546 shutdown 2024-03-26T19:02:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.438+0000 7f1117657740 1 -- 172.21.15.97:0/4006835127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 msgr2=0x562d80820b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.438+0000 7f1117657740 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80820b60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1100005cc0 tx=0x7f1100003a00 comp rx=0 tx=0).stop 2024-03-26T19:02:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.438+0000 7f1117657740 1 -- 172.21.15.97:0/4006835127 shutdown_connections 2024-03-26T19:02:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.438+0000 7f1117657740 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d8088ea00 0x562d80890e20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.438+0000 7f1117657740 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d80817900 0x562d80821b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.438+0000 7f1117657740 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d80812790 0x562d80820b60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.438+0000 7f1117657740 1 --2- 172.21.15.97:0/4006835127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d80811da0 0x562d80820620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.439+0000 7f1117657740 1 -- 172.21.15.97:0/4006835127 >> 172.21.15.97:0/4006835127 conn(0x562d8080d310 msgr2=0x562d8081c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.439+0000 7f10f3400640 0 client.25546 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:02:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.439+0000 7f1117657740 1 -- 172.21.15.97:0/4006835127 shutdown_connections 2024-03-26T19:02:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.439+0000 7f1117657740 1 -- 172.21.15.97:0/4006835127 wait complete. 2024-03-26T19:02:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.439+0000 7f1117657740 20 client.25546 trim_cache size 0 max 16384 2024-03-26T19:02:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:02:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:02:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:02:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:02:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.440+0000 7f1114a00640 1 -- 172.21.15.97:0/3658857007 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10f40a30c0 msgr2=0x7f10f40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.440+0000 7f1114a00640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10f40a30c0 0x7f10f40a5580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x562d80812ff0 tx=0x7f11080096c0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.440+0000 7f1114a00640 1 -- 172.21.15.97:0/3658857007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1110107f20 msgr2=0x7f111019e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:02:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.440+0000 7f1114a00640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1110107f20 0x7f111019e170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f10f802c8f0 tx=0x7f10f802c660 comp rx=0 tx=0).stop 2024-03-26T19:02:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.441+0000 7f1114a00640 1 -- 172.21.15.97:0/3658857007 shutdown_connections 2024-03-26T19:02:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.441+0000 7f1114a00640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f10f40a30c0 0x7f10f40a5580 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.441+0000 7f1114a00640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1110109a30 0x7f11101a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.441+0000 7f1114a00640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1110109070 0x7f111019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.441+0000 7f1114a00640 1 --2- 172.21.15.97:0/3658857007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1110107f20 0x7f111019e170 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:02:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.441+0000 7f1114a00640 1 -- 172.21.15.97:0/3658857007 >> 172.21.15.97:0/3658857007 conn(0x7f1110076ed0 msgr2=0x7f111010c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:02:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.441+0000 7f1114a00640 1 -- 172.21.15.97:0/3658857007 shutdown_connections 2024-03-26T19:02:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:02:56.441+0000 7f1114a00640 1 -- 172.21.15.97:0/3658857007 wait complete. 2024-03-26T19:02:56.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:56 smithi105 ceph-mon[36270]: pgmap v510: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 706 op/s 2024-03-26T19:02:56.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:56 smithi097 ceph-mon[31955]: pgmap v510: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 706 op/s 2024-03-26T19:02:58.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:02:58 smithi105 ceph-mon[36270]: pgmap v511: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 696 op/s 2024-03-26T19:02:58.604 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:02:58 smithi097 ceph-mon[31955]: pgmap v511: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 696 op/s 2024-03-26T19:02:58.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:02:58 smithi144 ceph-mon[36053]: pgmap v511: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 696 op/s 2024-03-26T19:03:00.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 215.0 s, 2114.4 tps, lat 47.150 ms stddev 53.206 2024-03-26T19:03:00.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:00 smithi105 ceph-mon[36270]: pgmap v512: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 MiB/s wr, 755 op/s 2024-03-26T19:03:00.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:00 smithi097 ceph-mon[31955]: pgmap v512: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 MiB/s wr, 755 op/s 2024-03-26T19:03:00.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:00 smithi144 ceph-mon[36053]: pgmap v512: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 MiB/s wr, 755 op/s 2024-03-26T19:03:02.406 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:02 smithi097 ceph-mon[31955]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 677 op/s 2024-03-26T19:03:02.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:02 smithi105 ceph-mon[36270]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 677 op/s 2024-03-26T19:03:02.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:02 smithi144 ceph-mon[36053]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 677 op/s 2024-03-26T19:03:04.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:04 smithi105 ceph-mon[36270]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 631 op/s 2024-03-26T19:03:04.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:04 smithi097 ceph-mon[31955]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 631 op/s 2024-03-26T19:03:04.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:04 smithi144 ceph-mon[36053]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 631 op/s 2024-03-26T19:03:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 220.0 s, 2236.0 tps, lat 44.801 ms stddev 56.915 2024-03-26T19:03:06.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:06 smithi105 ceph-mon[36270]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 735 op/s 2024-03-26T19:03:06.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:06 smithi097 ceph-mon[31955]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 735 op/s 2024-03-26T19:03:06.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:06 smithi144 ceph-mon[36053]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 735 op/s 2024-03-26T19:03:08.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:08 smithi105 ceph-mon[36270]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 713 op/s 2024-03-26T19:03:08.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:08 smithi097 ceph-mon[31955]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 713 op/s 2024-03-26T19:03:08.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:08 smithi144 ceph-mon[36053]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 713 op/s 2024-03-26T19:03:09.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:09 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:03:09.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:09 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:03:09.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:09 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:03:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.644+0000 7fe023000640 1 -- 172.21.15.97:0/3926912844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe01c104ac0 msgr2=0x7fe01c106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.644+0000 7fe023000640 1 --2- 172.21.15.97:0/3926912844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe01c104ac0 0x7fe01c106eb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe010009900 tx=0x7fe01002bc40 comp rx=0 tx=0).stop 2024-03-26T19:03:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.645+0000 7fe023000640 1 -- 172.21.15.97:0/3926912844 shutdown_connections 2024-03-26T19:03:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.645+0000 7fe023000640 1 --2- 172.21.15.97:0/3926912844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe01c1073f0 0x7fe01c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.645+0000 7fe023000640 1 --2- 172.21.15.97:0/3926912844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe01c104ac0 0x7fe01c106eb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.645+0000 7fe023000640 1 --2- 172.21.15.97:0/3926912844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe01c102190 0x7fe01c104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.645+0000 7fe023000640 1 -- 172.21.15.97:0/3926912844 >> 172.21.15.97:0/3926912844 conn(0x7fe01c0fbba0 msgr2=0x7fe01c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.645+0000 7fe023000640 1 -- 172.21.15.97:0/3926912844 shutdown_connections 2024-03-26T19:03:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.645+0000 7fe023000640 1 -- 172.21.15.97:0/3926912844 wait complete. 2024-03-26T19:03:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.646+0000 7fe023000640 1 Processor -- start 2024-03-26T19:03:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.647+0000 7fe023000640 1 -- start start 2024-03-26T19:03:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.647+0000 7fe023000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe01c102190 0x7fe01c1957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.647+0000 7fe023000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe01c104ac0 0x7fe01c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.647+0000 7fe023000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe01c1073f0 0x7fe01c1a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.647+0000 7fe023000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c10d3d0 con 0x7fe01c104ac0 2024-03-26T19:03:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.647+0000 7fe023000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c10d250 con 0x7fe01c102190 2024-03-26T19:03:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.647+0000 7fe023000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c10d550 con 0x7fe01c1073f0 2024-03-26T19:03:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.648+0000 7fe022600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe01c1073f0 0x7fe01c1a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.648+0000 7fe022600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe01c1073f0 0x7fe01c1a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:59456/0 (socket says 172.21.15.97:59456) 2024-03-26T19:03:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.648+0000 7fe022600640 1 -- 172.21.15.97:0/3506301990 learned_addr learned my addr 172.21.15.97:0/3506301990 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.648+0000 7fe021c00640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe01c102190 0x7fe01c1957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.649+0000 7fe021200640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe01c104ac0 0x7fe01c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.649+0000 7fe022600640 1 -- 172.21.15.97:0/3506301990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe01c102190 msgr2=0x7fe01c1957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.649+0000 7fe022600640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe01c102190 0x7fe01c1957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.649+0000 7fe022600640 1 -- 172.21.15.97:0/3506301990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe01c104ac0 msgr2=0x7fe01c195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.649+0000 7fe022600640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe01c104ac0 0x7fe01c195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.649+0000 7fe022600640 1 -- 172.21.15.97:0/3506301990 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c1a5c40 con 0x7fe01c1073f0 2024-03-26T19:03:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.650+0000 7fe022600640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe01c1073f0 0x7fe01c1a5700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe01400cad0 tx=0x7fe01400ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.652+0000 7fe01a000640 1 -- 172.21.15.97:0/3506301990 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01400ad60 con 0x7fe01c1073f0 2024-03-26T19:03:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.652+0000 7fe01a000640 1 -- 172.21.15.97:0/3506301990 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe014005020 con 0x7fe01c1073f0 2024-03-26T19:03:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.652+0000 7fe01a000640 1 -- 172.21.15.97:0/3506301990 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01400f6b0 con 0x7fe01c1073f0 2024-03-26T19:03:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.652+0000 7fe023000640 1 -- 172.21.15.97:0/3506301990 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c1a5f30 con 0x7fe01c1073f0 2024-03-26T19:03:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.652+0000 7fe023000640 1 -- 172.21.15.97:0/3506301990 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c1a63a0 con 0x7fe01c1073f0 2024-03-26T19:03:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.658+0000 7fe027376740 1 Processor -- start 2024-03-26T19:03:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.664+0000 7fe01a000640 1 -- 172.21.15.97:0/3506301990 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fe01400f850 con 0x7fe01c1073f0 2024-03-26T19:03:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.666+0000 7fe027376740 1 -- start start 2024-03-26T19:03:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.669+0000 7fe027376740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162e8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.669+0000 7fe027376740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.669+0000 7fe027376740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162eff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.669+0000 7fe027376740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c1624ce80 con 0x557c162e7f60 2024-03-26T19:03:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.669+0000 7fe027376740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557c161f5660 con 0x557c162edac0 2024-03-26T19:03:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.669+0000 7fe027376740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557c16248e30 con 0x557c162e8950 2024-03-26T19:03:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.670+0000 7fe01a000640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe0040b4070 0x7fe0040b6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.670+0000 7fe01a000640 1 -- 172.21.15.97:0/3506301990 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe0140cf370 con 0x7fe01c1073f0 2024-03-26T19:03:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.672+0000 7fe021c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.672+0000 7fe021c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162ed450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:59460/0 (socket says 172.21.15.97:59460) 2024-03-26T19:03:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.672+0000 7fe021c00640 1 -- 172.21.15.97:0/1440102487 learned_addr learned my addr 172.21.15.97:0/1440102487 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.672+0000 7fe022600640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe0040b4070 0x7fe0040b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.673+0000 7fe021200640 1 --2- 172.21.15.97:0/1440102487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.674+0000 7fe021200640 1 --2- 172.21.15.97:0/1440102487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162e8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.674+0000 7fe021200640 1 -- 172.21.15.97:0/1440102487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 msgr2=0x557c162ed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.675+0000 7fe021200640 1 --2- 172.21.15.97:0/1440102487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162ed450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.675+0000 7fe021200640 1 -- 172.21.15.97:0/1440102487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 msgr2=0x557c162e8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.675+0000 7fe021200640 1 --2- 172.21.15.97:0/1440102487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162e8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.675+0000 7fe021200640 1 -- 172.21.15.97:0/1440102487 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c15f61210 con 0x557c162edac0 2024-03-26T19:03:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.675+0000 7fe022600640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe0040b4070 0x7fe0040b6530 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe014097ed0 tx=0x7fe014005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.676+0000 7fe021200640 1 --2- 172.21.15.97:0/1440102487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162eff50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe010009230 tx=0x7fe010005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b965e727bcc114ed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.677+0000 7fe00aa00640 1 -- 172.21.15.97:0/1440102487 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010044070 con 0x557c162edac0 2024-03-26T19:03:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.677+0000 7fe00aa00640 1 -- 172.21.15.97:0/1440102487 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe01002cd10 con 0x557c162edac0 2024-03-26T19:03:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.677+0000 7fe00aa00640 1 -- 172.21.15.97:0/1440102487 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010005200 con 0x557c162edac0 2024-03-26T19:03:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.678+0000 7fe027376740 1 -- 172.21.15.97:0/1440102487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 msgr2=0x557c162eff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.678+0000 7fe027376740 1 --2- 172.21.15.97:0/1440102487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162eff50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe010009230 tx=0x7fe010005fd0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.680+0000 7fe027376740 1 -- 172.21.15.97:0/1440102487 shutdown_connections 2024-03-26T19:03:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.680+0000 7fe027376740 1 --2- 172.21.15.97:0/1440102487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162eff50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.680+0000 7fe027376740 1 --2- 172.21.15.97:0/1440102487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.680+0000 7fe027376740 1 --2- 172.21.15.97:0/1440102487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162e8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.680+0000 7fe027376740 1 -- 172.21.15.97:0/1440102487 >> 172.21.15.97:0/1440102487 conn(0x557c162e34d0 msgr2=0x557c162e38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.681+0000 7fe027376740 1 -- 172.21.15.97:0/1440102487 shutdown_connections 2024-03-26T19:03:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.681+0000 7fe027376740 1 -- 172.21.15.97:0/1440102487 wait complete. 2024-03-26T19:03:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.681+0000 7fe027376740 1 -- start start 2024-03-26T19:03:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.682+0000 7fe027376740 1 Processor -- start 2024-03-26T19:03:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.685+0000 7fe027376740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.685+0000 7fe027376740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.686+0000 7fe027376740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.686+0000 7fe027376740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c162f0a10 con 0x557c162e7f60 2024-03-26T19:03:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.686+0000 7fe027376740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557c16208ac0 con 0x557c162edac0 2024-03-26T19:03:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.686+0000 7fe027376740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557c162f0b90 con 0x557c162e8950 2024-03-26T19:03:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.686+0000 7fe027376740 10 client.0 _ll_register_callbacks cb 0x557c16208490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:03:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.686+0000 7fe021c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.687+0000 7fe021c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162f24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:59470/0 (socket says 172.21.15.97:59470) 2024-03-26T19:03:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.687+0000 7fe021c00640 1 -- 172.21.15.97:0/843490604 learned_addr learned my addr 172.21.15.97:0/843490604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.687+0000 7fe021c00640 1 -- 172.21.15.97:0/843490604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 msgr2=0x557c162f94e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:03:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.687+0000 7fe021c00640 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162f94e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.687+0000 7fe021c00640 1 -- 172.21.15.97:0/843490604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 msgr2=0x557c162f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.687+0000 7fe021c00640 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.687+0000 7fe021c00640 1 -- 172.21.15.97:0/843490604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c1602d620 con 0x557c162e8950 2024-03-26T19:03:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.688+0000 7fe021c00640 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162f24a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe00c010a50 tx=0x7fe00c010f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6344d938ed12f27 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.688+0000 7fdfff400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:03:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.689+0000 7fdfff400640 1 -- 172.21.15.97:0/843490604 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00fb70 con 0x557c162e8950 2024-03-26T19:03:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.689+0000 7fdfff400640 1 -- 172.21.15.97:0/843490604 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe00c00fd10 con 0x557c162e8950 2024-03-26T19:03:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.689+0000 7fe027376740 1 -- 172.21.15.97:0/843490604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe01002cd10 con 0x557c162e8950 2024-03-26T19:03:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.690+0000 7fdfff400640 1 -- 172.21.15.97:0/843490604 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe00c005020 con 0x557c162e8950 2024-03-26T19:03:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.690+0000 7fe027376740 10 client.25558 fetch_fsmap learned FSMap version 27 2024-03-26T19:03:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.690+0000 7fe027376740 1 -- 172.21.15.97:0/843490604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c162fc920 con 0x557c162e8950 2024-03-26T19:03:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.690+0000 7fdfff400640 1 -- 172.21.15.97:0/843490604 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe01002cd10 con 0x557c162e8950 2024-03-26T19:03:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.691+0000 7fdfff400640 1 -- 172.21.15.97:0/843490604 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe00c019410 con 0x557c162e8950 2024-03-26T19:03:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.691+0000 7fe027376740 10 client.25558 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:03:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.691+0000 7fe027376740 10 client.25558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:03:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.691+0000 7fe027376740 20 client.25558 populate_metadata read hostname 'smithi097' 2024-03-26T19:03:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.691+0000 7fe027376740 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557c162fe0d0 0x557c16300550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.692+0000 7fe021200640 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557c162fe0d0 0x557c16300550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.694+0000 7fe027376740 4 client.25558 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:03:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.694+0000 7fe027376740 1 -- 172.21.15.97:0/843490604 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c1625a3c0 con 0x557c162fe0d0 2024-03-26T19:03:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.694+0000 7fe021200640 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557c162fe0d0 0x557c16300550 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d031fdefc3689d6c server_cookie=5da7ea0d7f4ff711 in_seq=0 out_seq=0 2024-03-26T19:03:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.695+0000 7fdfff400640 10 client.25558 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.695+0000 7fdfff400640 1 -- 172.21.15.97:0/843490604 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557c1625a3c0 con 0x557c162fe0d0 2024-03-26T19:03:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.695+0000 7fdfff400640 10 client.25558 handle_command_reply: tid=0 2024-03-26T19:03:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.695+0000 7fe027376740 1 client.25558 shutdown 2024-03-26T19:03:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.697+0000 7fe027376740 1 -- 172.21.15.97:0/843490604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 msgr2=0x557c162f24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.699+0000 7fe027376740 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162f24a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe00c010a50 tx=0x7fe00c010f10 comp rx=0 tx=0).stop 2024-03-26T19:03:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.700+0000 7fe027376740 1 -- 172.21.15.97:0/843490604 shutdown_connections 2024-03-26T19:03:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.700+0000 7fe027376740 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557c162fe0d0 0x557c16300550 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.700+0000 7fe027376740 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.700+0000 7fe027376740 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162f24a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.700+0000 7fe027376740 1 --2- 172.21.15.97:0/843490604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.700+0000 7fe027376740 1 -- 172.21.15.97:0/843490604 >> 172.21.15.97:0/843490604 conn(0x557c162e34d0 msgr2=0x557c162e26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.700+0000 7fdfff400640 0 client.25558 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.704+0000 7fe027376740 1 -- 172.21.15.97:0/843490604 shutdown_connections 2024-03-26T19:03:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.704+0000 7fe027376740 1 -- 172.21.15.97:0/843490604 wait complete. 2024-03-26T19:03:09.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.704+0000 7fe027376740 20 client.25558 trim_cache size 0 max 16384 2024-03-26T19:03:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.761+0000 7fe027376740 1 Processor -- start 2024-03-26T19:03:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.762+0000 7fe027376740 1 -- start start 2024-03-26T19:03:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.762+0000 7fe027376740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.762+0000 7fe027376740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.762+0000 7fe027376740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.762+0000 7fe027376740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c1a6830 con 0x557c162e7f60 2024-03-26T19:03:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.762+0000 7fe027376740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c1a8370 con 0x557c162edac0 2024-03-26T19:03:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.762+0000 7fe027376740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c1a6f00 con 0x557c162e8950 2024-03-26T19:03:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.762+0000 7fe021200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.763+0000 7fe021200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162e9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33414/0 (socket says 172.21.15.97:33414) 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.763+0000 7fe021200640 1 -- 172.21.15.97:0/2202091315 learned_addr learned my addr 172.21.15.97:0/2202091315 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.763+0000 7fe021200640 1 --2- 172.21.15.97:0/2202091315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.763+0000 7fe021200640 1 -- 172.21.15.97:0/2202091315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 msgr2=0x557c162f94a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.763+0000 7fe021200640 1 --2- 172.21.15.97:0/2202091315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162f94a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.763+0000 7fe021200640 1 -- 172.21.15.97:0/2202091315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 msgr2=0x557c162e9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.763+0000 7fe021200640 1 --2- 172.21.15.97:0/2202091315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162e9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.763+0000 7fe021200640 1 -- 172.21.15.97:0/2202091315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c162deae0 con 0x557c162e7f60 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.764+0000 7fe021200640 1 --2- 172.21.15.97:0/2202091315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162e9040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe01002efd0 tx=0x7fe010009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b19483a4569d9a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.764+0000 7fe00a000640 1 -- 172.21.15.97:0/2202091315 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010044040 con 0x557c162e7f60 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.764+0000 7fe00a000640 1 -- 172.21.15.97:0/2202091315 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe010004740 con 0x557c162e7f60 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.764+0000 7fe00a000640 1 -- 172.21.15.97:0/2202091315 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0100515b0 con 0x557c162e7f60 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.765+0000 7fe027376740 1 -- 172.21.15.97:0/2202091315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 msgr2=0x557c162e9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.765+0000 7fe027376740 1 --2- 172.21.15.97:0/2202091315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162e9040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe01002efd0 tx=0x7fe010009260 comp rx=0 tx=0).stop 2024-03-26T19:03:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.765+0000 7fe027376740 1 -- 172.21.15.97:0/2202091315 shutdown_connections 2024-03-26T19:03:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.765+0000 7fe027376740 1 --2- 172.21.15.97:0/2202091315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162edac0 0x557c162e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.765+0000 7fe027376740 1 --2- 172.21.15.97:0/2202091315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e8950 0x557c162f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.765+0000 7fe027376740 1 --2- 172.21.15.97:0/2202091315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162e7f60 0x557c162e9040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.765+0000 7fe027376740 1 -- 172.21.15.97:0/2202091315 >> 172.21.15.97:0/2202091315 conn(0x557c162e34d0 msgr2=0x557c162e6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.765+0000 7fe027376740 1 -- 172.21.15.97:0/2202091315 shutdown_connections 2024-03-26T19:03:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.765+0000 7fe027376740 1 -- 172.21.15.97:0/2202091315 wait complete. 2024-03-26T19:03:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.766+0000 7fe027376740 1 -- start start 2024-03-26T19:03:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.766+0000 7fe027376740 1 Processor -- start 2024-03-26T19:03:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.768+0000 7fe027376740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e7f60 0x557c162f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.768+0000 7fe027376740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162e8950 0x557c162f7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.768+0000 7fe027376740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162edac0 0x557c162f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.768+0000 7fe027376740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c1a7ca0 con 0x557c162edac0 2024-03-26T19:03:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.768+0000 7fe027376740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c1a75d0 con 0x557c162e8950 2024-03-26T19:03:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.768+0000 7fe027376740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c071c70 con 0x557c162e7f60 2024-03-26T19:03:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.768+0000 7fe027376740 10 client.0 _ll_register_callbacks cb 0x7fe00c010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:03:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.768+0000 7fe021200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e7f60 0x557c162f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.768+0000 7fe021200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e7f60 0x557c162f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:59474/0 (socket says 172.21.15.97:59474) 2024-03-26T19:03:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.768+0000 7fe021200640 1 -- 172.21.15.97:0/1205686021 learned_addr learned my addr 172.21.15.97:0/1205686021 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.769+0000 7fe021200640 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162edac0 0x557c162f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.769+0000 7fe021200640 1 -- 172.21.15.97:0/1205686021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162e8950 msgr2=0x557c162f7350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:03:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.769+0000 7fe021200640 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162e8950 0x557c162f7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.769+0000 7fe021200640 1 -- 172.21.15.97:0/1205686021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162edac0 msgr2=0x557c162f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.769+0000 7fe021200640 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162edac0 0x557c162f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.769+0000 7fe021200640 1 -- 172.21.15.97:0/1205686021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c162f8950 con 0x557c162e7f60 2024-03-26T19:03:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.770+0000 7fe021200640 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e7f60 0x557c162f6e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe01002c7c0 tx=0x7fe01002c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b27dc229f2c9d29 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.770+0000 7fdfff400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.770+0000 7fdfff400640 1 -- 172.21.15.97:0/1205686021 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010044040 con 0x557c162e7f60 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.770+0000 7fdfff400640 1 -- 172.21.15.97:0/1205686021 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe010004b80 con 0x557c162e7f60 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fe027376740 1 -- 172.21.15.97:0/1205686021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe010004740 con 0x557c162e7f60 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fdfff400640 1 -- 172.21.15.97:0/1205686021 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe010004f10 con 0x557c162e7f60 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fe027376740 10 client.25561 fetch_fsmap learned FSMap version 27 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fe027376740 1 -- 172.21.15.97:0/1205686021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c16364650 con 0x557c162e7f60 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fdfff400640 1 -- 172.21.15.97:0/1205686021 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe010004740 con 0x557c162e7f60 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fdfff400640 1 -- 172.21.15.97:0/1205686021 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe010049a20 con 0x557c162e7f60 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fe027376740 10 client.25561 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fe027376740 10 client.25561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fe027376740 20 client.25561 populate_metadata read hostname 'smithi097' 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fe027376740 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557c16364b00 0x557c16366f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fe027376740 4 client.25561 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:03:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.771+0000 7fe027376740 1 -- 172.21.15.97:0/1205686021 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c16254710 con 0x557c16364b00 2024-03-26T19:03:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.772+0000 7fe021c00640 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557c16364b00 0x557c16366f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.772+0000 7fe021c00640 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557c16364b00 0x557c16366f20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff153c9b9a2f8cf server_cookie=9c70a533dd1bc90b in_seq=0 out_seq=0 2024-03-26T19:03:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.772+0000 7fdfff400640 10 client.25561 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.790+0000 7fdfff400640 1 -- 172.21.15.97:0/1205686021 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c16254710 con 0x557c16364b00 2024-03-26T19:03:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.790+0000 7fdfff400640 10 client.25561 handle_command_reply: tid=0 2024-03-26T19:03:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.790+0000 7fe027376740 1 client.25561 shutdown 2024-03-26T19:03:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 1 -- 172.21.15.97:0/1205686021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e7f60 msgr2=0x557c162f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e7f60 0x557c162f6e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe01002c7c0 tx=0x7fe01002c340 comp rx=0 tx=0).stop 2024-03-26T19:03:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 1 -- 172.21.15.97:0/1205686021 shutdown_connections 2024-03-26T19:03:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x557c16364b00 0x557c16366f20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c162edac0 0x557c162f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c162e8950 0x557c162f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 1 --2- 172.21.15.97:0/1205686021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c162e7f60 0x557c162f6e10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 1 -- 172.21.15.97:0/1205686021 >> 172.21.15.97:0/1205686021 conn(0x557c162e34d0 msgr2=0x557c162e1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fdfff400640 0 client.25561 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 1 -- 172.21.15.97:0/1205686021 shutdown_connections 2024-03-26T19:03:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 1 -- 172.21.15.97:0/1205686021 wait complete. 2024-03-26T19:03:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.791+0000 7fe027376740 20 client.25561 trim_cache size 0 max 16384 2024-03-26T19:03:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:03:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:03:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:03:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 -- 172.21.15.97:0/3506301990 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe0040b4070 msgr2=0x7fe0040b6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe0040b4070 0x7fe0040b6530 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe014097ed0 tx=0x7fe014005960 comp rx=0 tx=0).stop 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 -- 172.21.15.97:0/3506301990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe01c1073f0 msgr2=0x7fe01c1a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe01c1073f0 0x7fe01c1a5700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe01400cad0 tx=0x7fe01400ced0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 -- 172.21.15.97:0/3506301990 shutdown_connections 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe0040b4070 0x7fe0040b6530 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe01c1073f0 0x7fe01c1a5700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe01c104ac0 0x7fe01c195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 --2- 172.21.15.97:0/3506301990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe01c102190 0x7fe01c1957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 -- 172.21.15.97:0/3506301990 >> 172.21.15.97:0/3506301990 conn(0x7fe01c0fbba0 msgr2=0x7fe01c1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 -- 172.21.15.97:0/3506301990 shutdown_connections 2024-03-26T19:03:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:09.792+0000 7fe023000640 1 -- 172.21.15.97:0/3506301990 wait complete. 2024-03-26T19:03:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 225.0 s, 2270.6 tps, lat 44.075 ms stddev 57.923 2024-03-26T19:03:10.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:10 smithi105 ceph-mon[36270]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 710 op/s 2024-03-26T19:03:10.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:10 smithi097 ceph-mon[31955]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 710 op/s 2024-03-26T19:03:10.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:10 smithi144 ceph-mon[36053]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 710 op/s 2024-03-26T19:03:11.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:03:11.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:03:11.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:03:11.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:03:11.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:03:11.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:03:11.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:03:11.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:03:11.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:03:12.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:12 smithi105 ceph-mon[36270]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 675 op/s 2024-03-26T19:03:12.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:12 smithi144 ceph-mon[36053]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 675 op/s 2024-03-26T19:03:12.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:12 smithi097 ceph-mon[31955]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 675 op/s 2024-03-26T19:03:14.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:14 smithi105 ceph-mon[36270]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-03-26T19:03:14.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:14 smithi097 ceph-mon[31955]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-03-26T19:03:14.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:14 smithi144 ceph-mon[36053]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-03-26T19:03:15.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 230.0 s, 2192.2 tps, lat 45.572 ms stddev 54.088 2024-03-26T19:03:16.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:16 smithi105 ceph-mon[36270]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 741 op/s 2024-03-26T19:03:16.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:16 smithi097 ceph-mon[31955]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 741 op/s 2024-03-26T19:03:16.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:16 smithi144 ceph-mon[36053]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 741 op/s 2024-03-26T19:03:18.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:18 smithi097 ceph-mon[31955]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.5 MiB/s wr, 664 op/s 2024-03-26T19:03:18.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:18 smithi144 ceph-mon[36053]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.5 MiB/s wr, 664 op/s 2024-03-26T19:03:18.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:18 smithi105 ceph-mon[36270]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.5 MiB/s wr, 664 op/s 2024-03-26T19:03:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 235.0 s, 2414.0 tps, lat 41.455 ms stddev 57.337 2024-03-26T19:03:20.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:20 smithi097 ceph-mon[31955]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 670 op/s 2024-03-26T19:03:20.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:20 smithi144 ceph-mon[36053]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 670 op/s 2024-03-26T19:03:20.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:20 smithi105 ceph-mon[36270]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 670 op/s 2024-03-26T19:03:22.514 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:22 smithi105 ceph-mon[36270]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.9 MiB/s wr, 741 op/s 2024-03-26T19:03:22.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:22 smithi097 ceph-mon[31955]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.9 MiB/s wr, 741 op/s 2024-03-26T19:03:22.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:22 smithi144 ceph-mon[36053]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.9 MiB/s wr, 741 op/s 2024-03-26T19:03:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.970+0000 7f0a98e00640 1 -- 172.21.15.97:0/2979935491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a94107f20 msgr2=0x7f0a94108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.970+0000 7f0a98e00640 1 --2- 172.21.15.97:0/2979935491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a94107f20 0x7f0a94108340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009900 tx=0x7f0a8402bc60 comp rx=0 tx=0).stop 2024-03-26T19:03:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.972+0000 7f0a98e00640 1 -- 172.21.15.97:0/2979935491 shutdown_connections 2024-03-26T19:03:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.972+0000 7f0a98e00640 1 --2- 172.21.15.97:0/2979935491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a94109a30 0x7f0a94110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.972+0000 7f0a98e00640 1 --2- 172.21.15.97:0/2979935491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a94109070 0x7f0a941094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.972+0000 7f0a98e00640 1 --2- 172.21.15.97:0/2979935491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a94107f20 0x7f0a94108340 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.972+0000 7f0a98e00640 1 -- 172.21.15.97:0/2979935491 >> 172.21.15.97:0/2979935491 conn(0x7f0a94076ed0 msgr2=0x7f0a940792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.972+0000 7f0a98e00640 1 -- 172.21.15.97:0/2979935491 shutdown_connections 2024-03-26T19:03:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.972+0000 7f0a98e00640 1 -- 172.21.15.97:0/2979935491 wait complete. 2024-03-26T19:03:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.974+0000 7f0a98e00640 1 Processor -- start 2024-03-26T19:03:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.974+0000 7f0a98e00640 1 -- start start 2024-03-26T19:03:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.974+0000 7f0a98e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a94107f20 0x7f0a9419e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a98e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a94109070 0x7f0a9419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a98e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a94109a30 0x7f0a941a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a98e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a94113c00 con 0x7f0a94109a30 2024-03-26T19:03:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a98e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0a94113a80 con 0x7f0a94107f20 2024-03-26T19:03:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a98e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a94113d80 con 0x7f0a94109070 2024-03-26T19:03:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a91400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a94109070 0x7f0a9419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a91400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a94109070 0x7f0a9419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53894/0 (socket says 172.21.15.97:53894) 2024-03-26T19:03:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a91400640 1 -- 172.21.15.97:0/614736298 learned_addr learned my addr 172.21.15.97:0/614736298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a91e00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a94107f20 0x7f0a9419e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a91e00640 1 -- 172.21.15.97:0/614736298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a94109070 msgr2=0x7f0a9419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a91e00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a94109070 0x7f0a9419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a91e00640 1 -- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a94109a30 msgr2=0x7f0a941a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.975+0000 7f0a92800640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a94109a30 0x7f0a941a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.976+0000 7f0a91e00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a94109a30 0x7f0a941a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.976+0000 7f0a91e00640 1 -- 172.21.15.97:0/614736298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a941a5d70 con 0x7f0a94107f20 2024-03-26T19:03:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.976+0000 7f0a92800640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a94109a30 0x7f0a941a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:03:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.976+0000 7f0a91e00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a94107f20 0x7f0a9419e150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0a84036040 tx=0x7f0a840040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.976+0000 7f0a8aa00640 1 -- 172.21.15.97:0/614736298 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84043020 con 0x7f0a94107f20 2024-03-26T19:03:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.976+0000 7f0a8aa00640 1 -- 172.21.15.97:0/614736298 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a840044c0 con 0x7f0a94107f20 2024-03-26T19:03:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.976+0000 7f0a8aa00640 1 -- 172.21.15.97:0/614736298 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84003e20 con 0x7f0a94107f20 2024-03-26T19:03:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.977+0000 7f0a98e00640 1 -- 172.21.15.97:0/614736298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a941a6000 con 0x7f0a94107f20 2024-03-26T19:03:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.977+0000 7f0a98e00640 1 -- 172.21.15.97:0/614736298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a941a64d0 con 0x7f0a94107f20 2024-03-26T19:03:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.981+0000 7f0a9bc26740 1 Processor -- start 2024-03-26T19:03:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.982+0000 7f0a9bc26740 1 -- start start 2024-03-26T19:03:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.983+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 0x55d452a9d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.983+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9d790 0x55d452aa2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.983+0000 7f0a92800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 0x55d452a9d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.983+0000 7f0a92800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 0x55d452a9d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45594/0 (socket says 172.21.15.97:45594) 2024-03-26T19:03:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.983+0000 7f0a92800640 1 -- 172.21.15.97:0/2341811849 learned_addr learned my addr 172.21.15.97:0/2341811849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.984+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aa4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.984+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2341811849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d452a01e80 con 0x55d452a9cda0 2024-03-26T19:03:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.984+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2341811849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d4529aa660 con 0x55d452aa2900 2024-03-26T19:03:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.984+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2341811849 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4529fde30 con 0x55d452a9d790 2024-03-26T19:03:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.984+0000 7f0a8aa00640 1 -- 172.21.15.97:0/614736298 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f0a84046020 con 0x7f0a94107f20 2024-03-26T19:03:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.984+0000 7f0a91400640 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aa4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.985+0000 7f0a91e00640 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9d790 0x55d452aa2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.985+0000 7f0a91400640 1 -- 172.21.15.97:0/2341811849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9d790 msgr2=0x55d452aa2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.985+0000 7f0a91400640 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9d790 0x55d452aa2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.985+0000 7f0a91400640 1 -- 172.21.15.97:0/2341811849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 msgr2=0x55d452a9d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.985+0000 7f0a91400640 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 0x55d452a9d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.985+0000 7f0a91400640 1 -- 172.21.15.97:0/2341811849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d452716210 con 0x55d452aa2900 2024-03-26T19:03:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.985+0000 7f0a92800640 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 0x55d452a9d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:03:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.985+0000 7f0a91400640 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aa4d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000bc60 tx=0x7f0a800107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58adb79c8764acc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.985+0000 7f0a7f400640 1 -- 172.21.15.97:0/2341811849 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80004220 con 0x55d452aa2900 2024-03-26T19:03:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.986+0000 7f0a7f400640 1 -- 172.21.15.97:0/2341811849 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a8000be10 con 0x55d452aa2900 2024-03-26T19:03:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.986+0000 7f0a7f400640 1 -- 172.21.15.97:0/2341811849 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80004d50 con 0x55d452aa2900 2024-03-26T19:03:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.986+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2341811849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 msgr2=0x55d452aa4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.986+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aa4d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000bc60 tx=0x7f0a800107a0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.989+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2341811849 shutdown_connections 2024-03-26T19:03:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.989+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aa4d90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.989+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9d790 0x55d452aa2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.989+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2341811849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 0x55d452a9d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.989+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2341811849 >> 172.21.15.97:0/2341811849 conn(0x55d452a98310 msgr2=0x55d452a98710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.989+0000 7f0a8aa00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0a780a30c0 0x7f0a780a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.989+0000 7f0a8aa00640 1 -- 172.21.15.97:0/614736298 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0a840f1540 con 0x7f0a94107f20 2024-03-26T19:03:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.990+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2341811849 shutdown_connections 2024-03-26T19:03:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.990+0000 7f0a91400640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0a780a30c0 0x7f0a780a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.990+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2341811849 wait complete. 2024-03-26T19:03:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.991+0000 7f0a9bc26740 1 -- start start 2024-03-26T19:03:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.991+0000 7f0a9bc26740 1 Processor -- start 2024-03-26T19:03:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.992+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 0x55d452aa6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.992+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9d790 0x55d452aa72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.992+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.992+0000 7f0a92800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.992+0000 7f0a92800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:45928/0 (socket says 172.21.15.97:45928) 2024-03-26T19:03:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.992+0000 7f0a92800640 1 -- 172.21.15.97:0/2518098229 learned_addr learned my addr 172.21.15.97:0/2518098229 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.992+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d452aa58b0 con 0x55d452a9cda0 2024-03-26T19:03:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.992+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d4529bdac0 con 0x55d452aa2900 2024-03-26T19:03:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.992+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d452aa5a30 con 0x55d452a9d790 2024-03-26T19:03:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.992+0000 7f0a91e00640 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 0x55d452aa6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.993+0000 7f0a91400640 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9d790 0x55d452aa72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.993+0000 7f0a9bc26740 10 client.0 _ll_register_callbacks cb 0x55d4529bd490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:03:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.993+0000 7f0a92800640 1 -- 172.21.15.97:0/2518098229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9d790 msgr2=0x55d452aa72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.993+0000 7f0a92800640 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9d790 0x55d452aa72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.993+0000 7f0a92800640 1 -- 172.21.15.97:0/2518098229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 msgr2=0x55d452aa6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.993+0000 7f0a91400640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0a780a30c0 0x7f0a780a5580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c00 tx=0x7f0a80013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.993+0000 7f0a92800640 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 0x55d452aa6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.993+0000 7f0a92800640 1 -- 172.21.15.97:0/2518098229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4527e2620 con 0x55d452aa2900 2024-03-26T19:03:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.993+0000 7f0a92800640 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aae300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c012780 tx=0x7f0a8c012c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=438a82c166f3daaa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.993+0000 7f0a73e00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:03:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.994+0000 7f0a73e00640 1 -- 172.21.15.97:0/2518098229 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c010dc0 con 0x55d452aa2900 2024-03-26T19:03:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.994+0000 7f0a73e00640 1 -- 172.21.15.97:0/2518098229 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0a8c01a3e0 con 0x55d452aa2900 2024-03-26T19:03:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.994+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a8000be10 con 0x55d452aa2900 2024-03-26T19:03:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.995+0000 7f0a73e00640 1 -- 172.21.15.97:0/2518098229 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a8c01a5e0 con 0x55d452aa2900 2024-03-26T19:03:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.995+0000 7f0a9bc26740 10 client.25607 fetch_fsmap learned FSMap version 27 2024-03-26T19:03:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.995+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d452ab16c0 con 0x55d452aa2900 2024-03-26T19:03:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.996+0000 7f0a73e00640 1 -- 172.21.15.97:0/2518098229 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a8000be10 con 0x55d452aa2900 2024-03-26T19:03:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.996+0000 7f0a73e00640 1 -- 172.21.15.97:0/2518098229 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0a8c019070 con 0x55d452aa2900 2024-03-26T19:03:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.997+0000 7f0a9bc26740 10 client.25607 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:03:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.997+0000 7f0a9bc26740 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:03:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.997+0000 7f0a9bc26740 20 client.25607 populate_metadata read hostname 'smithi097' 2024-03-26T19:03:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.997+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d452ab2e00 0x55d452ab5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.997+0000 7f0a9bc26740 4 client.25607 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:03:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.997+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d452a0f3c0 con 0x55d452ab2e00 2024-03-26T19:03:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.997+0000 7f0a91e00640 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d452ab2e00 0x55d452ab5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:22.999+0000 7f0a91e00640 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d452ab2e00 0x55d452ab5280 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f877d5f3252e092 server_cookie=a846add74f65fef2 in_seq=0 out_seq=0 2024-03-26T19:03:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.000+0000 7f0a73e00640 10 client.25607 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.001+0000 7f0a73e00640 1 -- 172.21.15.97:0/2518098229 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d452a0f3c0 con 0x55d452ab2e00 2024-03-26T19:03:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.001+0000 7f0a73e00640 10 client.25607 handle_command_reply: tid=0 2024-03-26T19:03:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.003+0000 7f0a9bc26740 1 client.25607 shutdown 2024-03-26T19:03:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.005+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 msgr2=0x55d452aae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.005+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aae300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c012780 tx=0x7f0a8c012c40 comp rx=0 tx=0).stop 2024-03-26T19:03:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.005+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 shutdown_connections 2024-03-26T19:03:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.006+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d452ab2e00 0x55d452ab5280 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.006+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452aa2900 0x55d452aae300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.006+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9d790 0x55d452aa72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.006+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/2518098229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452a9cda0 0x55d452aa6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.006+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 >> 172.21.15.97:0/2518098229 conn(0x55d452a98310 msgr2=0x55d452a97540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.006+0000 7f0a73e00640 0 client.25607 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.007+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 shutdown_connections 2024-03-26T19:03:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.007+0000 7f0a9bc26740 1 -- 172.21.15.97:0/2518098229 wait complete. 2024-03-26T19:03:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.008+0000 7f0a9bc26740 20 client.25607 trim_cache size 0 max 16384 2024-03-26T19:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.061+0000 7f0a9bc26740 1 Processor -- start 2024-03-26T19:03:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.063+0000 7f0a9bc26740 1 -- start start 2024-03-26T19:03:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9cda0 0x55d452a9e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 0x55d452aae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aa2900 0x55d452ab9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a9bc26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a941af3d0 con 0x55d452aa2900 2024-03-26T19:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a9bc26740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0a941b0f10 con 0x55d452a9d790 2024-03-26T19:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a9bc26740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a941afaa0 con 0x55d452a9cda0 2024-03-26T19:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a91e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9cda0 0x55d452a9e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a91400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 0x55d452aae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a91e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9cda0 0x55d452a9e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53930/0 (socket says 172.21.15.97:53930) 2024-03-26T19:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a91e00640 1 -- 172.21.15.97:0/1186704647 learned_addr learned my addr 172.21.15.97:0/1186704647 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a92800640 1 --2- 172.21.15.97:0/1186704647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aa2900 0x55d452ab9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a91e00640 1 -- 172.21.15.97:0/1186704647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 msgr2=0x55d452aae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a91e00640 1 --2- 172.21.15.97:0/1186704647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 0x55d452aae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a91e00640 1 -- 172.21.15.97:0/1186704647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aa2900 msgr2=0x55d452ab9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.064+0000 7f0a91e00640 1 --2- 172.21.15.97:0/1186704647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aa2900 0x55d452ab9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.065+0000 7f0a91e00640 1 -- 172.21.15.97:0/1186704647 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d452aa5e60 con 0x55d452a9cda0 2024-03-26T19:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.065+0000 7f0a92800640 1 --2- 172.21.15.97:0/1186704647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aa2900 0x55d452ab9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.065+0000 7f0a91e00640 1 --2- 172.21.15.97:0/1186704647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9cda0 0x55d452a9e940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0a840bdbd0 tx=0x7f0a84005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eca2abcba512dff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.065+0000 7f0a7ea00640 1 -- 172.21.15.97:0/1186704647 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a840c1d90 con 0x55d452a9cda0 2024-03-26T19:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.066+0000 7f0a7ea00640 1 -- 172.21.15.97:0/1186704647 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a840c6650 con 0x55d452a9cda0 2024-03-26T19:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.066+0000 7f0a7ea00640 1 -- 172.21.15.97:0/1186704647 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a840035f0 con 0x55d452a9cda0 2024-03-26T19:03:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.067+0000 7f0a9bc26740 1 -- 172.21.15.97:0/1186704647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9cda0 msgr2=0x55d452a9e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.067+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/1186704647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9cda0 0x55d452a9e940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0a840bdbd0 tx=0x7f0a84005750 comp rx=0 tx=0).stop 2024-03-26T19:03:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.067+0000 7f0a9bc26740 1 -- 172.21.15.97:0/1186704647 shutdown_connections 2024-03-26T19:03:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.067+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/1186704647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aa2900 0x55d452ab9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.067+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/1186704647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 0x55d452aae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.067+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/1186704647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452a9cda0 0x55d452a9e940 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.067+0000 7f0a9bc26740 1 -- 172.21.15.97:0/1186704647 >> 172.21.15.97:0/1186704647 conn(0x55d452a98310 msgr2=0x55d452a9e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.067+0000 7f0a9bc26740 1 -- 172.21.15.97:0/1186704647 shutdown_connections 2024-03-26T19:03:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.067+0000 7f0a9bc26740 1 -- 172.21.15.97:0/1186704647 wait complete. 2024-03-26T19:03:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.067+0000 7f0a9bc26740 1 -- start start 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a9bc26740 1 Processor -- start 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 0x55d452aac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452aa2900 0x55d452aac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a9bc26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aad750 0x55d452aadbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a9bc26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a941b0840 con 0x55d452aad750 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a9bc26740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0a941b0170 con 0x55d452a9d790 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a9bc26740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a941ad3b0 con 0x55d452aa2900 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a9bc26740 10 client.0 _ll_register_callbacks cb 0x7f0a8c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a91e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 0x55d452aac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a91400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452aa2900 0x55d452aac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a91e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 0x55d452aac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:45944/0 (socket says 172.21.15.97:45944) 2024-03-26T19:03:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a91e00640 1 -- 172.21.15.97:0/3892740052 learned_addr learned my addr 172.21.15.97:0/3892740052 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.068+0000 7f0a92800640 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aad750 0x55d452aadbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.069+0000 7f0a91400640 1 -- 172.21.15.97:0/3892740052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 msgr2=0x55d452aac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.069+0000 7f0a91400640 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 0x55d452aac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.069+0000 7f0a91400640 1 -- 172.21.15.97:0/3892740052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aad750 msgr2=0x55d452aadbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.069+0000 7f0a91400640 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aad750 0x55d452aadbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.069+0000 7f0a91400640 1 -- 172.21.15.97:0/3892740052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d452a98d80 con 0x55d452aa2900 2024-03-26T19:03:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.069+0000 7f0a92800640 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aad750 0x55d452aadbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:03:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.069+0000 7f0a91e00640 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 0x55d452aac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:03:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.070+0000 7f0a91400640 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452aa2900 0x55d452aac740 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0a80011010 tx=0x7f0a800107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b515df4a89afc80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.070+0000 7f0a73e00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:03:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.070+0000 7f0a73e00640 1 -- 172.21.15.97:0/3892740052 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80005610 con 0x55d452aa2900 2024-03-26T19:03:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.070+0000 7f0a73e00640 1 -- 172.21.15.97:0/3892740052 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0a80023070 con 0x55d452aa2900 2024-03-26T19:03:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.070+0000 7f0a9bc26740 1 -- 172.21.15.97:0/3892740052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a840c6650 con 0x55d452aa2900 2024-03-26T19:03:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.071+0000 7f0a73e00640 1 -- 172.21.15.97:0/3892740052 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a80004020 con 0x55d452aa2900 2024-03-26T19:03:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.071+0000 7f0a9bc26740 10 client.25576 fetch_fsmap learned FSMap version 27 2024-03-26T19:03:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.071+0000 7f0a9bc26740 1 -- 172.21.15.97:0/3892740052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d452a93730 con 0x55d452aa2900 2024-03-26T19:03:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.071+0000 7f0a73e00640 1 -- 172.21.15.97:0/3892740052 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a840c6650 con 0x55d452aa2900 2024-03-26T19:03:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.072+0000 7f0a73e00640 1 -- 172.21.15.97:0/3892740052 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0a800038c0 con 0x55d452aa2900 2024-03-26T19:03:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.072+0000 7f0a9bc26740 10 client.25576 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:03:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.072+0000 7f0a9bc26740 10 client.25576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:03:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.072+0000 7f0a9bc26740 20 client.25576 populate_metadata read hostname 'smithi097' 2024-03-26T19:03:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.072+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d452b1a0e0 0x55d452b1c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.072+0000 7f0a9bc26740 4 client.25576 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:03:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.072+0000 7f0a9bc26740 1 -- 172.21.15.97:0/3892740052 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d452a09710 con 0x55d452b1a0e0 2024-03-26T19:03:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.073+0000 7f0a91e00640 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d452b1a0e0 0x55d452b1c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.074+0000 7f0a91e00640 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d452b1a0e0 0x55d452b1c4d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=161cc4e31d116af3 server_cookie=76bfd5067d7b409 in_seq=0 out_seq=0 2024-03-26T19:03:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.074+0000 7f0a73e00640 10 client.25576 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.102+0000 7f0a73e00640 1 -- 172.21.15.97:0/3892740052 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d452a09710 con 0x55d452b1a0e0 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.102+0000 7f0a73e00640 10 client.25576 handle_command_reply: tid=0 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.102+0000 7f0a9bc26740 1 client.25576 shutdown 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.102+0000 7f0a9bc26740 1 -- 172.21.15.97:0/3892740052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452aa2900 msgr2=0x55d452aac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.102+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452aa2900 0x55d452aac740 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0a80011010 tx=0x7f0a800107a0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.103+0000 7f0a9bc26740 1 -- 172.21.15.97:0/3892740052 shutdown_connections 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.103+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d452b1a0e0 0x55d452b1c4d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.103+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d452aad750 0x55d452aadbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.103+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d452aa2900 0x55d452aac740 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.103+0000 7f0a73e00640 0 client.25576 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.103+0000 7f0a9bc26740 1 --2- 172.21.15.97:0/3892740052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d452a9d790 0x55d452aac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.103+0000 7f0a9bc26740 1 -- 172.21.15.97:0/3892740052 >> 172.21.15.97:0/3892740052 conn(0x55d452a98310 msgr2=0x55d452a97540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.103+0000 7f0a9bc26740 1 -- 172.21.15.97:0/3892740052 shutdown_connections 2024-03-26T19:03:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.103+0000 7f0a9bc26740 1 -- 172.21.15.97:0/3892740052 wait complete. 2024-03-26T19:03:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.103+0000 7f0a9bc26740 20 client.25576 trim_cache size 0 max 16384 2024-03-26T19:03:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:03:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:03:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:03:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:03:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 -- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0a780a30c0 msgr2=0x7f0a780a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0a780a30c0 0x7f0a780a5580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c00 tx=0x7f0a80013040 comp rx=0 tx=0).stop 2024-03-26T19:03:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 -- 172.21.15.97:0/614736298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a94107f20 msgr2=0x7f0a9419e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a94107f20 0x7f0a9419e150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0a84036040 tx=0x7f0a840040c0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 -- 172.21.15.97:0/614736298 shutdown_connections 2024-03-26T19:03:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0a780a30c0 0x7f0a780a5580 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a94109a30 0x7f0a941a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a94109070 0x7f0a9419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 --2- 172.21.15.97:0/614736298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a94107f20 0x7f0a9419e150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 -- 172.21.15.97:0/614736298 >> 172.21.15.97:0/614736298 conn(0x7f0a94076ed0 msgr2=0x7f0a9410c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 -- 172.21.15.97:0/614736298 shutdown_connections 2024-03-26T19:03:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:23.104+0000 7f0a98e00640 1 -- 172.21.15.97:0/614736298 wait complete. 2024-03-26T19:03:24.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:24 smithi097 ceph-mon[31955]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 750 op/s 2024-03-26T19:03:24.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:24 smithi144 ceph-mon[36053]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 750 op/s 2024-03-26T19:03:24.712 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:24 smithi105 ceph-mon[36270]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 750 op/s 2024-03-26T19:03:25.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 240.0 s, 1921.6 tps, lat 52.158 ms stddev 61.879 2024-03-26T19:03:26.212 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:25 smithi105 ceph-mon[36270]: pgmap v525: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 21 MiB/s wr, 874 op/s 2024-03-26T19:03:26.355 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:25 smithi097 ceph-mon[31955]: pgmap v525: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 21 MiB/s wr, 874 op/s 2024-03-26T19:03:26.378 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:25 smithi144 ceph-mon[36053]: pgmap v525: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 21 MiB/s wr, 874 op/s 2024-03-26T19:03:28.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:28 smithi105 ceph-mon[36270]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 787 op/s 2024-03-26T19:03:28.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:28 smithi097 ceph-mon[31955]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 787 op/s 2024-03-26T19:03:28.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:28 smithi144 ceph-mon[36053]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 787 op/s 2024-03-26T19:03:30.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 245.0 s, 2009.5 tps, lat 49.632 ms stddev 65.936 2024-03-26T19:03:30.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:30 smithi105 ceph-mon[36270]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 833 op/s 2024-03-26T19:03:30.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:30 smithi097 ceph-mon[31955]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 833 op/s 2024-03-26T19:03:30.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:30 smithi144 ceph-mon[36053]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 833 op/s 2024-03-26T19:03:32.407 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:32 smithi097 ceph-mon[31955]: pgmap v528: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 826 op/s 2024-03-26T19:03:32.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:32 smithi105 ceph-mon[36270]: pgmap v528: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 826 op/s 2024-03-26T19:03:32.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:32 smithi144 ceph-mon[36053]: pgmap v528: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 826 op/s 2024-03-26T19:03:34.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:34 smithi105 ceph-mon[36270]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 755 op/s 2024-03-26T19:03:34.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:34 smithi097 ceph-mon[31955]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 755 op/s 2024-03-26T19:03:34.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:34 smithi144 ceph-mon[36053]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 755 op/s 2024-03-26T19:03:35.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 250.0 s, 2174.4 tps, lat 46.065 ms stddev 57.229 2024-03-26T19:03:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.271+0000 7fbf09800640 1 -- 172.21.15.97:0/19542866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf04103b10 msgr2=0x7fbf04103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.271+0000 7fbf09800640 1 --2- 172.21.15.97:0/19542866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf04103b10 0x7fbf04103f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef402bc60 comp rx=0 tx=0).stop 2024-03-26T19:03:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.271+0000 7fbf09800640 1 -- 172.21.15.97:0/19542866 shutdown_connections 2024-03-26T19:03:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.271+0000 7fbf09800640 1 --2- 172.21.15.97:0/19542866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf041056c0 0x7fbf0410bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.271+0000 7fbf09800640 1 --2- 172.21.15.97:0/19542866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf04104d00 0x7fbf04105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.271+0000 7fbf09800640 1 --2- 172.21.15.97:0/19542866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf04103b10 0x7fbf04103f30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.271+0000 7fbf09800640 1 -- 172.21.15.97:0/19542866 >> 172.21.15.97:0/19542866 conn(0x7fbf040ff080 msgr2=0x7fbf041014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.271+0000 7fbf09800640 1 -- 172.21.15.97:0/19542866 shutdown_connections 2024-03-26T19:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.271+0000 7fbf09800640 1 -- 172.21.15.97:0/19542866 wait complete. 2024-03-26T19:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.272+0000 7fbf09800640 1 Processor -- start 2024-03-26T19:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.272+0000 7fbf09800640 1 -- start start 2024-03-26T19:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.272+0000 7fbf09800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf04103b10 0x7fbf04199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.272+0000 7fbf09800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf04104d00 0x7fbf0419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.272+0000 7fbf09800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf041056c0 0x7fbf041a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.272+0000 7fbf09800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0410f760 con 0x7fbf04103b10 2024-03-26T19:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf09800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0410f5e0 con 0x7fbf04104d00 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf09800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0410f8e0 con 0x7fbf041056c0 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf02000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf04104d00 0x7fbf0419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf02000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf04104d00 0x7fbf0419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:41206/0 (socket says 172.21.15.97:41206) 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf02000640 1 -- 172.21.15.97:0/3827927732 learned_addr learned my addr 172.21.15.97:0/3827927732 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf03400640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf041056c0 0x7fbf041a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf02a00640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf04103b10 0x7fbf04199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf02000640 1 -- 172.21.15.97:0/3827927732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf041056c0 msgr2=0x7fbf041a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf02000640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf041056c0 0x7fbf041a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf02000640 1 -- 172.21.15.97:0/3827927732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf04103b10 msgr2=0x7fbf04199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf02000640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf04103b10 0x7fbf04199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf02000640 1 -- 172.21.15.97:0/3827927732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf041a19b0 con 0x7fbf04104d00 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf03400640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf041056c0 0x7fbf041a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.273+0000 7fbf02000640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf04104d00 0x7fbf0419a2e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbef000e9f0 tx=0x7fbef000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.274+0000 7fbefb400640 1 -- 172.21.15.97:0/3827927732 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef000ccc0 con 0x7fbf04104d00 2024-03-26T19:03:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.274+0000 7fbf09800640 1 -- 172.21.15.97:0/3827927732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf041a1ca0 con 0x7fbf04104d00 2024-03-26T19:03:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.274+0000 7fbefb400640 1 -- 172.21.15.97:0/3827927732 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef0010910 con 0x7fbf04104d00 2024-03-26T19:03:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.274+0000 7fbefb400640 1 -- 172.21.15.97:0/3827927732 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00188a0 con 0x7fbf04104d00 2024-03-26T19:03:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.274+0000 7fbf09800640 1 -- 172.21.15.97:0/3827927732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041a21c0 con 0x7fbf04104d00 2024-03-26T19:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.278+0000 7fbefb400640 1 -- 172.21.15.97:0/3827927732 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fbef0010ac0 con 0x7fbf04104d00 2024-03-26T19:03:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.278+0000 7fbf0c580740 1 Processor -- start 2024-03-26T19:03:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.279+0000 7fbf0c580740 1 -- start start 2024-03-26T19:03:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.279+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.279+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c4790 0x5648215c9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.279+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.279+0000 7fbf0c580740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564821528e80 con 0x5648215c9900 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.279+0000 7fbf0c580740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5648214d1660 con 0x5648215c3da0 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.279+0000 7fbf0c580740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564821524e30 con 0x5648215c4790 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.279+0000 7fbf03400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.279+0000 7fbf03400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:41212/0 (socket says 172.21.15.97:41212) 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.279+0000 7fbf03400640 1 -- 172.21.15.97:0/2764812040 learned_addr learned my addr 172.21.15.97:0/2764812040 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbf02a00640 1 --2- 172.21.15.97:0/2764812040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c4790 0x5648215c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbefb400640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fbee80a3040 0x7fbee80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbf02000640 1 --2- 172.21.15.97:0/2764812040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbefb400640 1 -- 172.21.15.97:0/3827927732 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fbef00cf840 con 0x7fbf04104d00 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbf03400640 1 -- 172.21.15.97:0/2764812040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c4790 msgr2=0x5648215c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbf03400640 1 --2- 172.21.15.97:0/2764812040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c4790 0x5648215c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbf03400640 1 -- 172.21.15.97:0/2764812040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 msgr2=0x5648215cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbf03400640 1 --2- 172.21.15.97:0/2764812040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbf03400640 1 -- 172.21.15.97:0/2764812040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56482123d210 con 0x5648215c3da0 2024-03-26T19:03:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbf02000640 1 --2- 172.21.15.97:0/2764812040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:03:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbf03400640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fbee80a3040 0x7fbee80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.280+0000 7fbf03400640 1 --2- 172.21.15.97:0/2764812040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215c41c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbefc011990 tx=0x7fbefc011e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd2f11f1c5ccd533 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.281+0000 7fbeefe00640 1 -- 172.21.15.97:0/2764812040 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc017050 con 0x5648215c3da0 2024-03-26T19:03:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.281+0000 7fbeefe00640 1 -- 172.21.15.97:0/2764812040 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbefc00fcc0 con 0x5648215c3da0 2024-03-26T19:03:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.281+0000 7fbeefe00640 1 -- 172.21.15.97:0/2764812040 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc010970 con 0x5648215c3da0 2024-03-26T19:03:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.281+0000 7fbf0c580740 1 -- 172.21.15.97:0/2764812040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 msgr2=0x5648215c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.281+0000 7fbf03400640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fbee80a3040 0x7fbee80a5500 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbefc021000 tx=0x7fbefc010b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.281+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2764812040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215c41c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbefc011990 tx=0x7fbefc011e50 comp rx=0 tx=0).stop 2024-03-26T19:03:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.282+0000 7fbf0c580740 1 -- 172.21.15.97:0/2764812040 shutdown_connections 2024-03-26T19:03:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.282+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2764812040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.282+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2764812040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c4790 0x5648215c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.282+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2764812040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215c41c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.282+0000 7fbf0c580740 1 -- 172.21.15.97:0/2764812040 >> 172.21.15.97:0/2764812040 conn(0x5648215bf310 msgr2=0x5648215bf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.282+0000 7fbf0c580740 1 -- 172.21.15.97:0/2764812040 shutdown_connections 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.282+0000 7fbf0c580740 1 -- 172.21.15.97:0/2764812040 wait complete. 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.282+0000 7fbf0c580740 1 -- start start 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.283+0000 7fbf0c580740 1 Processor -- start 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.283+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215d6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.283+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c4790 0x5648215d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.283+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215ddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.283+0000 7fbf0c580740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5648215cc8b0 con 0x5648215c9900 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.283+0000 7fbf0c580740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5648214e4ac0 con 0x5648215c3da0 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.283+0000 7fbf0c580740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5648215cca30 con 0x5648215c4790 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.283+0000 7fbf0c580740 10 client.0 _ll_register_callbacks cb 0x5648214e4490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.283+0000 7fbf02a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215d6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.283+0000 7fbf02a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215d6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:41214/0 (socket says 172.21.15.97:41214) 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.284+0000 7fbf02a00640 1 -- 172.21.15.97:0/3729865296 learned_addr learned my addr 172.21.15.97:0/3729865296 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.284+0000 7fbf03400640 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215ddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.284+0000 7fbf02a00640 1 -- 172.21.15.97:0/3729865296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c4790 msgr2=0x5648215d6a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:03:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.284+0000 7fbf02a00640 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c4790 0x5648215d6a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.284+0000 7fbf02a00640 1 -- 172.21.15.97:0/3729865296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 msgr2=0x5648215ddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.284+0000 7fbf02a00640 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215ddad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.284+0000 7fbf02a00640 1 -- 172.21.15.97:0/3729865296 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564821309620 con 0x5648215c3da0 2024-03-26T19:03:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.284+0000 7fbf03400640 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:03:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.285+0000 7fbf02a00640 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215d6550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fd2580575d8f671 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.285+0000 7fbeecc00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:03:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.285+0000 7fbeecc00640 1 -- 172.21.15.97:0/3729865296 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4005d50 con 0x5648215c3da0 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.285+0000 7fbeecc00640 1 -- 172.21.15.97:0/3729865296 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fbef4005030 con 0x5648215c3da0 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.285+0000 7fbf0c580740 1 -- 172.21.15.97:0/3729865296 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbefc00fcc0 con 0x5648215c3da0 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.285+0000 7fbeecc00640 1 -- 172.21.15.97:0/3729865296 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbef4050020 con 0x5648215c3da0 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.285+0000 7fbf0c580740 10 client.25625 fetch_fsmap learned FSMap version 27 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.285+0000 7fbf0c580740 1 -- 172.21.15.97:0/3729865296 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648215e0e20 con 0x5648215c3da0 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.286+0000 7fbeecc00640 1 -- 172.21.15.97:0/3729865296 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbefc00fcc0 con 0x5648215c3da0 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.286+0000 7fbeecc00640 1 -- 172.21.15.97:0/3729865296 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbef4003710 con 0x5648215c3da0 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.286+0000 7fbf0c580740 10 client.25625 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.286+0000 7fbf0c580740 10 client.25625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.286+0000 7fbf0c580740 20 client.25625 populate_metadata read hostname 'smithi097' 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.286+0000 7fbf0c580740 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5648215e2550 0x5648215e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.286+0000 7fbf0c580740 4 client.25625 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.286+0000 7fbf0c580740 1 -- 172.21.15.97:0/3729865296 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648215363c0 con 0x5648215e2550 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.286+0000 7fbf02000640 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5648215e2550 0x5648215e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.287+0000 7fbf02000640 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5648215e2550 0x5648215e49e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9997218abbdc3c45 server_cookie=133eca0f318c6aba in_seq=0 out_seq=0 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.287+0000 7fbeecc00640 10 client.25625 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbeecc00640 1 -- 172.21.15.97:0/3729865296 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5648215363c0 con 0x5648215e2550 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbeecc00640 10 client.25625 handle_command_reply: tid=0 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 client.25625 shutdown 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 -- 172.21.15.97:0/3729865296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 msgr2=0x5648215d6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215d6550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef4004270 comp rx=0 tx=0).stop 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 -- 172.21.15.97:0/3729865296 shutdown_connections 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5648215e2550 0x5648215e49e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c4790 0x5648215d6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 --2- 172.21.15.97:0/3729865296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c3da0 0x5648215d6550 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 -- 172.21.15.97:0/3729865296 >> 172.21.15.97:0/3729865296 conn(0x5648215bf310 msgr2=0x5648215be540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbeecc00640 0 client.25625 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 -- 172.21.15.97:0/3729865296 shutdown_connections 2024-03-26T19:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.288+0000 7fbf0c580740 1 -- 172.21.15.97:0/3729865296 wait complete. 2024-03-26T19:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.289+0000 7fbf0c580740 20 client.25625 trim_cache size 0 max 16384 2024-03-26T19:03:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.319+0000 7fbf0c580740 1 Processor -- start 2024-03-26T19:03:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.322+0000 7fbf0c580740 1 -- start start 2024-03-26T19:03:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.322+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c3da0 0x5648215d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.322+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 0x5648215dda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.322+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215d60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.322+0000 7fbf0c580740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a6b40 con 0x5648215c9900 2024-03-26T19:03:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.322+0000 7fbf0c580740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a8680 con 0x5648215c4790 2024-03-26T19:03:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.322+0000 7fbf0c580740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a7210 con 0x5648215c3da0 2024-03-26T19:03:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.323+0000 7fbf02000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 0x5648215dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.323+0000 7fbf03400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215d60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.324+0000 7fbf03400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215d60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60398/0 (socket says 172.21.15.97:60398) 2024-03-26T19:03:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.324+0000 7fbf03400640 1 -- 172.21.15.97:0/2018789606 learned_addr learned my addr 172.21.15.97:0/2018789606 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.325+0000 7fbf02000640 1 -- 172.21.15.97:0/2018789606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c3da0 msgr2=0x5648215d5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.325+0000 7fbf02000640 1 --2- 172.21.15.97:0/2018789606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c3da0 0x5648215d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.325+0000 7fbf02000640 1 -- 172.21.15.97:0/2018789606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 msgr2=0x5648215d60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.325+0000 7fbf02000640 1 --2- 172.21.15.97:0/2018789606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215d60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.325+0000 7fbf02000640 1 -- 172.21.15.97:0/2018789606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648215ba730 con 0x5648215c4790 2024-03-26T19:03:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.325+0000 7fbf02000640 1 --2- 172.21.15.97:0/2018789606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 0x5648215dda90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbef0098880 tx=0x7fbef0009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b19dd9411dcfd32b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.325+0000 7fbeef400640 1 -- 172.21.15.97:0/2018789606 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00a4950 con 0x5648215c4790 2024-03-26T19:03:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.325+0000 7fbeef400640 1 -- 172.21.15.97:0/2018789606 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef009a240 con 0x5648215c4790 2024-03-26T19:03:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.325+0000 7fbeef400640 1 -- 172.21.15.97:0/2018789606 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef009b390 con 0x5648215c4790 2024-03-26T19:03:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.326+0000 7fbf0c580740 1 -- 172.21.15.97:0/2018789606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 msgr2=0x5648215dda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.327+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2018789606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 0x5648215dda90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbef0098880 tx=0x7fbef0009700 comp rx=0 tx=0).stop 2024-03-26T19:03:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.327+0000 7fbf0c580740 1 -- 172.21.15.97:0/2018789606 shutdown_connections 2024-03-26T19:03:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.327+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2018789606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c9900 0x5648215d60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.327+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2018789606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 0x5648215dda90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.327+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2018789606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c3da0 0x5648215d5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.327+0000 7fbf0c580740 1 -- 172.21.15.97:0/2018789606 >> 172.21.15.97:0/2018789606 conn(0x5648215bf310 msgr2=0x5648215c2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.328+0000 7fbf0c580740 1 -- 172.21.15.97:0/2018789606 shutdown_connections 2024-03-26T19:03:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.328+0000 7fbf0c580740 1 -- 172.21.15.97:0/2018789606 wait complete. 2024-03-26T19:03:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.328+0000 7fbf0c580740 1 -- start start 2024-03-26T19:03:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.329+0000 7fbf0c580740 1 Processor -- start 2024-03-26T19:03:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.330+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c3da0 0x5648215d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.331+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 0x5648215d9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.331+0000 7fbf0c580740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c9900 0x5648215dc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.331+0000 7fbf02a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c3da0 0x5648215d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.331+0000 7fbf02a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c3da0 0x5648215d93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60414/0 (socket says 172.21.15.97:60414) 2024-03-26T19:03:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.331+0000 7fbf03400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c9900 0x5648215dc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.332+0000 7fbf03400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c9900 0x5648215dc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:40538/0 (socket says 172.21.15.97:40538) 2024-03-26T19:03:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.332+0000 7fbf03400640 1 -- 172.21.15.97:0/2950547278 learned_addr learned my addr 172.21.15.97:0/2950547278 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.332+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a7fb0 con 0x5648215c3da0 2024-03-26T19:03:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.332+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a78e0 con 0x5648215c4790 2024-03-26T19:03:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.332+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a4b40 con 0x5648215c9900 2024-03-26T19:03:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.332+0000 7fbf0c580740 10 client.0 _ll_register_callbacks cb 0x7fbef40095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:03:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.331+0000 7fbf02000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 0x5648215d9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.335+0000 7fbf03400640 1 -- 172.21.15.97:0/2950547278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 msgr2=0x5648215d9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.335+0000 7fbf03400640 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 0x5648215d9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.335+0000 7fbf03400640 1 -- 172.21.15.97:0/2950547278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c3da0 msgr2=0x5648215d93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.335+0000 7fbf03400640 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c3da0 0x5648215d93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.335+0000 7fbf03400640 1 -- 172.21.15.97:0/2950547278 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648215bfd80 con 0x5648215c9900 2024-03-26T19:03:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.337+0000 7fbf02000640 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 0x5648215d9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:03:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.338+0000 7fbf03400640 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c9900 0x5648215dc8d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00fff0 tx=0x7fbefc018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f04e2488f34c733b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.338+0000 7fbeecc00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:03:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.342+0000 7fbeecc00640 1 -- 172.21.15.97:0/2950547278 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc004b80 con 0x5648215c9900 2024-03-26T19:03:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.342+0000 7fbeecc00640 1 -- 172.21.15.97:0/2950547278 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fbefc004d20 con 0x5648215c9900 2024-03-26T19:03:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.342+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef009a240 con 0x5648215c9900 2024-03-26T19:03:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.343+0000 7fbeecc00640 1 -- 172.21.15.97:0/2950547278 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbefc009820 con 0x5648215c9900 2024-03-26T19:03:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.344+0000 7fbf0c580740 10 client.25588 fetch_fsmap learned FSMap version 27 2024-03-26T19:03:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.344+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648215d0df0 con 0x5648215c9900 2024-03-26T19:03:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.344+0000 7fbeecc00640 1 -- 172.21.15.97:0/2950547278 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef009a240 con 0x5648215c9900 2024-03-26T19:03:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.344+0000 7fbeecc00640 1 -- 172.21.15.97:0/2950547278 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbefc010760 con 0x5648215c9900 2024-03-26T19:03:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.344+0000 7fbf0c580740 10 client.25588 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:03:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.344+0000 7fbf0c580740 10 client.25588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:03:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.344+0000 7fbf0c580740 20 client.25588 populate_metadata read hostname 'smithi097' 2024-03-26T19:03:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.344+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5648215d14f0 0x5648215d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.344+0000 7fbf0c580740 4 client.25588 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:03:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.344+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564821530710 con 0x5648215d14f0 2024-03-26T19:03:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.345+0000 7fbf02a00640 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5648215d14f0 0x5648215d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.347+0000 7fbeecc00640 10 client.25588 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.347+0000 7fbf02a00640 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5648215d14f0 0x5648215d3910 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be9e0ec1fb51caf server_cookie=3d8c0556713590ef in_seq=0 out_seq=0 2024-03-26T19:03:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbeecc00640 1 -- 172.21.15.97:0/2950547278 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564821530710 con 0x5648215d14f0 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbeecc00640 10 client.25588 handle_command_reply: tid=0 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 client.25588 shutdown 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c9900 msgr2=0x5648215dc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c9900 0x5648215dc8d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00fff0 tx=0x7fbefc018000 comp rx=0 tx=0).stop 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 shutdown_connections 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5648215d14f0 0x5648215d3910 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648215c9900 0x5648215dc8d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648215c4790 0x5648215d9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 --2- 172.21.15.97:0/2950547278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648215c3da0 0x5648215d93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbeecc00640 0 client.25588 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 >> 172.21.15.97:0/2950547278 conn(0x5648215bf310 msgr2=0x5648215c5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 shutdown_connections 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.384+0000 7fbf0c580740 1 -- 172.21.15.97:0/2950547278 wait complete. 2024-03-26T19:03:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf0c580740 20 client.25588 trim_cache size 0 max 16384 2024-03-26T19:03:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:03:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:03:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:03:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 -- 172.21.15.97:0/3827927732 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fbee80a3040 msgr2=0x7fbee80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fbee80a3040 0x7fbee80a5500 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbefc021000 tx=0x7fbefc010b70 comp rx=0 tx=0).stop 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 -- 172.21.15.97:0/3827927732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf04104d00 msgr2=0x7fbf0419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf04104d00 0x7fbf0419a2e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbef000e9f0 tx=0x7fbef000edf0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 -- 172.21.15.97:0/3827927732 shutdown_connections 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fbee80a3040 0x7fbee80a5500 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf041056c0 0x7fbf041a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf04104d00 0x7fbf0419a2e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 --2- 172.21.15.97:0/3827927732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf04103b10 0x7fbf04199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 -- 172.21.15.97:0/3827927732 >> 172.21.15.97:0/3827927732 conn(0x7fbf040ff080 msgr2=0x7fbf04108000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 -- 172.21.15.97:0/3827927732 shutdown_connections 2024-03-26T19:03:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:36.385+0000 7fbf09800640 1 -- 172.21.15.97:0/3827927732 wait complete. 2024-03-26T19:03:36.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:36 smithi105 ceph-mon[36270]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 MiB/s wr, 799 op/s 2024-03-26T19:03:36.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:36 smithi097 ceph-mon[31955]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 MiB/s wr, 799 op/s 2024-03-26T19:03:36.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:36 smithi144 ceph-mon[36053]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 MiB/s wr, 799 op/s 2024-03-26T19:03:38.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:38 smithi105 ceph-mon[36270]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 680 op/s 2024-03-26T19:03:38.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:38 smithi097 ceph-mon[31955]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 680 op/s 2024-03-26T19:03:38.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:38 smithi144 ceph-mon[36053]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 680 op/s 2024-03-26T19:03:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 255.0 s, 2104.4 tps, lat 47.524 ms stddev 60.641 2024-03-26T19:03:40.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:40 smithi105 ceph-mon[36270]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 MiB/s wr, 774 op/s 2024-03-26T19:03:40.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:40 smithi097 ceph-mon[31955]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 MiB/s wr, 774 op/s 2024-03-26T19:03:40.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:40 smithi144 ceph-mon[36053]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 MiB/s wr, 774 op/s 2024-03-26T19:03:42.431 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:42 smithi144 ceph-mon[36053]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 MiB/s wr, 713 op/s 2024-03-26T19:03:42.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:42 smithi105 ceph-mon[36270]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 MiB/s wr, 713 op/s 2024-03-26T19:03:42.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:42 smithi097 ceph-mon[31955]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 MiB/s wr, 713 op/s 2024-03-26T19:03:44.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:44 smithi105 ceph-mon[36270]: pgmap v534: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 683 op/s 2024-03-26T19:03:44.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:44 smithi097 ceph-mon[31955]: pgmap v534: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 683 op/s 2024-03-26T19:03:44.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:44 smithi144 ceph-mon[36053]: pgmap v534: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 683 op/s 2024-03-26T19:03:45.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 260.0 s, 2550.6 tps, lat 39.237 ms stddev 47.469 2024-03-26T19:03:46.462 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:46 smithi105 ceph-mon[36270]: pgmap v535: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 748 op/s 2024-03-26T19:03:46.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:46 smithi097 ceph-mon[31955]: pgmap v535: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 748 op/s 2024-03-26T19:03:46.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:46 smithi144 ceph-mon[36053]: pgmap v535: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 748 op/s 2024-03-26T19:03:48.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:48 smithi105 ceph-mon[36270]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 MiB/s wr, 647 op/s 2024-03-26T19:03:48.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:48 smithi097 ceph-mon[31955]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 MiB/s wr, 647 op/s 2024-03-26T19:03:48.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:48 smithi144 ceph-mon[36053]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 MiB/s wr, 647 op/s 2024-03-26T19:03:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.589+0000 7fb2aaa00640 1 -- 172.21.15.97:0/3189114263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a4104d00 msgr2=0x7fb2a4105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.589+0000 7fb2aaa00640 1 --2- 172.21.15.97:0/3189114263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a4104d00 0x7fb2a4105180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb290009900 tx=0x7fb29002bc30 comp rx=0 tx=0).stop 2024-03-26T19:03:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.590+0000 7fb2aaa00640 1 -- 172.21.15.97:0/3189114263 shutdown_connections 2024-03-26T19:03:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.590+0000 7fb2aaa00640 1 --2- 172.21.15.97:0/3189114263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a41056c0 0x7fb2a410bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.590+0000 7fb2aaa00640 1 --2- 172.21.15.97:0/3189114263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a4104d00 0x7fb2a4105180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.590+0000 7fb2aaa00640 1 --2- 172.21.15.97:0/3189114263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2a4103b10 0x7fb2a4103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.590+0000 7fb2aaa00640 1 -- 172.21.15.97:0/3189114263 >> 172.21.15.97:0/3189114263 conn(0x7fb2a40ff080 msgr2=0x7fb2a41014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.590+0000 7fb2aaa00640 1 -- 172.21.15.97:0/3189114263 shutdown_connections 2024-03-26T19:03:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.591+0000 7fb2aaa00640 1 -- 172.21.15.97:0/3189114263 wait complete. 2024-03-26T19:03:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.592+0000 7fb2aaa00640 1 Processor -- start 2024-03-26T19:03:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.592+0000 7fb2aaa00640 1 -- start start 2024-03-26T19:03:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.592+0000 7fb2aaa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4103b10 0x7fb2a4079670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.592+0000 7fb2aaa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a4104d00 0x7fb2a4079bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.592+0000 7fb2aaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2a41056c0 0x7fb2a40761e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.592+0000 7fb2aaa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a410f7e0 con 0x7fb2a41056c0 2024-03-26T19:03:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.592+0000 7fb2aaa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a410f660 con 0x7fb2a4103b10 2024-03-26T19:03:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.592+0000 7fb2aaa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a410f960 con 0x7fb2a4104d00 2024-03-26T19:03:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.593+0000 7fb2a3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a4104d00 0x7fb2a4079bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.593+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4103b10 0x7fb2a4079670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.593+0000 7fb2a3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a4104d00 0x7fb2a4079bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:48280/0 (socket says 172.21.15.97:48280) 2024-03-26T19:03:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.593+0000 7fb2a3400640 1 -- 172.21.15.97:0/4078570500 learned_addr learned my addr 172.21.15.97:0/4078570500 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.593+0000 7fb2a3400640 1 -- 172.21.15.97:0/4078570500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4103b10 msgr2=0x7fb2a4079670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.593+0000 7fb2a3400640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4103b10 0x7fb2a4079670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.593+0000 7fb2a3400640 1 -- 172.21.15.97:0/4078570500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2a41056c0 msgr2=0x7fb2a40761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.593+0000 7fb2a3400640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2a41056c0 0x7fb2a40761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.593+0000 7fb2a3400640 1 -- 172.21.15.97:0/4078570500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a4076aa0 con 0x7fb2a4104d00 2024-03-26T19:03:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.596+0000 7fb2a3400640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a4104d00 0x7fb2a4079bb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb29002c240 tx=0x7fb290004e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.596+0000 7fb2a0c00640 1 -- 172.21.15.97:0/4078570500 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290043020 con 0x7fb2a4104d00 2024-03-26T19:03:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.596+0000 7fb2a0c00640 1 -- 172.21.15.97:0/4078570500 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2900054e0 con 0x7fb2a4104d00 2024-03-26T19:03:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.596+0000 7fb2a0c00640 1 -- 172.21.15.97:0/4078570500 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290003d00 con 0x7fb2a4104d00 2024-03-26T19:03:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.596+0000 7fb2aaa00640 1 -- 172.21.15.97:0/4078570500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a41a60f0 con 0x7fb2a4104d00 2024-03-26T19:03:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.596+0000 7fb2aaa00640 1 -- 172.21.15.97:0/4078570500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a41a64f0 con 0x7fb2a4104d00 2024-03-26T19:03:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.601+0000 7fb2a0c00640 1 -- 172.21.15.97:0/4078570500 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fb290046020 con 0x7fb2a4104d00 2024-03-26T19:03:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.601+0000 7fb2ad5f1740 1 Processor -- start 2024-03-26T19:03:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.602+0000 7fb2ad5f1740 1 -- start start 2024-03-26T19:03:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.602+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 0x55d61af091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.602+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af0e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.602+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af0e900 0x55d61af10d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.602+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d61ae6de80 con 0x55d61af09790 2024-03-26T19:03:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.602+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d61ae16660 con 0x55d61af08da0 2024-03-26T19:03:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.602+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d61ae69e30 con 0x55d61af0e900 2024-03-26T19:03:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.603+0000 7fb2a8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 0x55d61af091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.604+0000 7fb2a8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 0x55d61af091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:40374/0 (socket says 172.21.15.97:40374) 2024-03-26T19:03:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.604+0000 7fb2a8a00640 1 -- 172.21.15.97:0/2934507179 learned_addr learned my addr 172.21.15.97:0/2934507179 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.604+0000 7fb2a8a00640 1 -- 172.21.15.97:0/2934507179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af0e900 msgr2=0x55d61af10d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:03:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.604+0000 7fb2a8a00640 1 --2- 172.21.15.97:0/2934507179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af0e900 0x55d61af10d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.604+0000 7fb2a8a00640 1 -- 172.21.15.97:0/2934507179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 msgr2=0x55d61af0e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.604+0000 7fb2a8a00640 1 --2- 172.21.15.97:0/2934507179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af0e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.604+0000 7fb2a8a00640 1 -- 172.21.15.97:0/2934507179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d61ab82210 con 0x55d61af08da0 2024-03-26T19:03:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.604+0000 7fb2a8a00640 1 --2- 172.21.15.97:0/2934507179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 0x55d61af091c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009ea0 tx=0x7fb29c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8eda5e370991f53 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.604+0000 7fb2a0c00640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb2880a3090 0x7fb2880a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.605+0000 7fb2a0c00640 1 -- 172.21.15.97:0/4078570500 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb2900f1550 con 0x7fb2a4104d00 2024-03-26T19:03:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.605+0000 7fb299200640 1 -- 172.21.15.97:0/2934507179 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c004d20 con 0x55d61af08da0 2024-03-26T19:03:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.605+0000 7fb299200640 1 -- 172.21.15.97:0/2934507179 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29c00ee60 con 0x55d61af08da0 2024-03-26T19:03:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.606+0000 7fb2a3e00640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb2880a3090 0x7fb2880a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.606+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/2934507179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 msgr2=0x55d61af091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.606+0000 7fb299200640 1 -- 172.21.15.97:0/2934507179 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c004d20 con 0x55d61af08da0 2024-03-26T19:03:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.606+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/2934507179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 0x55d61af091c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009ea0 tx=0x7fb29c009f90 comp rx=0 tx=0).stop 2024-03-26T19:03:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.606+0000 7fb2a3e00640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb2880a3090 0x7fb2880a5550 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55d61af09b90 tx=0x7fb29400a750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.607+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/2934507179 shutdown_connections 2024-03-26T19:03:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.608+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/2934507179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af0e900 0x55d61af10d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.608+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/2934507179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af0e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.608+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/2934507179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 0x55d61af091c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.608+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/2934507179 >> 172.21.15.97:0/2934507179 conn(0x55d61af04310 msgr2=0x55d61af04710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.608+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/2934507179 shutdown_connections 2024-03-26T19:03:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.608+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/2934507179 wait complete. 2024-03-26T19:03:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.608+0000 7fb2ad5f1740 1 -- start start 2024-03-26T19:03:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.609+0000 7fb2ad5f1740 1 Processor -- start 2024-03-26T19:03:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.609+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 0x55d61af12df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.609+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af09790 0x55d61af13330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.609+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af0e900 0x55d61af1a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.609+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d61af11900 con 0x55d61af0e900 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.609+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d61ae29ac0 con 0x55d61af09790 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.609+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d61af11a80 con 0x55d61af08da0 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.609+0000 7fb2ad5f1740 10 client.0 _ll_register_callbacks cb 0x55d61ae29490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 0x55d61af12df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 0x55d61af12df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:48296/0 (socket says 172.21.15.97:48296) 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a3e00640 1 -- 172.21.15.97:0/392336222 learned_addr learned my addr 172.21.15.97:0/392336222 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a3400640 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af09790 0x55d61af13330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a8a00640 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af0e900 0x55d61af1a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a3400640 1 -- 172.21.15.97:0/392336222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 msgr2=0x55d61af12df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a3400640 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 0x55d61af12df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a3400640 1 -- 172.21.15.97:0/392336222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af0e900 msgr2=0x55d61af1a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a3400640 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af0e900 0x55d61af1a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a3400640 1 -- 172.21.15.97:0/392336222 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d61ac4e620 con 0x55d61af09790 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb2a3400640 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af09790 0x55d61af13330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb2900bd950 tx=0x7fb290005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7baa4ec165898f87 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.610+0000 7fb28d600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb28d600640 1 -- 172.21.15.97:0/392336222 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290004c80 con 0x55d61af09790 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb28d600640 1 -- 172.21.15.97:0/392336222 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb290004310 con 0x55d61af09790 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/392336222 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29c00ee60 con 0x55d61af09790 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb28d600640 1 -- 172.21.15.97:0/392336222 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb2900035f0 con 0x55d61af09790 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb2ad5f1740 10 client.25643 fetch_fsmap learned FSMap version 27 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/392336222 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d61af1d4d0 con 0x55d61af09790 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb28d600640 1 -- 172.21.15.97:0/392336222 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29c00ee60 con 0x55d61af09790 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb28d600640 1 -- 172.21.15.97:0/392336222 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb2900f10f0 con 0x55d61af09790 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb2ad5f1740 10 client.25643 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb2ad5f1740 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb2ad5f1740 20 client.25643 populate_metadata read hostname 'smithi097' 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d61af1eb60 0x55d61af20f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb2ad5f1740 4 client.25643 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:03:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.611+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/392336222 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d61ae7b3c0 con 0x55d61af1eb60 2024-03-26T19:03:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.612+0000 7fb2a3e00640 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d61af1eb60 0x55d61af20f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.612+0000 7fb2a3e00640 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d61af1eb60 0x55d61af20f50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8c9a55959ea073b server_cookie=8f552675014d5b63 in_seq=0 out_seq=0 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.612+0000 7fb28d600640 10 client.25643 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.613+0000 7fb28d600640 1 -- 172.21.15.97:0/392336222 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d61ae7b3c0 con 0x55d61af1eb60 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.613+0000 7fb28d600640 10 client.25643 handle_command_reply: tid=0 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.613+0000 7fb2ad5f1740 1 client.25643 shutdown 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.614+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/392336222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af09790 msgr2=0x55d61af13330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.614+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af09790 0x55d61af13330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb2900bd950 tx=0x7fb290005fd0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.614+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/392336222 shutdown_connections 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.614+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d61af1eb60 0x55d61af20f50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.614+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af0e900 0x55d61af1a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.614+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af09790 0x55d61af13330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.614+0000 7fb28d600640 0 client.25643 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.614+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/392336222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 0x55d61af12df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.614+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/392336222 >> 172.21.15.97:0/392336222 conn(0x55d61af04310 msgr2=0x55d61af03540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.614+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/392336222 shutdown_connections 2024-03-26T19:03:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.615+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/392336222 wait complete. 2024-03-26T19:03:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.615+0000 7fb2ad5f1740 20 client.25643 trim_cache size 0 max 16384 2024-03-26T19:03:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.632+0000 7fb2ad5f1740 1 Processor -- start 2024-03-26T19:03:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.633+0000 7fb2ad5f1740 1 -- start start 2024-03-26T19:03:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.633+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 0x55d61af02cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.633+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af1a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.633+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af0e900 0x55d61af031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.633+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41af110 con 0x55d61af09790 2024-03-26T19:03:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.633+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41b0c50 con 0x55d61af08da0 2024-03-26T19:03:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.633+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41af7e0 con 0x55d61af0e900 2024-03-26T19:03:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.634+0000 7fb2a3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af1a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.634+0000 7fb2a3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af1a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57160/0 (socket says 172.21.15.97:57160) 2024-03-26T19:03:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.634+0000 7fb2a3400640 1 -- 172.21.15.97:0/1406742400 learned_addr learned my addr 172.21.15.97:0/1406742400 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.634+0000 7fb2a8a00640 1 --2- 172.21.15.97:0/1406742400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af0e900 0x55d61af031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.635+0000 7fb2a3400640 1 -- 172.21.15.97:0/1406742400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af0e900 msgr2=0x55d61af031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.635+0000 7fb2a3400640 1 --2- 172.21.15.97:0/1406742400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af0e900 0x55d61af031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.635+0000 7fb2a3400640 1 -- 172.21.15.97:0/1406742400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 msgr2=0x55d61af02cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:03:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.635+0000 7fb2a3400640 1 --2- 172.21.15.97:0/1406742400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 0x55d61af02cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.635+0000 7fb2a3400640 1 -- 172.21.15.97:0/1406742400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d61af21580 con 0x55d61af09790 2024-03-26T19:03:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.636+0000 7fb2a3400640 1 --2- 172.21.15.97:0/1406742400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af1a330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb290009850 tx=0x7fb2900bc160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cc51afcedfc8a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.636+0000 7fb28fe00640 1 -- 172.21.15.97:0/1406742400 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900a1070 con 0x55d61af09790 2024-03-26T19:03:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.637+0000 7fb28fe00640 1 -- 172.21.15.97:0/1406742400 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb290005690 con 0x55d61af09790 2024-03-26T19:03:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.637+0000 7fb28fe00640 1 -- 172.21.15.97:0/1406742400 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900f10f0 con 0x55d61af09790 2024-03-26T19:03:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.638+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/1406742400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 msgr2=0x55d61af1a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.638+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/1406742400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af1a330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb290009850 tx=0x7fb2900bc160 comp rx=0 tx=0).stop 2024-03-26T19:03:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.638+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/1406742400 shutdown_connections 2024-03-26T19:03:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.638+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/1406742400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af0e900 0x55d61af031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.638+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/1406742400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af1a330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.638+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/1406742400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af08da0 0x55d61af02cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.638+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/1406742400 >> 172.21.15.97:0/1406742400 conn(0x55d61af04310 msgr2=0x55d61af07e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.638+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/1406742400 shutdown_connections 2024-03-26T19:03:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.638+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/1406742400 wait complete. 2024-03-26T19:03:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.639+0000 7fb2ad5f1740 1 -- start start 2024-03-26T19:03:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.639+0000 7fb2ad5f1740 1 Processor -- start 2024-03-26T19:03:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.640+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 0x55d61af17db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.640+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af182f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.640+0000 7fb2ad5f1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af0e900 0x55d61af19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.640+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41b0580 con 0x55d61af09790 2024-03-26T19:03:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.640+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41afeb0 con 0x55d61af0e900 2024-03-26T19:03:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.640+0000 7fb2ad5f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41ad0f0 con 0x55d61af08da0 2024-03-26T19:03:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.640+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 0x55d61af17db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.640+0000 7fb2a8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af0e900 0x55d61af19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.640+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 0x55d61af17db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:48316/0 (socket says 172.21.15.97:48316) 2024-03-26T19:03:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.640+0000 7fb2a3e00640 1 -- 172.21.15.97:0/3954292836 learned_addr learned my addr 172.21.15.97:0/3954292836 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:03:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.641+0000 7fb2ad5f1740 10 client.0 _ll_register_callbacks cb 0x7fb2900b9ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:03:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.641+0000 7fb2a3400640 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.641+0000 7fb2a8a00640 1 -- 172.21.15.97:0/3954292836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 msgr2=0x55d61af17db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.641+0000 7fb2a8a00640 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 0x55d61af17db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.641+0000 7fb2a8a00640 1 -- 172.21.15.97:0/3954292836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 msgr2=0x55d61af182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:03:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.641+0000 7fb2a8a00640 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af182f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.641+0000 7fb2a8a00640 1 -- 172.21.15.97:0/3954292836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d61af198f0 con 0x55d61af0e900 2024-03-26T19:03:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.642+0000 7fb2a8a00640 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af0e900 0x55d61af19300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55d61af11de0 tx=0x7fb29c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8d449364ed77c28 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:03:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.642+0000 7fb28d600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:03:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.642+0000 7fb28d600640 1 -- 172.21.15.97:0/3954292836 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c01d040 con 0x55d61af0e900 2024-03-26T19:03:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.642+0000 7fb28d600640 1 -- 172.21.15.97:0/3954292836 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fb29c004600 con 0x55d61af0e900 2024-03-26T19:03:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.642+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/3954292836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d61ae75710 con 0x55d61af0e900 2024-03-26T19:03:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.643+0000 7fb28d600640 1 -- 172.21.15.97:0/3954292836 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb29c00ee60 con 0x55d61af0e900 2024-03-26T19:03:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.643+0000 7fb2ad5f1740 10 client.25646 fetch_fsmap learned FSMap version 27 2024-03-26T19:03:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.643+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/3954292836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d61af85240 con 0x55d61af0e900 2024-03-26T19:03:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.644+0000 7fb28d600640 1 -- 172.21.15.97:0/3954292836 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d61ae75710 con 0x55d61af0e900 2024-03-26T19:03:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.644+0000 7fb28d600640 1 -- 172.21.15.97:0/3954292836 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb29c024430 con 0x55d61af0e900 2024-03-26T19:03:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.644+0000 7fb2ad5f1740 10 client.25646 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:03:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.644+0000 7fb2ad5f1740 10 client.25646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:03:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.644+0000 7fb2ad5f1740 20 client.25646 populate_metadata read hostname 'smithi097' 2024-03-26T19:03:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.644+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d61af85800 0x55d61af87bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:03:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.644+0000 7fb2ad5f1740 4 client.25646 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:03:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.644+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/3954292836 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d61adf3630 con 0x55d61af85800 2024-03-26T19:03:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.645+0000 7fb2a3e00640 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d61af85800 0x55d61af87bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:03:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.645+0000 7fb28d600640 10 client.25646 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.645+0000 7fb2a3e00640 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d61af85800 0x55d61af87bf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8647c2b58962fb0 server_cookie=132b1537164f9c36 in_seq=0 out_seq=0 2024-03-26T19:03:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.690+0000 7fb28d600640 1 -- 172.21.15.97:0/3954292836 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d61adf3630 con 0x55d61af85800 2024-03-26T19:03:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.690+0000 7fb28d600640 10 client.25646 handle_command_reply: tid=0 2024-03-26T19:03:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.690+0000 7fb2ad5f1740 1 client.25646 shutdown 2024-03-26T19:03:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.697+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/3954292836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af0e900 msgr2=0x55d61af19300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.697+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af0e900 0x55d61af19300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55d61af11de0 tx=0x7fb29c009500 comp rx=0 tx=0).stop 2024-03-26T19:03:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.700+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/3954292836 shutdown_connections 2024-03-26T19:03:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.700+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d61af85800 0x55d61af87bf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.700+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d61af0e900 0x55d61af19300 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.700+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d61af09790 0x55d61af182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.700+0000 7fb2ad5f1740 1 --2- 172.21.15.97:0/3954292836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d61af08da0 0x55d61af17db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.700+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/3954292836 >> 172.21.15.97:0/3954292836 conn(0x55d61af04310 msgr2=0x55d61af0a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.700+0000 7fb28d600640 0 client.25646 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:03:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.701+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/3954292836 shutdown_connections 2024-03-26T19:03:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.701+0000 7fb2ad5f1740 1 -- 172.21.15.97:0/3954292836 wait complete. 2024-03-26T19:03:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.701+0000 7fb2ad5f1740 20 client.25646 trim_cache size 0 max 16384 2024-03-26T19:03:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:03:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:03:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:03:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:03:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.703+0000 7fb2aaa00640 1 -- 172.21.15.97:0/4078570500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb2880a3090 msgr2=0x7fb2880a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.703+0000 7fb2aaa00640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb2880a3090 0x7fb2880a5550 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55d61af09b90 tx=0x7fb29400a750 comp rx=0 tx=0).stop 2024-03-26T19:03:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.703+0000 7fb2aaa00640 1 -- 172.21.15.97:0/4078570500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a4104d00 msgr2=0x7fb2a4079bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:03:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.703+0000 7fb2aaa00640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a4104d00 0x7fb2a4079bb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb29002c240 tx=0x7fb290004e30 comp rx=0 tx=0).stop 2024-03-26T19:03:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.704+0000 7fb2aaa00640 1 -- 172.21.15.97:0/4078570500 shutdown_connections 2024-03-26T19:03:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.704+0000 7fb2aaa00640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb2880a3090 0x7fb2880a5550 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.704+0000 7fb2aaa00640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2a41056c0 0x7fb2a40761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.704+0000 7fb2aaa00640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a4104d00 0x7fb2a4079bb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.704+0000 7fb2aaa00640 1 --2- 172.21.15.97:0/4078570500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a4103b10 0x7fb2a4079670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:03:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.704+0000 7fb2aaa00640 1 -- 172.21.15.97:0/4078570500 >> 172.21.15.97:0/4078570500 conn(0x7fb2a40ff080 msgr2=0x7fb2a4107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:03:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.704+0000 7fb2aaa00640 1 -- 172.21.15.97:0/4078570500 shutdown_connections 2024-03-26T19:03:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:03:49.704+0000 7fb2aaa00640 1 -- 172.21.15.97:0/4078570500 wait complete. 2024-03-26T19:03:50.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 265.0 s, 2374.2 tps, lat 41.003 ms stddev 54.308 2024-03-26T19:03:50.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:50 smithi105 ceph-mon[36270]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 727 op/s 2024-03-26T19:03:50.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:50 smithi097 ceph-mon[31955]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 727 op/s 2024-03-26T19:03:50.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:50 smithi144 ceph-mon[36053]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 727 op/s 2024-03-26T19:03:52.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:52 smithi097 ceph-mon[31955]: pgmap v538: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 758 op/s 2024-03-26T19:03:52.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:52 smithi144 ceph-mon[36053]: pgmap v538: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 758 op/s 2024-03-26T19:03:52.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:52 smithi105 ceph-mon[36270]: pgmap v538: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 758 op/s 2024-03-26T19:03:54.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:54 smithi097 ceph-mon[31955]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 1.00k op/s 2024-03-26T19:03:54.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:54 smithi144 ceph-mon[36053]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 1.00k op/s 2024-03-26T19:03:54.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:54 smithi105 ceph-mon[36270]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 1.00k op/s 2024-03-26T19:03:55.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 270.0 s, 1674.6 tps, lat 60.398 ms stddev 110.723 2024-03-26T19:03:56.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:56 smithi097 ceph-mon[31955]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 46 MiB/s wr, 1.60k op/s 2024-03-26T19:03:56.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:56 smithi144 ceph-mon[36053]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 46 MiB/s wr, 1.60k op/s 2024-03-26T19:03:56.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:56 smithi105 ceph-mon[36270]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 46 MiB/s wr, 1.60k op/s 2024-03-26T19:03:58.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:03:58 smithi097 ceph-mon[31955]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 44 MiB/s wr, 1.60k op/s 2024-03-26T19:03:58.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:03:58 smithi144 ceph-mon[36053]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 44 MiB/s wr, 1.60k op/s 2024-03-26T19:03:58.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:03:58 smithi105 ceph-mon[36270]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 44 MiB/s wr, 1.60k op/s 2024-03-26T19:04:00.025 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 275.0 s, 2354.2 tps, lat 42.979 ms stddev 54.914 2024-03-26T19:04:00.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:00 smithi097 ceph-mon[31955]: pgmap v542: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 44 MiB/s wr, 1.66k op/s 2024-03-26T19:04:00.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:00 smithi144 ceph-mon[36053]: pgmap v542: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 44 MiB/s wr, 1.66k op/s 2024-03-26T19:04:00.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:00 smithi105 ceph-mon[36270]: pgmap v542: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 44 MiB/s wr, 1.66k op/s 2024-03-26T19:04:02.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:02 smithi097 ceph-mon[31955]: pgmap v543: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 41 MiB/s wr, 1.62k op/s 2024-03-26T19:04:02.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:02 smithi144 ceph-mon[36053]: pgmap v543: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 41 MiB/s wr, 1.62k op/s 2024-03-26T19:04:02.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:02 smithi105 ceph-mon[36270]: pgmap v543: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 41 MiB/s wr, 1.62k op/s 2024-03-26T19:04:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.914+0000 7f9897e00640 1 -- 172.21.15.97:0/3188532735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98901034b0 msgr2=0x7f9890109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.914+0000 7f9897e00640 1 --2- 172.21.15.97:0/3188532735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98901034b0 0x7f9890109d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9888009be0 tx=0x7f988802bef0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.914+0000 7f9897e00640 1 -- 172.21.15.97:0/3188532735 shutdown_connections 2024-03-26T19:04:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.914+0000 7f9897e00640 1 --2- 172.21.15.97:0/3188532735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98901034b0 0x7f9890109d40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.914+0000 7f9897e00640 1 --2- 172.21.15.97:0/3188532735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9890102af0 0x7f9890102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.914+0000 7f9897e00640 1 --2- 172.21.15.97:0/3188532735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9890101900 0x7f9890101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.914+0000 7f9897e00640 1 -- 172.21.15.97:0/3188532735 >> 172.21.15.97:0/3188532735 conn(0x7f98900fce50 msgr2=0x7f98900ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.915+0000 7f9897e00640 1 -- 172.21.15.97:0/3188532735 shutdown_connections 2024-03-26T19:04:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.915+0000 7f9897e00640 1 -- 172.21.15.97:0/3188532735 wait complete. 2024-03-26T19:04:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9897e00640 1 Processor -- start 2024-03-26T19:04:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9897e00640 1 -- start start 2024-03-26T19:04:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9897e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9890101900 0x7f989019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9897e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9890102af0 0x7f989019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9897e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98901034b0 0x7f98901a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9897e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f989010d5b0 con 0x7f98901034b0 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9897e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f989010d430 con 0x7f9890102af0 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9897e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f989010d730 con 0x7f9890101900 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9894a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9890102af0 0x7f989019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9894a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9890102af0 0x7f989019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:36654/0 (socket says 172.21.15.97:36654) 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.916+0000 7f9894a00640 1 -- 172.21.15.97:0/3251811915 learned_addr learned my addr 172.21.15.97:0/3251811915 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.917+0000 7f9895e00640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98901034b0 0x7f98901a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.917+0000 7f9895400640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9890101900 0x7f989019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.917+0000 7f9894a00640 1 -- 172.21.15.97:0/3251811915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9890101900 msgr2=0x7f989019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.917+0000 7f9894a00640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9890101900 0x7f989019e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.917+0000 7f9894a00640 1 -- 172.21.15.97:0/3251811915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98901034b0 msgr2=0x7f98901a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.917+0000 7f9894a00640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98901034b0 0x7f98901a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.917+0000 7f9894a00640 1 -- 172.21.15.97:0/3251811915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98901a5d20 con 0x7f9890102af0 2024-03-26T19:04:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.917+0000 7f9895400640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9890101900 0x7f989019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:04:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.918+0000 7f9894a00640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9890102af0 0x7f989019e640 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f987c00ea60 tx=0x7f987c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.918+0000 7f988e000640 1 -- 172.21.15.97:0/3251811915 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c00cde0 con 0x7f9890102af0 2024-03-26T19:04:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.918+0000 7f988e000640 1 -- 172.21.15.97:0/3251811915 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f987c0035f0 con 0x7f9890102af0 2024-03-26T19:04:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.918+0000 7f9897e00640 1 -- 172.21.15.97:0/3251811915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98901a5fb0 con 0x7f9890102af0 2024-03-26T19:04:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.918+0000 7f9897e00640 1 -- 172.21.15.97:0/3251811915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901a64d0 con 0x7f9890102af0 2024-03-26T19:04:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.919+0000 7f988e000640 1 -- 172.21.15.97:0/3251811915 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c010670 con 0x7f9890102af0 2024-03-26T19:04:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.926+0000 7f988e000640 1 -- 172.21.15.97:0/3251811915 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f987c0037a0 con 0x7f9890102af0 2024-03-26T19:04:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.928+0000 7f988e000640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98780a2fc0 0x7f98780a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.929+0000 7f988e000640 1 -- 172.21.15.97:0/3251811915 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f987c0cf7d0 con 0x7f9890102af0 2024-03-26T19:04:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.930+0000 7f9895400640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98780a2fc0 0x7f98780a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.931+0000 7f989ac2f740 1 Processor -- start 2024-03-26T19:04:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.932+0000 7f9895400640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98780a2fc0 0x7f98780a5480 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9890102950 tx=0x7f9884009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.932+0000 7f989ac2f740 1 -- start start 2024-03-26T19:04:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.935+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 0x55b10b8d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.935+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8d9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.935+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d9900 0x55b10b8dbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.935+0000 7f989ac2f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b10b838e80 con 0x55b10b8d3da0 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.935+0000 7f989ac2f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b10b7e1660 con 0x55b10b8d4790 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.935+0000 7f989ac2f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b10b834e30 con 0x55b10b8d9900 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.935+0000 7f9894a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8d9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.935+0000 7f9894a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8d9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:36664/0 (socket says 172.21.15.97:36664) 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.935+0000 7f9894a00640 1 -- 172.21.15.97:0/3233943381 learned_addr learned my addr 172.21.15.97:0/3233943381 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.936+0000 7f9895400640 1 --2- 172.21.15.97:0/3233943381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 0x55b10b8d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.936+0000 7f9894a00640 1 -- 172.21.15.97:0/3233943381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d9900 msgr2=0x55b10b8dbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.936+0000 7f9894a00640 1 --2- 172.21.15.97:0/3233943381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d9900 0x55b10b8dbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.936+0000 7f9894a00640 1 -- 172.21.15.97:0/3233943381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 msgr2=0x55b10b8d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.936+0000 7f9894a00640 1 --2- 172.21.15.97:0/3233943381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 0x55b10b8d41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.936+0000 7f9894a00640 1 -- 172.21.15.97:0/3233943381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10b54d210 con 0x55b10b8d4790 2024-03-26T19:04:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.937+0000 7f9894a00640 1 --2- 172.21.15.97:0/3233943381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8d9290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f987c00d010 tx=0x7f987c09be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=becd0e6e15ab8859 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.937+0000 7f9882600640 1 -- 172.21.15.97:0/3233943381 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0b4070 con 0x55b10b8d4790 2024-03-26T19:04:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.938+0000 7f9882600640 1 -- 172.21.15.97:0/3233943381 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f987c09bf30 con 0x55b10b8d4790 2024-03-26T19:04:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.938+0000 7f9882600640 1 -- 172.21.15.97:0/3233943381 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c003b30 con 0x55b10b8d4790 2024-03-26T19:04:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.940+0000 7f989ac2f740 1 -- 172.21.15.97:0/3233943381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 msgr2=0x55b10b8d9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.940+0000 7f989ac2f740 1 --2- 172.21.15.97:0/3233943381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8d9290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f987c00d010 tx=0x7f987c09be30 comp rx=0 tx=0).stop 2024-03-26T19:04:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.940+0000 7f989ac2f740 1 -- 172.21.15.97:0/3233943381 shutdown_connections 2024-03-26T19:04:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.940+0000 7f989ac2f740 1 --2- 172.21.15.97:0/3233943381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d9900 0x55b10b8dbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.940+0000 7f989ac2f740 1 --2- 172.21.15.97:0/3233943381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8d9290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.940+0000 7f989ac2f740 1 --2- 172.21.15.97:0/3233943381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 0x55b10b8d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.940+0000 7f989ac2f740 1 -- 172.21.15.97:0/3233943381 >> 172.21.15.97:0/3233943381 conn(0x55b10b8cf310 msgr2=0x55b10b8cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.941+0000 7f989ac2f740 1 -- 172.21.15.97:0/3233943381 shutdown_connections 2024-03-26T19:04:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.941+0000 7f989ac2f740 1 -- 172.21.15.97:0/3233943381 wait complete. 2024-03-26T19:04:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.941+0000 7f989ac2f740 1 -- start start 2024-03-26T19:04:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.942+0000 7f989ac2f740 1 Processor -- start 2024-03-26T19:04:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.943+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 0x55b10b8eee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.943+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.943+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d9900 0x55b10b8f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.943+0000 7f989ac2f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b10b8ed8c0 con 0x55b10b8d3da0 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.943+0000 7f989ac2f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b10b7f4ac0 con 0x55b10b8d4790 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.943+0000 7f989ac2f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b10b8eda40 con 0x55b10b8d9900 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.943+0000 7f9894a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.943+0000 7f9894a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8ef3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:36676/0 (socket says 172.21.15.97:36676) 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.943+0000 7f9894a00640 1 -- 172.21.15.97:0/2349742529 learned_addr learned my addr 172.21.15.97:0/2349742529 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.943+0000 7f989ac2f740 10 client.0 _ll_register_callbacks cb 0x55b10b7f4490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.944+0000 7f9894a00640 1 -- 172.21.15.97:0/2349742529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d9900 msgr2=0x55b10b8f63f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.944+0000 7f9894a00640 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d9900 0x55b10b8f63f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.944+0000 7f9894a00640 1 -- 172.21.15.97:0/2349742529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 msgr2=0x55b10b8eee70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.944+0000 7f9895400640 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 0x55b10b8eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.944+0000 7f9894a00640 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 0x55b10b8eee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.944+0000 7f9894a00640 1 -- 172.21.15.97:0/2349742529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b10b619620 con 0x55b10b8d4790 2024-03-26T19:04:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.944+0000 7f989ac2f740 1 -- 172.21.15.97:0/2349742529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f987c09bf30 con 0x55b10b8d4790 2024-03-26T19:04:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.944+0000 7f9895400640 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 0x55b10b8eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:04:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.944+0000 7f9894a00640 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8ef3b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f987c098900 tx=0x7f987c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74f6dd5622a0bd14 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.944+0000 7f9876a00640 10 client.25658 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.945+0000 7f9876a00640 1 -- 172.21.15.97:0/2349742529 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0b4030 con 0x55b10b8d4790 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.945+0000 7f9876a00640 1 -- 172.21.15.97:0/2349742529 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f987c003b30 con 0x55b10b8d4790 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.945+0000 7f9876a00640 1 -- 172.21.15.97:0/2349742529 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f987c0a4940 con 0x55b10b8d4790 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.945+0000 7f989ac2f740 10 client.25658 fetch_fsmap learned FSMap version 27 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.945+0000 7f989ac2f740 1 -- 172.21.15.97:0/2349742529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55b10b8f9680 con 0x55b10b8d4790 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.945+0000 7f9876a00640 1 -- 172.21.15.97:0/2349742529 <== mon.2 v2:172.21.15.105:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f987c0985a0 con 0x55b10b8d4790 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.945+0000 7f989ac2f740 10 client.25658 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.945+0000 7f989ac2f740 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.946+0000 7f989ac2f740 20 client.25658 populate_metadata read hostname 'smithi097' 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.946+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b10b8fa9d0 0x55b10b8fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.946+0000 7f989ac2f740 4 client.25658 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.946+0000 7f989ac2f740 1 -- 172.21.15.97:0/2349742529 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b10b8463c0 con 0x55b10b8fa9d0 2024-03-26T19:04:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.946+0000 7f9895400640 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b10b8fa9d0 0x55b10b8fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.946+0000 7f9895400640 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b10b8fa9d0 0x55b10b8fcdc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19a39a8a2199dc1 server_cookie=3aee976e20761131 in_seq=0 out_seq=0 2024-03-26T19:04:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.946+0000 7f9876a00640 10 client.25658 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.947+0000 7f9876a00640 1 -- 172.21.15.97:0/2349742529 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b10b8463c0 con 0x55b10b8fa9d0 2024-03-26T19:04:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.947+0000 7f9876a00640 10 client.25658 handle_command_reply: tid=0 2024-03-26T19:04:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.948+0000 7f989ac2f740 1 client.25658 shutdown 2024-03-26T19:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.948+0000 7f989ac2f740 1 -- 172.21.15.97:0/2349742529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 msgr2=0x55b10b8ef3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.948+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8ef3b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f987c098900 tx=0x7f987c005740 comp rx=0 tx=0).stop 2024-03-26T19:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.949+0000 7f989ac2f740 1 -- 172.21.15.97:0/2349742529 shutdown_connections 2024-03-26T19:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.949+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b10b8fa9d0 0x55b10b8fcdc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.949+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d9900 0x55b10b8f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.949+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8ef3b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.949+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2349742529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d3da0 0x55b10b8eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.949+0000 7f989ac2f740 1 -- 172.21.15.97:0/2349742529 >> 172.21.15.97:0/2349742529 conn(0x55b10b8cf310 msgr2=0x55b10b8ce540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.949+0000 7f9876a00640 0 client.25658 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.949+0000 7f989ac2f740 1 -- 172.21.15.97:0/2349742529 shutdown_connections 2024-03-26T19:04:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.949+0000 7f989ac2f740 1 -- 172.21.15.97:0/2349742529 wait complete. 2024-03-26T19:04:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.949+0000 7f989ac2f740 20 client.25658 trim_cache size 0 max 16384 2024-03-26T19:04:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.981+0000 7f989ac2f740 1 Processor -- start 2024-03-26T19:04:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.981+0000 7f989ac2f740 1 -- start start 2024-03-26T19:04:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.981+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.981+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d4790 0x55b10b8f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d9900 0x55b10b8d4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f989ac2f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98901a6b10 con 0x55b10b8d4790 2024-03-26T19:04:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f989ac2f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f98901a8650 con 0x55b10b8d9900 2024-03-26T19:04:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f989ac2f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98901a71e0 con 0x55b10b8d3da0 2024-03-26T19:04:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d9900 0x55b10b8d4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d9900 0x55b10b8d4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:36686/0 (socket says 172.21.15.97:36686) 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8fc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:57894/0 (socket says 172.21.15.97:57894) 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895e00640 1 -- 172.21.15.97:0/1455985149 learned_addr learned my addr 172.21.15.97:0/1455985149 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895e00640 1 -- 172.21.15.97:0/1455985149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 msgr2=0x55b10b8fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895e00640 1 --2- 172.21.15.97:0/1455985149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895e00640 1 -- 172.21.15.97:0/1455985149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d4790 msgr2=0x55b10b8f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895e00640 1 --2- 172.21.15.97:0/1455985149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d4790 0x55b10b8f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895e00640 1 -- 172.21.15.97:0/1455985149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10b8cdab0 con 0x55b10b8d9900 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.982+0000 7f9895400640 1 --2- 172.21.15.97:0/1455985149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.983+0000 7f9895e00640 1 --2- 172.21.15.97:0/1455985149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d9900 0x55b10b8d4e80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98880334e0 tx=0x7f9888009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bee00e5e542153cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.983+0000 7f9881c00640 1 -- 172.21.15.97:0/1455985149 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888043070 con 0x55b10b8d9900 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.983+0000 7f9881c00640 1 -- 172.21.15.97:0/1455985149 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9888033580 con 0x55b10b8d9900 2024-03-26T19:04:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.983+0000 7f9881c00640 1 -- 172.21.15.97:0/1455985149 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988802caf0 con 0x55b10b8d9900 2024-03-26T19:04:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.984+0000 7f989ac2f740 1 -- 172.21.15.97:0/1455985149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d9900 msgr2=0x55b10b8d4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.984+0000 7f989ac2f740 1 --2- 172.21.15.97:0/1455985149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d9900 0x55b10b8d4e80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98880334e0 tx=0x7f9888009330 comp rx=0 tx=0).stop 2024-03-26T19:04:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.984+0000 7f989ac2f740 1 -- 172.21.15.97:0/1455985149 shutdown_connections 2024-03-26T19:04:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.984+0000 7f989ac2f740 1 --2- 172.21.15.97:0/1455985149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d9900 0x55b10b8d4e80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.984+0000 7f989ac2f740 1 --2- 172.21.15.97:0/1455985149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d4790 0x55b10b8f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.984+0000 7f989ac2f740 1 --2- 172.21.15.97:0/1455985149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.984+0000 7f989ac2f740 1 -- 172.21.15.97:0/1455985149 >> 172.21.15.97:0/1455985149 conn(0x55b10b8cf310 msgr2=0x55b10b8d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.984+0000 7f989ac2f740 1 -- 172.21.15.97:0/1455985149 shutdown_connections 2024-03-26T19:04:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.985+0000 7f989ac2f740 1 -- 172.21.15.97:0/1455985149 wait complete. 2024-03-26T19:04:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.985+0000 7f989ac2f740 1 -- start start 2024-03-26T19:04:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.985+0000 7f989ac2f740 1 Processor -- start 2024-03-26T19:04:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.987+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.987+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.987+0000 7f989ac2f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d9900 0x55b10b8f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.987+0000 7f9895400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.987+0000 7f9895400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8f3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:57906/0 (socket says 172.21.15.97:57906) 2024-03-26T19:04:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f9895400640 1 -- 172.21.15.97:0/2908793436 learned_addr learned my addr 172.21.15.97:0/2908793436 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.987+0000 7f9894a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98901a7f80 con 0x55b10b8d9900 2024-03-26T19:04:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f98901a78b0 con 0x55b10b8d4790 2024-03-26T19:04:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9890071e20 con 0x55b10b8d3da0 2024-03-26T19:04:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f989ac2f740 10 client.0 _ll_register_callbacks cb 0x7f987c003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:04:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f9895e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d9900 0x55b10b8f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f9895400640 1 -- 172.21.15.97:0/2908793436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 msgr2=0x55b10b8f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f9895400640 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f9895400640 1 -- 172.21.15.97:0/2908793436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d9900 msgr2=0x55b10b8f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f9895400640 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d9900 0x55b10b8f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f9895400640 1 -- 172.21.15.97:0/2908793436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b10b8f5ac0 con 0x55b10b8d3da0 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.988+0000 7f9894a00640 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.989+0000 7f9895400640 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8f3f80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55b10b8fd3c0 tx=0x7f9884014070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95e475b33c7f2d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.989+0000 7f9876a00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.989+0000 7f9876a00640 1 -- 172.21.15.97:0/2908793436 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988402b070 con 0x55b10b8d3da0 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.989+0000 7f9876a00640 1 -- 172.21.15.97:0/2908793436 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f98840129a0 con 0x55b10b8d3da0 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.989+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9888033580 con 0x55b10b8d3da0 2024-03-26T19:04:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.989+0000 7f9876a00640 1 -- 172.21.15.97:0/2908793436 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9884012ba0 con 0x55b10b8d3da0 2024-03-26T19:04:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.990+0000 7f989ac2f740 10 client.25615 fetch_fsmap learned FSMap version 27 2024-03-26T19:04:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.990+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b10b961040 con 0x55b10b8d3da0 2024-03-26T19:04:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.991+0000 7f9876a00640 1 -- 172.21.15.97:0/2908793436 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9888033580 con 0x55b10b8d3da0 2024-03-26T19:04:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.991+0000 7f9876a00640 1 -- 172.21.15.97:0/2908793436 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f98840305f0 con 0x55b10b8d3da0 2024-03-26T19:04:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.991+0000 7f989ac2f740 10 client.25615 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:04:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.991+0000 7f989ac2f740 10 client.25615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:04:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.991+0000 7f989ac2f740 20 client.25615 populate_metadata read hostname 'smithi097' 2024-03-26T19:04:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.991+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b10b9619e0 0x55b10b963e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.991+0000 7f989ac2f740 4 client.25615 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:04:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.991+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b10b840710 con 0x55b10b9619e0 2024-03-26T19:04:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.992+0000 7f9894a00640 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b10b9619e0 0x55b10b963e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.992+0000 7f9876a00640 10 client.25615 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:02.993+0000 7f9894a00640 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b10b9619e0 0x55b10b963e00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54961d01443fd33 server_cookie=bf3949c29750ecb5 in_seq=0 out_seq=0 2024-03-26T19:04:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.041+0000 7f9876a00640 1 -- 172.21.15.97:0/2908793436 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b10b840710 con 0x55b10b9619e0 2024-03-26T19:04:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.041+0000 7f9876a00640 10 client.25615 handle_command_reply: tid=0 2024-03-26T19:04:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.041+0000 7f989ac2f740 1 client.25615 shutdown 2024-03-26T19:04:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.042+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 msgr2=0x55b10b8f3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.042+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8f3f80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55b10b8fd3c0 tx=0x7f9884014070 comp rx=0 tx=0).stop 2024-03-26T19:04:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.042+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 shutdown_connections 2024-03-26T19:04:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.042+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b10b9619e0 0x55b10b963e00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.042+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b10b8d9900 0x55b10b8f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.042+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b10b8d4790 0x55b10b8f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.042+0000 7f989ac2f740 1 --2- 172.21.15.97:0/2908793436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b10b8d3da0 0x55b10b8f3f80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.042+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 >> 172.21.15.97:0/2908793436 conn(0x55b10b8cf310 msgr2=0x55b10b8cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.042+0000 7f9876a00640 0 client.25615 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.043+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 shutdown_connections 2024-03-26T19:04:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.043+0000 7f989ac2f740 1 -- 172.21.15.97:0/2908793436 wait complete. 2024-03-26T19:04:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.043+0000 7f989ac2f740 20 client.25615 trim_cache size 0 max 16384 2024-03-26T19:04:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:04:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:04:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:04:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:04:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 -- 172.21.15.97:0/3251811915 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98780a2fc0 msgr2=0x7f98780a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98780a2fc0 0x7f98780a5480 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9890102950 tx=0x7f9884009260 comp rx=0 tx=0).stop 2024-03-26T19:04:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 -- 172.21.15.97:0/3251811915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9890102af0 msgr2=0x7f989019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9890102af0 0x7f989019e640 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f987c00ea60 tx=0x7f987c00ef20 comp rx=0 tx=0).stop 2024-03-26T19:04:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 -- 172.21.15.97:0/3251811915 shutdown_connections 2024-03-26T19:04:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98780a2fc0 0x7f98780a5480 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98901034b0 0x7f98901a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9890102af0 0x7f989019e640 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 --2- 172.21.15.97:0/3251811915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9890101900 0x7f989019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 -- 172.21.15.97:0/3251811915 >> 172.21.15.97:0/3251811915 conn(0x7f98900fce50 msgr2=0x7f9890103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 -- 172.21.15.97:0/3251811915 shutdown_connections 2024-03-26T19:04:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:03.044+0000 7f9897e00640 1 -- 172.21.15.97:0/3251811915 wait complete. 2024-03-26T19:04:04.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:04 smithi097 ceph-mon[31955]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 37 MiB/s wr, 1.51k op/s 2024-03-26T19:04:04.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:04 smithi144 ceph-mon[36053]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 37 MiB/s wr, 1.51k op/s 2024-03-26T19:04:04.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:04 smithi105 ceph-mon[36270]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 37 MiB/s wr, 1.51k op/s 2024-03-26T19:04:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 280.0 s, 2139.4 tps, lat 46.542 ms stddev 49.738 2024-03-26T19:04:06.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:06 smithi097 ceph-mon[31955]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 29 MiB/s wr, 1.36k op/s 2024-03-26T19:04:06.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:06 smithi144 ceph-mon[36053]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 29 MiB/s wr, 1.36k op/s 2024-03-26T19:04:06.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:06 smithi105 ceph-mon[36270]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 29 MiB/s wr, 1.36k op/s 2024-03-26T19:04:08.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:08 smithi097 ceph-mon[31955]: pgmap v546: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 749 op/s 2024-03-26T19:04:08.628 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:08 smithi144 ceph-mon[36053]: pgmap v546: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 749 op/s 2024-03-26T19:04:08.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:08 smithi105 ceph-mon[36270]: pgmap v546: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 749 op/s 2024-03-26T19:04:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 285.0 s, 2085.3 tps, lat 48.034 ms stddev 54.659 2024-03-26T19:04:10.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:10 smithi097 ceph-mon[31955]: pgmap v547: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 730 op/s 2024-03-26T19:04:10.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:10 smithi144 ceph-mon[36053]: pgmap v547: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 730 op/s 2024-03-26T19:04:10.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:10 smithi105 ceph-mon[36270]: pgmap v547: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 730 op/s 2024-03-26T19:04:11.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:11 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:04:11.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:11 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:04:11.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:11 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:04:12.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:12 smithi097 ceph-mon[31955]: pgmap v548: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.1 MiB/s wr, 707 op/s 2024-03-26T19:04:12.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:12 smithi144 ceph-mon[36053]: pgmap v548: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.1 MiB/s wr, 707 op/s 2024-03-26T19:04:12.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:12 smithi105 ceph-mon[36270]: pgmap v548: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.1 MiB/s wr, 707 op/s 2024-03-26T19:04:13.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:13 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:04:13.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:13 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:04:13.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:13 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:04:13.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:13 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:04:13.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:13 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:04:13.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:13 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:04:13.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:13 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:04:13.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:13 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:04:13.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:13 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:04:14.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:14 smithi097 ceph-mon[31955]: pgmap v549: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.6 MiB/s wr, 670 op/s 2024-03-26T19:04:14.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:14 smithi144 ceph-mon[36053]: pgmap v549: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.6 MiB/s wr, 670 op/s 2024-03-26T19:04:14.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:14 smithi105 ceph-mon[36270]: pgmap v549: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.6 MiB/s wr, 670 op/s 2024-03-26T19:04:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 290.0 s, 1803.8 tps, lat 54.733 ms stddev 62.987 2024-03-26T19:04:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.237+0000 7f06e7800640 1 -- 172.21.15.97:0/3861226492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06e010bc90 msgr2=0x7f06e0112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.237+0000 7f06e7800640 1 --2- 172.21.15.97:0/3861226492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06e010bc90 0x7f06e0112520 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009ca0 tx=0x7f06d802c0f0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.237+0000 7f06e7800640 1 -- 172.21.15.97:0/3861226492 shutdown_connections 2024-03-26T19:04:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.237+0000 7f06e7800640 1 --2- 172.21.15.97:0/3861226492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06e010bc90 0x7f06e0112520 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.237+0000 7f06e7800640 1 --2- 172.21.15.97:0/3861226492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06e010b2d0 0x7f06e010b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.237+0000 7f06e7800640 1 --2- 172.21.15.97:0/3861226492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06e010a0e0 0x7f06e010a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.238+0000 7f06e7800640 1 -- 172.21.15.97:0/3861226492 >> 172.21.15.97:0/3861226492 conn(0x7f06e0070880 msgr2=0x7f06e0072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.238+0000 7f06e7800640 1 -- 172.21.15.97:0/3861226492 shutdown_connections 2024-03-26T19:04:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.238+0000 7f06e7800640 1 -- 172.21.15.97:0/3861226492 wait complete. 2024-03-26T19:04:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.240+0000 7f06e7800640 1 Processor -- start 2024-03-26T19:04:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.240+0000 7f06e7800640 1 -- start start 2024-03-26T19:04:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.241+0000 7f06e7800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06e010a0e0 0x7f06e019e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.241+0000 7f06e7800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06e010b2d0 0x7f06e019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.242+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06e010a0e0 0x7f06e019e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.242+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06e010a0e0 0x7f06e019e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:33040/0 (socket says 172.21.15.97:33040) 2024-03-26T19:04:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.242+0000 7f06e4e00640 1 -- 172.21.15.97:0/3244451175 learned_addr learned my addr 172.21.15.97:0/3244451175 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.242+0000 7f06dfe00640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06e010b2d0 0x7f06e019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.242+0000 7f06e7800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06e010bc90 0x7f06e01b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.242+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0115df0 con 0x7f06e010b2d0 2024-03-26T19:04:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.242+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0115c70 con 0x7f06e010bc90 2024-03-26T19:04:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.242+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0115f70 con 0x7f06e010a0e0 2024-03-26T19:04:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.243+0000 7f06e5800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06e010bc90 0x7f06e01b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.244+0000 7f06e5800640 1 -- 172.21.15.97:0/3244451175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06e010a0e0 msgr2=0x7f06e019e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.244+0000 7f06e5800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06e010a0e0 0x7f06e019e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.244+0000 7f06e5800640 1 -- 172.21.15.97:0/3244451175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06e010b2d0 msgr2=0x7f06e019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.244+0000 7f06e5800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06e010b2d0 0x7f06e019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.244+0000 7f06e5800640 1 -- 172.21.15.97:0/3244451175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e01a38e0 con 0x7f06e010bc90 2024-03-26T19:04:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.245+0000 7f06e5800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06e010bc90 0x7f06e01b6d00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f06d8037040 tx=0x7f06d8035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.248+0000 7f06dd600640 1 -- 172.21.15.97:0/3244451175 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8039040 con 0x7f06e010bc90 2024-03-26T19:04:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.248+0000 7f06dd600640 1 -- 172.21.15.97:0/3244451175 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d8035e00 con 0x7f06e010bc90 2024-03-26T19:04:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.248+0000 7f06dd600640 1 -- 172.21.15.97:0/3244451175 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80043c0 con 0x7f06e010bc90 2024-03-26T19:04:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.248+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e01b7240 con 0x7f06e010bc90 2024-03-26T19:04:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.248+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e01b75f0 con 0x7f06e010bc90 2024-03-26T19:04:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.251+0000 7f06dd600640 1 -- 172.21.15.97:0/3244451175 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f06d8046030 con 0x7f06e010bc90 2024-03-26T19:04:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.251+0000 7f06ea5b3740 1 Processor -- start 2024-03-26T19:04:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.251+0000 7f06ea5b3740 1 -- start start 2024-03-26T19:04:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.251+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 0x556eae1bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.252+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1c4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.252+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1c4900 0x556eae1c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.252+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556eae123e80 con 0x556eae1c4900 2024-03-26T19:04:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.252+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556eae0cc660 con 0x556eae1bf790 2024-03-26T19:04:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.252+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556eae11fe30 con 0x556eae1beda0 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.252+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.252+0000 7f06dfe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 0x556eae1bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.252+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1c4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34202/0 (socket says 172.21.15.97:34202) 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.252+0000 7f06e4e00640 1 -- 172.21.15.97:0/964949656 learned_addr learned my addr 172.21.15.97:0/964949656 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.253+0000 7f06dfe00640 1 --2- 172.21.15.97:0/964949656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1c4900 0x556eae1c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.253+0000 7f06e4e00640 1 -- 172.21.15.97:0/964949656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 msgr2=0x556eae1bf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.253+0000 7f06e4e00640 1 --2- 172.21.15.97:0/964949656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 0x556eae1bf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.253+0000 7f06e4e00640 1 -- 172.21.15.97:0/964949656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1c4900 msgr2=0x556eae1c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.253+0000 7f06e4e00640 1 --2- 172.21.15.97:0/964949656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1c4900 0x556eae1c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.253+0000 7f06e4e00640 1 -- 172.21.15.97:0/964949656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eade38210 con 0x556eae1bf790 2024-03-26T19:04:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.253+0000 7f06dfe00640 1 --2- 172.21.15.97:0/964949656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1c4900 0x556eae1c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:04:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.253+0000 7f06dd600640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f06c80a3180 0x7f06c80a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.254+0000 7f06e4e00640 1 --2- 172.21.15.97:0/964949656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1c4290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009af0 tx=0x7f06d400e1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79ffab3d18c121c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.254+0000 7f06d2000640 1 -- 172.21.15.97:0/964949656 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4017070 con 0x556eae1bf790 2024-03-26T19:04:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.254+0000 7f06d2000640 1 -- 172.21.15.97:0/964949656 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d400ec10 con 0x556eae1bf790 2024-03-26T19:04:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.254+0000 7f06d2000640 1 -- 172.21.15.97:0/964949656 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d40126c0 con 0x556eae1bf790 2024-03-26T19:04:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.255+0000 7f06dd600640 1 -- 172.21.15.97:0/3244451175 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f06d80f0dc0 con 0x7f06e010bc90 2024-03-26T19:04:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.256+0000 7f06ea5b3740 1 -- 172.21.15.97:0/964949656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 msgr2=0x556eae1c4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.256+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/964949656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1c4290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009af0 tx=0x7f06d400e1d0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.256+0000 7f06ea5b3740 1 -- 172.21.15.97:0/964949656 shutdown_connections 2024-03-26T19:04:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.256+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/964949656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1c4900 0x556eae1c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.256+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/964949656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1c4290 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.256+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/964949656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 0x556eae1bf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.256+0000 7f06ea5b3740 1 -- 172.21.15.97:0/964949656 >> 172.21.15.97:0/964949656 conn(0x556eae1ba310 msgr2=0x556eae1ba710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.256+0000 7f06ea5b3740 1 -- 172.21.15.97:0/964949656 shutdown_connections 2024-03-26T19:04:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.256+0000 7f06ea5b3740 1 -- 172.21.15.97:0/964949656 wait complete. 2024-03-26T19:04:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.256+0000 7f06dfe00640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f06c80a3180 0x7f06c80a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.257+0000 7f06ea5b3740 1 -- start start 2024-03-26T19:04:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.257+0000 7f06ea5b3740 1 Processor -- start 2024-03-26T19:04:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.257+0000 7f06dfe00640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f06c80a3180 0x7f06c80a5640 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f06cc005d70 tx=0x7f06cc005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1beda0 0x556eae1c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 0x556eae1c9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 0x556eae1d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556eae1c78b0 con 0x556eae1bf790 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556eae0dfac0 con 0x556eae1beda0 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556eae1c7a30 con 0x556eae1c4900 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06ea5b3740 10 client.0 _ll_register_callbacks cb 0x556eae0df490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1beda0 0x556eae1c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06e5800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 0x556eae1d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1beda0 0x556eae1c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34210/0 (socket says 172.21.15.97:34210) 2024-03-26T19:04:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06e4e00640 1 -- 172.21.15.97:0/408525578 learned_addr learned my addr 172.21.15.97:0/408525578 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06dfe00640 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 0x556eae1c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06e5800640 1 -- 172.21.15.97:0/408525578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1beda0 msgr2=0x556eae1c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.258+0000 7f06e5800640 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1beda0 0x556eae1c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06e5800640 1 -- 172.21.15.97:0/408525578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 msgr2=0x556eae1c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06e5800640 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 0x556eae1c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06e5800640 1 -- 172.21.15.97:0/408525578 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eadf04620 con 0x556eae1c4900 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06dfe00640 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 0x556eae1c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06e5800640 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 0x556eae1d0340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f06d80324c0 tx=0x7f06d80bd150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fab6443bbdbe4ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06c6a00640 1 -- 172.21.15.97:0/408525578 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80d6060 con 0x556eae1c4900 2024-03-26T19:04:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06c6a00640 1 -- 172.21.15.97:0/408525578 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f06d8003ae0 con 0x556eae1c4900 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06ea5b3740 1 -- 172.21.15.97:0/408525578 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d400ec10 con 0x556eae1c4900 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06c6a00640 1 -- 172.21.15.97:0/408525578 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06d80bd550 con 0x556eae1c4900 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06ea5b3740 10 client.25627 fetch_fsmap learned FSMap version 27 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.259+0000 7f06ea5b3740 1 -- 172.21.15.97:0/408525578 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556eae1c78b0 con 0x556eae1c4900 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.260+0000 7f06c6a00640 1 -- 172.21.15.97:0/408525578 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d400ec10 con 0x556eae1c4900 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.260+0000 7f06c6a00640 1 -- 172.21.15.97:0/408525578 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f06d803c030 con 0x556eae1c4900 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.260+0000 7f06ea5b3740 10 client.25627 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.260+0000 7f06ea5b3740 10 client.25627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.260+0000 7f06ea5b3740 20 client.25627 populate_metadata read hostname 'smithi097' 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.260+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556eae1d4d10 0x556eae1d7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.260+0000 7f06ea5b3740 4 client.25627 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.260+0000 7f06ea5b3740 1 -- 172.21.15.97:0/408525578 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eae1313c0 con 0x556eae1d4d10 2024-03-26T19:04:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.261+0000 7f06e4e00640 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556eae1d4d10 0x556eae1d7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.261+0000 7f06e4e00640 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556eae1d4d10 0x556eae1d7150 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8ad0c366f62e15 server_cookie=afcc6d4f6d0a08ff in_seq=0 out_seq=0 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.261+0000 7f06c6a00640 10 client.25627 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.262+0000 7f06c6a00640 1 -- 172.21.15.97:0/408525578 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556eae1313c0 con 0x556eae1d4d10 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.262+0000 7f06c6a00640 10 client.25627 handle_command_reply: tid=0 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.263+0000 7f06ea5b3740 1 client.25627 shutdown 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.263+0000 7f06ea5b3740 1 -- 172.21.15.97:0/408525578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 msgr2=0x556eae1d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.263+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 0x556eae1d0340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f06d80324c0 tx=0x7f06d80bd150 comp rx=0 tx=0).stop 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.263+0000 7f06ea5b3740 1 -- 172.21.15.97:0/408525578 shutdown_connections 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.263+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556eae1d4d10 0x556eae1d7150 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.263+0000 7f06c6a00640 0 client.25627 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.263+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 0x556eae1d0340 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.263+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 0x556eae1c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.263+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/408525578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1beda0 0x556eae1c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.263+0000 7f06ea5b3740 1 -- 172.21.15.97:0/408525578 >> 172.21.15.97:0/408525578 conn(0x556eae1ba310 msgr2=0x556eae1b9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.264+0000 7f06ea5b3740 1 -- 172.21.15.97:0/408525578 shutdown_connections 2024-03-26T19:04:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.264+0000 7f06ea5b3740 1 -- 172.21.15.97:0/408525578 wait complete. 2024-03-26T19:04:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.264+0000 7f06ea5b3740 20 client.25627 trim_cache size 0 max 16384 2024-03-26T19:04:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.293+0000 7f06ea5b3740 1 Processor -- start 2024-03-26T19:04:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.293+0000 7f06ea5b3740 1 -- start start 2024-03-26T19:04:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.293+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1beda0 0x556eae1c0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.293+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.293+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 0x556eae1c0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.293+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01c01d0 con 0x556eae1beda0 2024-03-26T19:04:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.293+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01c1d10 con 0x556eae1bf790 2024-03-26T19:04:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.293+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01c08a0 con 0x556eae1c4900 2024-03-26T19:04:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.294+0000 7f06dfe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.294+0000 7f06dfe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34224/0 (socket says 172.21.15.97:34224) 2024-03-26T19:04:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.294+0000 7f06dfe00640 1 -- 172.21.15.97:0/2017418012 learned_addr learned my addr 172.21.15.97:0/2017418012 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.294+0000 7f06e5800640 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 0x556eae1c0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.294+0000 7f06dfe00640 1 -- 172.21.15.97:0/2017418012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 msgr2=0x556eae1c0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.294+0000 7f06e4e00640 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1beda0 0x556eae1c0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.294+0000 7f06dfe00640 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 0x556eae1c0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.294+0000 7f06dfe00640 1 -- 172.21.15.97:0/2017418012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1beda0 msgr2=0x556eae1c0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.294+0000 7f06dfe00640 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1beda0 0x556eae1c0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.294+0000 7f06dfe00640 1 -- 172.21.15.97:0/2017418012 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eae1c71e0 con 0x556eae1bf790 2024-03-26T19:04:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.295+0000 7f06e4e00640 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1beda0 0x556eae1c0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:04:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.295+0000 7f06e5800640 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 0x556eae1c0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:04:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.296+0000 7f06dfe00640 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1d0300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f06cc010600 tx=0x7f06cc013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=954bf04e83ee06e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.298+0000 7f06d1600640 1 -- 172.21.15.97:0/2017418012 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc004220 con 0x556eae1bf790 2024-03-26T19:04:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.298+0000 7f06d1600640 1 -- 172.21.15.97:0/2017418012 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06cc0043c0 con 0x556eae1bf790 2024-03-26T19:04:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.298+0000 7f06d1600640 1 -- 172.21.15.97:0/2017418012 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc003d60 con 0x556eae1bf790 2024-03-26T19:04:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.299+0000 7f06ea5b3740 1 -- 172.21.15.97:0/2017418012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 msgr2=0x556eae1d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.299+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1d0300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f06cc010600 tx=0x7f06cc013040 comp rx=0 tx=0).stop 2024-03-26T19:04:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.299+0000 7f06ea5b3740 1 -- 172.21.15.97:0/2017418012 shutdown_connections 2024-03-26T19:04:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.299+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1c4900 0x556eae1c0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.299+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1bf790 0x556eae1d0300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.299+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/2017418012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1beda0 0x556eae1c0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.299+0000 7f06ea5b3740 1 -- 172.21.15.97:0/2017418012 >> 172.21.15.97:0/2017418012 conn(0x556eae1ba310 msgr2=0x556eae1c8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.299+0000 7f06ea5b3740 1 -- 172.21.15.97:0/2017418012 shutdown_connections 2024-03-26T19:04:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.300+0000 7f06ea5b3740 1 -- 172.21.15.97:0/2017418012 wait complete. 2024-03-26T19:04:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.301+0000 7f06ea5b3740 1 -- start start 2024-03-26T19:04:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.302+0000 7f06ea5b3740 1 Processor -- start 2024-03-26T19:04:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.303+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 0x556eae1cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.303+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 0x556eae1cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.303+0000 7f06ea5b3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1c4900 0x556eae239080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.303+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01c1640 con 0x556eae1bf790 2024-03-26T19:04:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.303+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01c0f70 con 0x556eae1c4900 2024-03-26T19:04:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.303+0000 7f06ea5b3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01be1d0 con 0x556eae1beda0 2024-03-26T19:04:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.303+0000 7f06ea5b3740 10 client.0 _ll_register_callbacks cb 0x7f06d80b98e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:04:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.304+0000 7f06e5800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1c4900 0x556eae239080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.304+0000 7f06e5800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1c4900 0x556eae239080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34236/0 (socket says 172.21.15.97:34236) 2024-03-26T19:04:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.304+0000 7f06e5800640 1 -- 172.21.15.97:0/962314846 learned_addr learned my addr 172.21.15.97:0/962314846 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.304+0000 7f06dfe00640 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 0x556eae1cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.304+0000 7f06e4e00640 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 0x556eae1cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.305+0000 7f06e4e00640 1 -- 172.21.15.97:0/962314846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1c4900 msgr2=0x556eae239080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.305+0000 7f06e4e00640 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1c4900 0x556eae239080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.305+0000 7f06e4e00640 1 -- 172.21.15.97:0/962314846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 msgr2=0x556eae1cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.305+0000 7f06e4e00640 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 0x556eae1cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.305+0000 7f06e4e00640 1 -- 172.21.15.97:0/962314846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eae2395c0 con 0x556eae1beda0 2024-03-26T19:04:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.305+0000 7f06dfe00640 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 0x556eae1cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:04:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.305+0000 7f06ea5b3740 1 -- 172.21.15.97:0/962314846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06cc0043c0 con 0x556eae1beda0 2024-03-26T19:04:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.305+0000 7f06e5800640 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1c4900 0x556eae239080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:04:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.305+0000 7f06e4e00640 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 0x556eae1cd2a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06d400a010 tx=0x7f06d4006a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88c59703a566c959 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.305+0000 7f06c6a00640 10 client.25633 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:04:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.306+0000 7f06c6a00640 1 -- 172.21.15.97:0/962314846 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4017040 con 0x556eae1beda0 2024-03-26T19:04:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.306+0000 7f06c6a00640 1 -- 172.21.15.97:0/962314846 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f06d4029ba0 con 0x556eae1beda0 2024-03-26T19:04:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.306+0000 7f06c6a00640 1 -- 172.21.15.97:0/962314846 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06d4029e00 con 0x556eae1beda0 2024-03-26T19:04:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.306+0000 7f06ea5b3740 10 client.25633 fetch_fsmap learned FSMap version 27 2024-03-26T19:04:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.306+0000 7f06ea5b3740 1 -- 172.21.15.97:0/962314846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556eae23a7a0 con 0x556eae1beda0 2024-03-26T19:04:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.307+0000 7f06c6a00640 1 -- 172.21.15.97:0/962314846 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f06d4009460 con 0x556eae1beda0 2024-03-26T19:04:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.308+0000 7f06ea5b3740 10 client.25633 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:04:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.308+0000 7f06ea5b3740 10 client.25633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:04:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.308+0000 7f06ea5b3740 20 client.25633 populate_metadata read hostname 'smithi097' 2024-03-26T19:04:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.308+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556eae23ad60 0x556eae23d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.308+0000 7f06ea5b3740 4 client.25633 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:04:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.308+0000 7f06ea5b3740 1 -- 172.21.15.97:0/962314846 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556eae12b710 con 0x556eae23ad60 2024-03-26T19:04:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.308+0000 7f06dfe00640 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556eae23ad60 0x556eae23d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.309+0000 7f06dfe00640 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556eae23ad60 0x556eae23d150 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e42fdea674dbc29 server_cookie=1942db6f839dd84b in_seq=0 out_seq=0 2024-03-26T19:04:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.309+0000 7f06c6a00640 10 client.25633 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.332+0000 7f06c6a00640 1 -- 172.21.15.97:0/962314846 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556eae12b710 con 0x556eae23ad60 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.332+0000 7f06c6a00640 10 client.25633 handle_command_reply: tid=0 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.332+0000 7f06ea5b3740 1 client.25633 shutdown 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 1 -- 172.21.15.97:0/962314846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 msgr2=0x556eae1cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 0x556eae1cd2a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06d400a010 tx=0x7f06d4006a30 comp rx=0 tx=0).stop 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 1 -- 172.21.15.97:0/962314846 shutdown_connections 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x556eae23ad60 0x556eae23d150 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556eae1c4900 0x556eae239080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556eae1bf790 0x556eae1cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 1 --2- 172.21.15.97:0/962314846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556eae1beda0 0x556eae1cd2a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 1 -- 172.21.15.97:0/962314846 >> 172.21.15.97:0/962314846 conn(0x556eae1ba310 msgr2=0x556eae1bd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06c6a00640 0 client.25633 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 1 -- 172.21.15.97:0/962314846 shutdown_connections 2024-03-26T19:04:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 1 -- 172.21.15.97:0/962314846 wait complete. 2024-03-26T19:04:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.333+0000 7f06ea5b3740 20 client.25633 trim_cache size 0 max 16384 2024-03-26T19:04:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:04:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:04:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:04:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:04:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.335+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f06c80a3180 msgr2=0x7f06c80a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.335+0000 7f06e7800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f06c80a3180 0x7f06c80a5640 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f06cc005d70 tx=0x7f06cc005d00 comp rx=0 tx=0).stop 2024-03-26T19:04:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.335+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06e010bc90 msgr2=0x7f06e01b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.335+0000 7f06e7800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06e010bc90 0x7f06e01b6d00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f06d8037040 tx=0x7f06d8035d10 comp rx=0 tx=0).stop 2024-03-26T19:04:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.335+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 shutdown_connections 2024-03-26T19:04:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.335+0000 7f06e7800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f06c80a3180 0x7f06c80a5640 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.335+0000 7f06e7800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06e010bc90 0x7f06e01b6d00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.335+0000 7f06e7800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06e010b2d0 0x7f06e019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.335+0000 7f06e7800640 1 --2- 172.21.15.97:0/3244451175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06e010a0e0 0x7f06e019e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.335+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 >> 172.21.15.97:0/3244451175 conn(0x7f06e0070880 msgr2=0x7f06e010c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.336+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 shutdown_connections 2024-03-26T19:04:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:16.336+0000 7f06e7800640 1 -- 172.21.15.97:0/3244451175 wait complete. 2024-03-26T19:04:16.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:16 smithi097 ceph-mon[31955]: pgmap v550: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.3 MiB/s wr, 736 op/s 2024-03-26T19:04:16.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:16 smithi144 ceph-mon[36053]: pgmap v550: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.3 MiB/s wr, 736 op/s 2024-03-26T19:04:16.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:16 smithi105 ceph-mon[36270]: pgmap v550: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.3 MiB/s wr, 736 op/s 2024-03-26T19:04:18.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:18 smithi097 ceph-mon[31955]: pgmap v551: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 612 op/s 2024-03-26T19:04:18.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:18 smithi144 ceph-mon[36053]: pgmap v551: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 612 op/s 2024-03-26T19:04:18.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:18 smithi105 ceph-mon[36270]: pgmap v551: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 612 op/s 2024-03-26T19:04:20.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 295.0 s, 2461.8 tps, lat 41.224 ms stddev 49.882 2024-03-26T19:04:20.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:20 smithi097 ceph-mon[31955]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 675 op/s 2024-03-26T19:04:20.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:20 smithi144 ceph-mon[36053]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 675 op/s 2024-03-26T19:04:20.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:20 smithi105 ceph-mon[36270]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 675 op/s 2024-03-26T19:04:21.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:21 smithi144 ceph-mon[36053]: pgmap v553: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.5 MiB/s wr, 633 op/s 2024-03-26T19:04:21.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:21 smithi105 ceph-mon[36270]: pgmap v553: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.5 MiB/s wr, 633 op/s 2024-03-26T19:04:21.855 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:21 smithi097 ceph-mon[31955]: pgmap v553: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.5 MiB/s wr, 633 op/s 2024-03-26T19:04:24.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:24 smithi105 ceph-mon[36270]: pgmap v554: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-03-26T19:04:24.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:24 smithi097 ceph-mon[31955]: pgmap v554: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-03-26T19:04:24.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:24 smithi144 ceph-mon[36053]: pgmap v554: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-03-26T19:04:25.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 300.0 s, 2462.6 tps, lat 40.247 ms stddev 46.340 2024-03-26T19:04:26.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:26 smithi105 ceph-mon[36270]: pgmap v555: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.8 MiB/s wr, 739 op/s 2024-03-26T19:04:26.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:26 smithi097 ceph-mon[31955]: pgmap v555: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.8 MiB/s wr, 739 op/s 2024-03-26T19:04:26.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:26 smithi144 ceph-mon[36053]: pgmap v555: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.8 MiB/s wr, 739 op/s 2024-03-26T19:04:28.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:28 smithi105 ceph-mon[36270]: pgmap v556: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 659 op/s 2024-03-26T19:04:28.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:28 smithi097 ceph-mon[31955]: pgmap v556: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 659 op/s 2024-03-26T19:04:28.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:28 smithi144 ceph-mon[36053]: pgmap v556: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 659 op/s 2024-03-26T19:04:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.531+0000 7f8850200640 1 -- 172.21.15.97:0/83126016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8848102390 msgr2=0x7f8848108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.531+0000 7f8850200640 1 --2- 172.21.15.97:0/83126016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8848102390 0x7f8848108c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8840009ca0 tx=0x7f884002c100 comp rx=0 tx=0).stop 2024-03-26T19:04:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.532+0000 7f8850200640 1 -- 172.21.15.97:0/83126016 shutdown_connections 2024-03-26T19:04:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.532+0000 7f8850200640 1 --2- 172.21.15.97:0/83126016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8848102390 0x7f8848108c20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.532+0000 7f8850200640 1 --2- 172.21.15.97:0/83126016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88481019d0 0x7f8848101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.532+0000 7f8850200640 1 --2- 172.21.15.97:0/83126016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88481007e0 0x7f8848100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.532+0000 7f8850200640 1 -- 172.21.15.97:0/83126016 >> 172.21.15.97:0/83126016 conn(0x7f88480fbd50 msgr2=0x7f88480fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.532+0000 7f8850200640 1 -- 172.21.15.97:0/83126016 shutdown_connections 2024-03-26T19:04:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.532+0000 7f8850200640 1 -- 172.21.15.97:0/83126016 wait complete. 2024-03-26T19:04:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.533+0000 7f8850200640 1 Processor -- start 2024-03-26T19:04:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.534+0000 7f8850200640 1 -- start start 2024-03-26T19:04:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.534+0000 7f8850200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88481007e0 0x7f8848199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.534+0000 7f8850200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88481019d0 0x7f884819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.534+0000 7f8850200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8848102390 0x7f88481a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.534+0000 7f8850200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f884810a110 con 0x7f8848102390 2024-03-26T19:04:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.534+0000 7f8850200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8848109f90 con 0x7f88481019d0 2024-03-26T19:04:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.534+0000 7f8850200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f884810a290 con 0x7f88481007e0 2024-03-26T19:04:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.534+0000 7f884d800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88481007e0 0x7f8848199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.534+0000 7f884d800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88481007e0 0x7f8848199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56298/0 (socket says 172.21.15.97:56298) 2024-03-26T19:04:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.534+0000 7f884d800640 1 -- 172.21.15.97:0/3272754305 learned_addr learned my addr 172.21.15.97:0/3272754305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.535+0000 7f884e200640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8848102390 0x7f88481a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.535+0000 7f884d800640 1 -- 172.21.15.97:0/3272754305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88481019d0 msgr2=0x7f884819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.535+0000 7f884ce00640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88481019d0 0x7f884819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.535+0000 7f884d800640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88481019d0 0x7f884819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.535+0000 7f884d800640 1 -- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8848102390 msgr2=0x7f88481a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.535+0000 7f884d800640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8848102390 0x7f88481a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.535+0000 7f884d800640 1 -- 172.21.15.97:0/3272754305 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884806b3e0 con 0x7f88481007e0 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.535+0000 7f884e200640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8848102390 0x7f88481a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.535+0000 7f884ce00640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88481019d0 0x7f884819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.536+0000 7f884d800640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88481007e0 0x7f8848199d20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f883c00e9c0 tx=0x7f883c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.536+0000 7f8846000640 1 -- 172.21.15.97:0/3272754305 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c00cde0 con 0x7f88481007e0 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.536+0000 7f8846000640 1 -- 172.21.15.97:0/3272754305 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883c010910 con 0x7f88481007e0 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.536+0000 7f8846000640 1 -- 172.21.15.97:0/3272754305 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c00cde0 con 0x7f88481007e0 2024-03-26T19:04:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.536+0000 7f8850200640 1 -- 172.21.15.97:0/3272754305 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884806b6d0 con 0x7f88481007e0 2024-03-26T19:04:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.536+0000 7f8850200640 1 -- 172.21.15.97:0/3272754305 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884806bbd0 con 0x7f88481007e0 2024-03-26T19:04:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.541+0000 7f8846000640 1 -- 172.21.15.97:0/3272754305 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f883c01f020 con 0x7f88481007e0 2024-03-26T19:04:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.542+0000 7f885300e740 1 Processor -- start 2024-03-26T19:04:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.542+0000 7f885300e740 1 -- start start 2024-03-26T19:04:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.543+0000 7f885300e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 0x558c8fa711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.543+0000 7f885300e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa71790 0x558c8fa76290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.543+0000 7f885300e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 0x558c8fa78d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.543+0000 7f885300e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558c8f9d5e80 con 0x558c8fa70da0 2024-03-26T19:04:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.543+0000 7f885300e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c8f97e660 con 0x558c8fa71790 2024-03-26T19:04:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.543+0000 7f885300e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c8f9d1e30 con 0x558c8fa76900 2024-03-26T19:04:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.543+0000 7f8846000640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f88300a32d0 0x7f88300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.543+0000 7f884e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 0x558c8fa711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.544+0000 7f8846000640 1 -- 172.21.15.97:0/3272754305 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f883c0cf320 con 0x7f88481007e0 2024-03-26T19:04:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.544+0000 7f884ce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 0x558c8fa78d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.544+0000 7f884e200640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f88300a32d0 0x7f88300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.544+0000 7f884ce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 0x558c8fa78d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56304/0 (socket says 172.21.15.97:56304) 2024-03-26T19:04:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.544+0000 7f884ce00640 1 -- 172.21.15.97:0/1912168268 learned_addr learned my addr 172.21.15.97:0/1912168268 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.544+0000 7f884d800640 1 --2- 172.21.15.97:0/1912168268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa71790 0x558c8fa76290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.545+0000 7f884d800640 1 -- 172.21.15.97:0/1912168268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 msgr2=0x558c8fa78d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.545+0000 7f884d800640 1 --2- 172.21.15.97:0/1912168268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 0x558c8fa78d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.545+0000 7f884d800640 1 -- 172.21.15.97:0/1912168268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 msgr2=0x558c8fa711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.545+0000 7f884d800640 1 --2- 172.21.15.97:0/1912168268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 0x558c8fa711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.545+0000 7f884d800640 1 -- 172.21.15.97:0/1912168268 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8f6ea210 con 0x558c8fa71790 2024-03-26T19:04:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.545+0000 7f884e200640 1 --2- 172.21.15.97:0/1912168268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 0x558c8fa711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:04:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.545+0000 7f884d800640 1 --2- 172.21.15.97:0/1912168268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa71790 0x558c8fa76290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f883c09b720 tx=0x7f883c099c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4d6cf84724c349c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.546+0000 7f884e200640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f88300a32d0 0x7f88300a5790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8840005e30 tx=0x7f884002c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.546+0000 7f883aa00640 1 -- 172.21.15.97:0/1912168268 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c0b3020 con 0x558c8fa71790 2024-03-26T19:04:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.546+0000 7f883aa00640 1 -- 172.21.15.97:0/1912168268 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883c09fb70 con 0x558c8fa71790 2024-03-26T19:04:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.546+0000 7f883aa00640 1 -- 172.21.15.97:0/1912168268 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c0a4200 con 0x558c8fa71790 2024-03-26T19:04:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.547+0000 7f885300e740 1 -- 172.21.15.97:0/1912168268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa71790 msgr2=0x558c8fa76290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.547+0000 7f885300e740 1 --2- 172.21.15.97:0/1912168268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa71790 0x558c8fa76290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f883c09b720 tx=0x7f883c099c20 comp rx=0 tx=0).stop 2024-03-26T19:04:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.547+0000 7f885300e740 1 -- 172.21.15.97:0/1912168268 shutdown_connections 2024-03-26T19:04:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.547+0000 7f885300e740 1 --2- 172.21.15.97:0/1912168268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 0x558c8fa78d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.547+0000 7f885300e740 1 --2- 172.21.15.97:0/1912168268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa71790 0x558c8fa76290 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.547+0000 7f885300e740 1 --2- 172.21.15.97:0/1912168268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 0x558c8fa711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.547+0000 7f885300e740 1 -- 172.21.15.97:0/1912168268 >> 172.21.15.97:0/1912168268 conn(0x558c8fa6c310 msgr2=0x558c8fa6c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.547+0000 7f885300e740 1 -- 172.21.15.97:0/1912168268 shutdown_connections 2024-03-26T19:04:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.547+0000 7f885300e740 1 -- 172.21.15.97:0/1912168268 wait complete. 2024-03-26T19:04:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.548+0000 7f885300e740 1 -- start start 2024-03-26T19:04:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.548+0000 7f885300e740 1 Processor -- start 2024-03-26T19:04:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.549+0000 7f885300e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa70da0 0x558c8fa835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.549+0000 7f885300e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 0x558c8fa83b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.549+0000 7f885300e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 0x558c8fa8ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.549+0000 7f885300e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558c8fa820c0 con 0x558c8fa71790 2024-03-26T19:04:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.549+0000 7f885300e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c8f991ac0 con 0x558c8fa70da0 2024-03-26T19:04:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.549+0000 7f885300e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c8fa82240 con 0x558c8fa76900 2024-03-26T19:04:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.549+0000 7f885300e740 10 client.0 _ll_register_callbacks cb 0x558c8f991490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:04:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884d800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa70da0 0x558c8fa835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884d800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa70da0 0x558c8fa835f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:35644/0 (socket says 172.21.15.97:35644) 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884e200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 0x558c8fa8ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884d800640 1 -- 172.21.15.97:0/1406138233 learned_addr learned my addr 172.21.15.97:0/1406138233 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884ce00640 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 0x558c8fa83b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884e200640 1 -- 172.21.15.97:0/1406138233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa70da0 msgr2=0x558c8fa835f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884e200640 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa70da0 0x558c8fa835f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884e200640 1 -- 172.21.15.97:0/1406138233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 msgr2=0x558c8fa83b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884e200640 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 0x558c8fa83b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884e200640 1 -- 172.21.15.97:0/1406138233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8f7b6620 con 0x558c8fa76900 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884d800640 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa70da0 0x558c8fa835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.550+0000 7f884ce00640 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 0x558c8fa83b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.551+0000 7f884e200640 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 0x558c8fa8ab70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f884002c640 tx=0x7f88400095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80946aff9da9fbca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.551+0000 7f882f400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.551+0000 7f882f400640 1 -- 172.21.15.97:0/1406138233 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840052060 con 0x558c8fa76900 2024-03-26T19:04:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.551+0000 7f882f400640 1 -- 172.21.15.97:0/1406138233 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8840004040 con 0x558c8fa76900 2024-03-26T19:04:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.551+0000 7f885300e740 1 -- 172.21.15.97:0/1406138233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f883c09fb70 con 0x558c8fa76900 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.551+0000 7f882f400640 1 -- 172.21.15.97:0/1406138233 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8840039040 con 0x558c8fa76900 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.551+0000 7f885300e740 10 client.25645 fetch_fsmap learned FSMap version 27 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.551+0000 7f885300e740 1 -- 172.21.15.97:0/1406138233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c8fa8df50 con 0x558c8fa76900 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.552+0000 7f882f400640 1 -- 172.21.15.97:0/1406138233 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883c09fb70 con 0x558c8fa76900 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.552+0000 7f882f400640 1 -- 172.21.15.97:0/1406138233 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f884003f070 con 0x558c8fa76900 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.552+0000 7f885300e740 10 client.25645 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.552+0000 7f885300e740 10 client.25645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.552+0000 7f885300e740 20 client.25645 populate_metadata read hostname 'smithi097' 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.552+0000 7f885300e740 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558c8fa8f690 0x558c8fa91a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.552+0000 7f885300e740 4 client.25645 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.552+0000 7f885300e740 1 -- 172.21.15.97:0/1406138233 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c8f9e33c0 con 0x558c8fa8f690 2024-03-26T19:04:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.553+0000 7f884d800640 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558c8fa8f690 0x558c8fa91a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.554+0000 7f882f400640 10 client.25645 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.554+0000 7f884d800640 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558c8fa8f690 0x558c8fa91a80 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ea6c48432dd17e server_cookie=8541a6321d07f4af in_seq=0 out_seq=0 2024-03-26T19:04:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.554+0000 7f882f400640 1 -- 172.21.15.97:0/1406138233 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558c8f9e33c0 con 0x558c8fa8f690 2024-03-26T19:04:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.554+0000 7f882f400640 10 client.25645 handle_command_reply: tid=0 2024-03-26T19:04:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.555+0000 7f885300e740 1 client.25645 shutdown 2024-03-26T19:04:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.556+0000 7f885300e740 1 -- 172.21.15.97:0/1406138233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 msgr2=0x558c8fa8ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.556+0000 7f885300e740 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 0x558c8fa8ab70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f884002c640 tx=0x7f88400095c0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.557+0000 7f885300e740 1 -- 172.21.15.97:0/1406138233 shutdown_connections 2024-03-26T19:04:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.557+0000 7f885300e740 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558c8fa8f690 0x558c8fa91a80 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.557+0000 7f885300e740 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa76900 0x558c8fa8ab70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.557+0000 7f885300e740 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 0x558c8fa83b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.557+0000 7f885300e740 1 --2- 172.21.15.97:0/1406138233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa70da0 0x558c8fa835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.557+0000 7f885300e740 1 -- 172.21.15.97:0/1406138233 >> 172.21.15.97:0/1406138233 conn(0x558c8fa6c310 msgr2=0x558c8fa6b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.557+0000 7f882f400640 0 client.25645 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.557+0000 7f885300e740 1 -- 172.21.15.97:0/1406138233 shutdown_connections 2024-03-26T19:04:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.558+0000 7f885300e740 1 -- 172.21.15.97:0/1406138233 wait complete. 2024-03-26T19:04:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.558+0000 7f885300e740 20 client.25645 trim_cache size 0 max 16384 2024-03-26T19:04:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.584+0000 7f885300e740 1 Processor -- start 2024-03-26T19:04:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.584+0000 7f885300e740 1 -- start start 2024-03-26T19:04:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.585+0000 7f885300e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 0x558c8fa6b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.585+0000 7f885300e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa71790 0x558c8fa8ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.585+0000 7f885300e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 0x558c8fa96340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.585+0000 7f885300e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8848070560 con 0x558c8fa70da0 2024-03-26T19:04:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.585+0000 7f885300e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88480720a0 con 0x558c8fa76900 2024-03-26T19:04:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.585+0000 7f885300e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8848070c30 con 0x558c8fa71790 2024-03-26T19:04:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.585+0000 7f884d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 0x558c8fa6b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.585+0000 7f884e200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 0x558c8fa96340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.585+0000 7f884e200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 0x558c8fa96340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:35654/0 (socket says 172.21.15.97:35654) 2024-03-26T19:04:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.585+0000 7f884e200640 1 -- 172.21.15.97:0/2186664902 learned_addr learned my addr 172.21.15.97:0/2186664902 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.586+0000 7f884ce00640 1 --2- 172.21.15.97:0/2186664902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa71790 0x558c8fa8ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.586+0000 7f884e200640 1 -- 172.21.15.97:0/2186664902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa71790 msgr2=0x558c8fa8ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.586+0000 7f884e200640 1 --2- 172.21.15.97:0/2186664902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa71790 0x558c8fa8ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.587+0000 7f884e200640 1 -- 172.21.15.97:0/2186664902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 msgr2=0x558c8fa6b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.587+0000 7f884e200640 1 --2- 172.21.15.97:0/2186664902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 0x558c8fa6b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.587+0000 7f884e200640 1 -- 172.21.15.97:0/2186664902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8fa91df0 con 0x558c8fa76900 2024-03-26T19:04:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.587+0000 7f884d800640 1 --2- 172.21.15.97:0/2186664902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 0x558c8fa6b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:04:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.587+0000 7f884e200640 1 --2- 172.21.15.97:0/2186664902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 0x558c8fa96340 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8840037040 tx=0x7f8840009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94d82755a2cf3cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.587+0000 7f883a000640 1 -- 172.21.15.97:0/2186664902 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840039040 con 0x558c8fa76900 2024-03-26T19:04:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.587+0000 7f883a000640 1 -- 172.21.15.97:0/2186664902 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8840005310 con 0x558c8fa76900 2024-03-26T19:04:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.587+0000 7f883a000640 1 -- 172.21.15.97:0/2186664902 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840004d30 con 0x558c8fa76900 2024-03-26T19:04:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.588+0000 7f885300e740 1 -- 172.21.15.97:0/2186664902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 msgr2=0x558c8fa96340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.588+0000 7f885300e740 1 --2- 172.21.15.97:0/2186664902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 0x558c8fa96340 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8840037040 tx=0x7f8840009360 comp rx=0 tx=0).stop 2024-03-26T19:04:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.588+0000 7f885300e740 1 -- 172.21.15.97:0/2186664902 shutdown_connections 2024-03-26T19:04:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.588+0000 7f885300e740 1 --2- 172.21.15.97:0/2186664902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 0x558c8fa96340 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.588+0000 7f885300e740 1 --2- 172.21.15.97:0/2186664902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa71790 0x558c8fa8ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.588+0000 7f885300e740 1 --2- 172.21.15.97:0/2186664902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa70da0 0x558c8fa6b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.588+0000 7f885300e740 1 -- 172.21.15.97:0/2186664902 >> 172.21.15.97:0/2186664902 conn(0x558c8fa6c310 msgr2=0x558c8fa6acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.589+0000 7f885300e740 1 -- 172.21.15.97:0/2186664902 shutdown_connections 2024-03-26T19:04:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.589+0000 7f885300e740 1 -- 172.21.15.97:0/2186664902 wait complete. 2024-03-26T19:04:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.589+0000 7f885300e740 1 -- start start 2024-03-26T19:04:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.590+0000 7f885300e740 1 Processor -- start 2024-03-26T19:04:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.590+0000 7f885300e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa70da0 0x558c8fa88780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.590+0000 7f885300e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 0x558c8fa88cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.590+0000 7f885300e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 0x558c8fa89cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.590+0000 7f885300e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88480719d0 con 0x558c8fa71790 2024-03-26T19:04:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.590+0000 7f885300e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8848071300 con 0x558c8fa76900 2024-03-26T19:04:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.590+0000 7f885300e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f884806e520 con 0x558c8fa70da0 2024-03-26T19:04:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.590+0000 7f885300e740 10 client.0 _ll_register_callbacks cb 0x7f8840003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:04:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.591+0000 7f884d800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa70da0 0x558c8fa88780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.591+0000 7f884d800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa70da0 0x558c8fa88780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56334/0 (socket says 172.21.15.97:56334) 2024-03-26T19:04:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.591+0000 7f884d800640 1 -- 172.21.15.97:0/637953210 learned_addr learned my addr 172.21.15.97:0/637953210 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.591+0000 7f884e200640 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 0x558c8fa89cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.592+0000 7f884ce00640 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 0x558c8fa88cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.594+0000 7f884d800640 1 -- 172.21.15.97:0/637953210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 msgr2=0x558c8fa89cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.594+0000 7f884d800640 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 0x558c8fa89cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.594+0000 7f884d800640 1 -- 172.21.15.97:0/637953210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 msgr2=0x558c8fa88cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.594+0000 7f884d800640 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 0x558c8fa88cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.594+0000 7f884d800640 1 -- 172.21.15.97:0/637953210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8fa8a2c0 con 0x558c8fa70da0 2024-03-26T19:04:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.594+0000 7f884ce00640 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 0x558c8fa88cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:04:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.594+0000 7f884d800640 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa70da0 0x558c8fa88780 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f883c098360 tx=0x7f883c099f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e770cf17a0a15ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.594+0000 7f882f400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:04:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.595+0000 7f882f400640 1 -- 172.21.15.97:0/637953210 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c0b3040 con 0x558c8fa70da0 2024-03-26T19:04:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.595+0000 7f882f400640 1 -- 172.21.15.97:0/637953210 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f883c0862b0 con 0x558c8fa70da0 2024-03-26T19:04:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.595+0000 7f885300e740 1 -- 172.21.15.97:0/637953210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8840005310 con 0x558c8fa70da0 2024-03-26T19:04:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.596+0000 7f882f400640 1 -- 172.21.15.97:0/637953210 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f883c09fb70 con 0x558c8fa70da0 2024-03-26T19:04:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.597+0000 7f885300e740 10 client.25651 fetch_fsmap learned FSMap version 27 2024-03-26T19:04:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.597+0000 7f885300e740 1 -- 172.21.15.97:0/637953210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c8faf5f10 con 0x558c8fa70da0 2024-03-26T19:04:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.597+0000 7f882f400640 1 -- 172.21.15.97:0/637953210 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8840005310 con 0x558c8fa70da0 2024-03-26T19:04:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.597+0000 7f882f400640 1 -- 172.21.15.97:0/637953210 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f883c0ba220 con 0x558c8fa70da0 2024-03-26T19:04:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.597+0000 7f885300e740 10 client.25651 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:04:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.597+0000 7f885300e740 10 client.25651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:04:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.597+0000 7f885300e740 20 client.25651 populate_metadata read hostname 'smithi097' 2024-03-26T19:04:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.597+0000 7f885300e740 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558c8faf6400 0x558c8faf8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.598+0000 7f885300e740 4 client.25651 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:04:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.598+0000 7f885300e740 1 -- 172.21.15.97:0/637953210 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c8f9dd710 con 0x558c8faf6400 2024-03-26T19:04:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.598+0000 7f884ce00640 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558c8faf6400 0x558c8faf8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.604+0000 7f884ce00640 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558c8faf6400 0x558c8faf8820 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a765ca38f9ea5860 server_cookie=d540c9c94bffdbfb in_seq=0 out_seq=0 2024-03-26T19:04:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.604+0000 7f882f400640 10 client.25651 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.676+0000 7f882f400640 1 -- 172.21.15.97:0/637953210 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c8f9dd710 con 0x558c8faf6400 2024-03-26T19:04:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f882f400640 10 client.25651 handle_command_reply: tid=0 2024-03-26T19:04:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f885300e740 1 client.25651 shutdown 2024-03-26T19:04:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f885300e740 1 -- 172.21.15.97:0/637953210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa70da0 msgr2=0x558c8fa88780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f885300e740 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa70da0 0x558c8fa88780 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f883c098360 tx=0x7f883c099f30 comp rx=0 tx=0).stop 2024-03-26T19:04:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f885300e740 1 -- 172.21.15.97:0/637953210 shutdown_connections 2024-03-26T19:04:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f885300e740 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558c8faf6400 0x558c8faf8820 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f885300e740 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c8fa76900 0x558c8fa89cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f882f400640 0 client.25651 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f885300e740 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c8fa71790 0x558c8fa88cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f885300e740 1 --2- 172.21.15.97:0/637953210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c8fa70da0 0x558c8fa88780 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.677+0000 7f885300e740 1 -- 172.21.15.97:0/637953210 >> 172.21.15.97:0/637953210 conn(0x558c8fa6c310 msgr2=0x558c8fa6b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.678+0000 7f885300e740 1 -- 172.21.15.97:0/637953210 shutdown_connections 2024-03-26T19:04:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.678+0000 7f885300e740 1 -- 172.21.15.97:0/637953210 wait complete. 2024-03-26T19:04:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.678+0000 7f885300e740 20 client.25651 trim_cache size 0 max 16384 2024-03-26T19:04:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:04:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:04:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:04:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:04:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 -- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f88300a32d0 msgr2=0x7f88300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f88300a32d0 0x7f88300a5790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8840005e30 tx=0x7f884002c100 comp rx=0 tx=0).stop 2024-03-26T19:04:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 -- 172.21.15.97:0/3272754305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88481007e0 msgr2=0x7f8848199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88481007e0 0x7f8848199d20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f883c00e9c0 tx=0x7f883c00ee80 comp rx=0 tx=0).stop 2024-03-26T19:04:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 -- 172.21.15.97:0/3272754305 shutdown_connections 2024-03-26T19:04:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f88300a32d0 0x7f88300a5790 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8848102390 0x7f88481a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88481019d0 0x7f884819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 --2- 172.21.15.97:0/3272754305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88481007e0 0x7f8848199d20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 -- 172.21.15.97:0/3272754305 >> 172.21.15.97:0/3272754305 conn(0x7f88480fbd50 msgr2=0x7f88480fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 -- 172.21.15.97:0/3272754305 shutdown_connections 2024-03-26T19:04:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:29.680+0000 7f8850200640 1 -- 172.21.15.97:0/3272754305 wait complete. 2024-03-26T19:04:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 305.0 s, 1912.4 tps, lat 52.512 ms stddev 61.333 2024-03-26T19:04:30.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:30 smithi105 ceph-mon[36270]: pgmap v557: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 9.9 MiB/s wr, 770 op/s 2024-03-26T19:04:30.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:30 smithi097 ceph-mon[31955]: pgmap v557: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 9.9 MiB/s wr, 770 op/s 2024-03-26T19:04:30.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:30 smithi144 ceph-mon[36053]: pgmap v557: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 9.9 MiB/s wr, 770 op/s 2024-03-26T19:04:32.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:32 smithi105 ceph-mon[36270]: pgmap v558: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 9.5 MiB/s wr, 729 op/s 2024-03-26T19:04:32.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:32 smithi097 ceph-mon[31955]: pgmap v558: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 9.5 MiB/s wr, 729 op/s 2024-03-26T19:04:32.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:32 smithi144 ceph-mon[36053]: pgmap v558: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 9.5 MiB/s wr, 729 op/s 2024-03-26T19:04:34.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:34 smithi105 ceph-mon[36270]: pgmap v559: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 9.3 MiB/s wr, 709 op/s 2024-03-26T19:04:34.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:34 smithi097 ceph-mon[31955]: pgmap v559: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 9.3 MiB/s wr, 709 op/s 2024-03-26T19:04:34.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:34 smithi144 ceph-mon[36053]: pgmap v559: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 9.3 MiB/s wr, 709 op/s 2024-03-26T19:04:35.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 310.0 s, 2293.1 tps, lat 43.579 ms stddev 53.249 2024-03-26T19:04:36.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:36 smithi105 ceph-mon[36270]: pgmap v560: 97 pgs: 97 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 9.7 MiB/s wr, 806 op/s 2024-03-26T19:04:36.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:36 smithi097 ceph-mon[31955]: pgmap v560: 97 pgs: 97 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 9.7 MiB/s wr, 806 op/s 2024-03-26T19:04:36.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:36 smithi144 ceph-mon[36053]: pgmap v560: 97 pgs: 97 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 9.7 MiB/s wr, 806 op/s 2024-03-26T19:04:38.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:38 smithi105 ceph-mon[36270]: pgmap v561: 97 pgs: 97 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 8.4 MiB/s wr, 704 op/s 2024-03-26T19:04:38.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:38 smithi097 ceph-mon[31955]: pgmap v561: 97 pgs: 97 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 8.4 MiB/s wr, 704 op/s 2024-03-26T19:04:38.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:38 smithi144 ceph-mon[36053]: pgmap v561: 97 pgs: 97 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 8.4 MiB/s wr, 704 op/s 2024-03-26T19:04:40.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 315.0 s, 2253.0 tps, lat 44.122 ms stddev 53.476 2024-03-26T19:04:40.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:40 smithi097 ceph-mon[31955]: pgmap v562: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 9.4 MiB/s wr, 803 op/s 2024-03-26T19:04:40.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:40 smithi144 ceph-mon[36053]: pgmap v562: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 9.4 MiB/s wr, 803 op/s 2024-03-26T19:04:40.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:40 smithi105 ceph-mon[36270]: pgmap v562: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 9.4 MiB/s wr, 803 op/s 2024-03-26T19:04:42.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:42 smithi097 ceph-mon[31955]: pgmap v563: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.6 MiB/s wr, 727 op/s 2024-03-26T19:04:42.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:42 smithi144 ceph-mon[36053]: pgmap v563: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.6 MiB/s wr, 727 op/s 2024-03-26T19:04:42.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:42 smithi105 ceph-mon[36270]: pgmap v563: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.6 MiB/s wr, 727 op/s 2024-03-26T19:04:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.873+0000 7f9e6c200640 1 -- 172.21.15.97:0/481905004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e64103b20 msgr2=0x7f9e64103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.873+0000 7f9e6c200640 1 --2- 172.21.15.97:0/481905004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e64103b20 0x7f9e64103f40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009900 tx=0x7f9e5802bc40 comp rx=0 tx=0).stop 2024-03-26T19:04:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.873+0000 7f9e6c200640 1 -- 172.21.15.97:0/481905004 shutdown_connections 2024-03-26T19:04:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.873+0000 7f9e6c200640 1 --2- 172.21.15.97:0/481905004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e641056d0 0x7f9e6410bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.873+0000 7f9e6c200640 1 --2- 172.21.15.97:0/481905004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e64104d10 0x7f9e64105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.873+0000 7f9e6c200640 1 --2- 172.21.15.97:0/481905004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e64103b20 0x7f9e64103f40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.873+0000 7f9e6c200640 1 -- 172.21.15.97:0/481905004 >> 172.21.15.97:0/481905004 conn(0x7f9e640ff0b0 msgr2=0x7f9e641014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.873+0000 7f9e6c200640 1 -- 172.21.15.97:0/481905004 shutdown_connections 2024-03-26T19:04:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.873+0000 7f9e6c200640 1 -- 172.21.15.97:0/481905004 wait complete. 2024-03-26T19:04:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.874+0000 7f9e6c200640 1 Processor -- start 2024-03-26T19:04:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.875+0000 7f9e6c200640 1 -- start start 2024-03-26T19:04:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.875+0000 7f9e6c200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e64103b20 0x7f9e6419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.875+0000 7f9e6c200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e64104d10 0x7f9e6419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.875+0000 7f9e6c200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e641056d0 0x7f9e641a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.875+0000 7f9e6c200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e64077830 con 0x7f9e64103b20 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.875+0000 7f9e6c200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e640776b0 con 0x7f9e641056d0 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.875+0000 7f9e6c200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e640779b0 con 0x7f9e64104d10 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.876+0000 7f9e6a200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e641056d0 0x7f9e641a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.876+0000 7f9e6a200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e641056d0 0x7f9e641a5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:37780/0 (socket says 172.21.15.97:37780) 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.876+0000 7f9e6a200640 1 -- 172.21.15.97:0/30861224 learned_addr learned my addr 172.21.15.97:0/30861224 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.876+0000 7f9e68e00640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e64104d10 0x7f9e6419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.876+0000 7f9e69800640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e64103b20 0x7f9e6419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.877+0000 7f9e6a200640 1 -- 172.21.15.97:0/30861224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e64104d10 msgr2=0x7f9e6419e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.877+0000 7f9e6a200640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e64104d10 0x7f9e6419e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.877+0000 7f9e6a200640 1 -- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e64103b20 msgr2=0x7f9e6419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.877+0000 7f9e6a200640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e64103b20 0x7f9e6419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.877+0000 7f9e6a200640 1 -- 172.21.15.97:0/30861224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e641a5d90 con 0x7f9e641056d0 2024-03-26T19:04:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.877+0000 7f9e69800640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e64103b20 0x7f9e6419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:04:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.878+0000 7f9e6a200640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e641056d0 0x7f9e641a5720 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00cad0 tx=0x7f9e5c00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.878+0000 7f9e62000640 1 -- 172.21.15.97:0/30861224 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00ae20 con 0x7f9e641056d0 2024-03-26T19:04:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.878+0000 7f9e62000640 1 -- 172.21.15.97:0/30861224 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5c005020 con 0x7f9e641056d0 2024-03-26T19:04:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.878+0000 7f9e62000640 1 -- 172.21.15.97:0/30861224 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00f6f0 con 0x7f9e641056d0 2024-03-26T19:04:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.879+0000 7f9e6c200640 1 -- 172.21.15.97:0/30861224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e641a6080 con 0x7f9e641056d0 2024-03-26T19:04:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.879+0000 7f9e6c200640 1 -- 172.21.15.97:0/30861224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e641a6580 con 0x7f9e641056d0 2024-03-26T19:04:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.883+0000 7f9e6ef90740 1 Processor -- start 2024-03-26T19:04:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.884+0000 7f9e62000640 1 -- 172.21.15.97:0/30861224 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f9e5c0051d0 con 0x7f9e641056d0 2024-03-26T19:04:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.884+0000 7f9e6ef90740 1 -- start start 2024-03-26T19:04:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.884+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f908da0 0x562d0f9091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.884+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f909790 0x562d0f90e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.884+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f90e900 0x562d0f910d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.884+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562d0f86de80 con 0x562d0f90e900 2024-03-26T19:04:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.884+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562d0f816660 con 0x562d0f909790 2024-03-26T19:04:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.884+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562d0f869e30 con 0x562d0f908da0 2024-03-26T19:04:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.885+0000 7f9e69800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f909790 0x562d0f90e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.885+0000 7f9e69800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f909790 0x562d0f90e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:37782/0 (socket says 172.21.15.97:37782) 2024-03-26T19:04:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.885+0000 7f9e68e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f908da0 0x562d0f9091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.885+0000 7f9e69800640 1 -- 172.21.15.97:0/2893538850 learned_addr learned my addr 172.21.15.97:0/2893538850 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.885+0000 7f9e69800640 1 -- 172.21.15.97:0/2893538850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f908da0 msgr2=0x562d0f9091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.885+0000 7f9e69800640 1 --2- 172.21.15.97:0/2893538850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f908da0 0x562d0f9091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.885+0000 7f9e69800640 1 -- 172.21.15.97:0/2893538850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f90e900 msgr2=0x562d0f910d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.885+0000 7f9e69800640 1 --2- 172.21.15.97:0/2893538850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f90e900 0x562d0f910d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.885+0000 7f9e69800640 1 -- 172.21.15.97:0/2893538850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d0f582210 con 0x562d0f909790 2024-03-26T19:04:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.885+0000 7f9e68e00640 1 --2- 172.21.15.97:0/2893538850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f908da0 0x562d0f9091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.886+0000 7f9e69800640 1 --2- 172.21.15.97:0/2893538850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f909790 0x562d0f90e290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009900 tx=0x7f9e58005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28228026418d2973 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.886+0000 7f9e56a00640 1 -- 172.21.15.97:0/2893538850 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58042070 con 0x562d0f909790 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.886+0000 7f9e62000640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e4c0a3040 0x7f9e4c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.886+0000 7f9e56a00640 1 -- 172.21.15.97:0/2893538850 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5802cc10 con 0x562d0f909790 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.886+0000 7f9e56a00640 1 -- 172.21.15.97:0/2893538850 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e580048e0 con 0x562d0f909790 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.886+0000 7f9e62000640 1 -- 172.21.15.97:0/30861224 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9e5c0cf480 con 0x7f9e641056d0 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.886+0000 7f9e68e00640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e4c0a3040 0x7f9e4c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.887+0000 7f9e6ef90740 1 -- 172.21.15.97:0/2893538850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f909790 msgr2=0x562d0f90e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.887+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/2893538850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f909790 0x562d0f90e290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009900 tx=0x7f9e58005d50 comp rx=0 tx=0).stop 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.887+0000 7f9e6ef90740 1 -- 172.21.15.97:0/2893538850 shutdown_connections 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.887+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/2893538850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f90e900 0x562d0f910d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.887+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/2893538850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f909790 0x562d0f90e290 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.887+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/2893538850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f908da0 0x562d0f9091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.887+0000 7f9e6ef90740 1 -- 172.21.15.97:0/2893538850 >> 172.21.15.97:0/2893538850 conn(0x562d0f904310 msgr2=0x562d0f904710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.888+0000 7f9e6ef90740 1 -- 172.21.15.97:0/2893538850 shutdown_connections 2024-03-26T19:04:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.889+0000 7f9e68e00640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e4c0a3040 0x7f9e4c0a5500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9e50004650 tx=0x7f9e500095d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.889+0000 7f9e6ef90740 1 -- 172.21.15.97:0/2893538850 wait complete. 2024-03-26T19:04:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.890+0000 7f9e6ef90740 1 -- start start 2024-03-26T19:04:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.891+0000 7f9e6ef90740 1 Processor -- start 2024-03-26T19:04:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.892+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f908da0 0x562d0f912d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.892+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f909790 0x562d0f9132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.892+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f90e900 0x562d0f91a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.892+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562d0f911850 con 0x562d0f909790 2024-03-26T19:04:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.892+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562d0f829ac0 con 0x562d0f908da0 2024-03-26T19:04:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.892+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562d0f9119d0 con 0x562d0f90e900 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.892+0000 7f9e6ef90740 10 client.0 _ll_register_callbacks cb 0x562d0f829490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.893+0000 7f9e69800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f908da0 0x562d0f912d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.893+0000 7f9e69800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f908da0 0x562d0f912d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:37798/0 (socket says 172.21.15.97:37798) 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.893+0000 7f9e69800640 1 -- 172.21.15.97:0/1004542475 learned_addr learned my addr 172.21.15.97:0/1004542475 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.893+0000 7f9e68e00640 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f909790 0x562d0f9132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.893+0000 7f9e6a200640 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f90e900 0x562d0f91a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.893+0000 7f9e69800640 1 -- 172.21.15.97:0/1004542475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f90e900 msgr2=0x562d0f91a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.893+0000 7f9e69800640 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f90e900 0x562d0f91a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.893+0000 7f9e69800640 1 -- 172.21.15.97:0/1004542475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f909790 msgr2=0x562d0f9132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.893+0000 7f9e69800640 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f909790 0x562d0f9132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.894+0000 7f9e69800640 1 -- 172.21.15.97:0/1004542475 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d0f64e620 con 0x562d0f908da0 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.894+0000 7f9e68e00640 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f909790 0x562d0f9132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:04:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.894+0000 7f9e6a200640 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f90e900 0x562d0f91a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:04:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.894+0000 7f9e69800640 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f908da0 0x562d0f912d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e5802c7c0 tx=0x7f9e58034bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c906b7b28c233b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.894+0000 7f9e4b400640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:04:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.895+0000 7f9e4b400640 1 -- 172.21.15.97:0/1004542475 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58042050 con 0x562d0f908da0 2024-03-26T19:04:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.895+0000 7f9e4b400640 1 -- 172.21.15.97:0/1004542475 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9e58047450 con 0x562d0f908da0 2024-03-26T19:04:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.895+0000 7f9e6ef90740 1 -- 172.21.15.97:0/1004542475 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e5802cc10 con 0x562d0f908da0 2024-03-26T19:04:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.895+0000 7f9e4b400640 1 -- 172.21.15.97:0/1004542475 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e58047650 con 0x562d0f908da0 2024-03-26T19:04:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.895+0000 7f9e6ef90740 10 client.25706 fetch_fsmap learned FSMap version 27 2024-03-26T19:04:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.895+0000 7f9e6ef90740 1 -- 172.21.15.97:0/1004542475 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d0f91d660 con 0x562d0f908da0 2024-03-26T19:04:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.896+0000 7f9e4b400640 1 -- 172.21.15.97:0/1004542475 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5802cc10 con 0x562d0f908da0 2024-03-26T19:04:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.896+0000 7f9e4b400640 1 -- 172.21.15.97:0/1004542475 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9e5803d790 con 0x562d0f908da0 2024-03-26T19:04:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.896+0000 7f9e6ef90740 10 client.25706 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:04:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.896+0000 7f9e6ef90740 10 client.25706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:04:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.896+0000 7f9e6ef90740 20 client.25706 populate_metadata read hostname 'smithi097' 2024-03-26T19:04:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.896+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d0f91e9b0 0x562d0f920da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.896+0000 7f9e6ef90740 4 client.25706 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:04:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.896+0000 7f9e6ef90740 1 -- 172.21.15.97:0/1004542475 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d0f87b3c0 con 0x562d0f91e9b0 2024-03-26T19:04:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.897+0000 7f9e68e00640 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d0f91e9b0 0x562d0f920da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.911+0000 7f9e68e00640 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d0f91e9b0 0x562d0f920da0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91d6e17f1d838cd9 server_cookie=9f41e7fac2c4ee75 in_seq=0 out_seq=0 2024-03-26T19:04:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.911+0000 7f9e4b400640 10 client.25706 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.912+0000 7f9e4b400640 1 -- 172.21.15.97:0/1004542475 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562d0f87b3c0 con 0x562d0f91e9b0 2024-03-26T19:04:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.912+0000 7f9e4b400640 10 client.25706 handle_command_reply: tid=0 2024-03-26T19:04:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.912+0000 7f9e6ef90740 1 client.25706 shutdown 2024-03-26T19:04:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e6ef90740 1 -- 172.21.15.97:0/1004542475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f908da0 msgr2=0x562d0f912d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f908da0 0x562d0f912d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e5802c7c0 tx=0x7f9e58034bb0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e6ef90740 1 -- 172.21.15.97:0/1004542475 shutdown_connections 2024-03-26T19:04:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d0f91e9b0 0x562d0f920da0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f90e900 0x562d0f91a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f909790 0x562d0f9132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/1004542475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f908da0 0x562d0f912d70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e6ef90740 1 -- 172.21.15.97:0/1004542475 >> 172.21.15.97:0/1004542475 conn(0x562d0f904310 msgr2=0x562d0f903540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e4b400640 0 client.25706 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e6ef90740 1 -- 172.21.15.97:0/1004542475 shutdown_connections 2024-03-26T19:04:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.913+0000 7f9e6ef90740 1 -- 172.21.15.97:0/1004542475 wait complete. 2024-03-26T19:04:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.914+0000 7f9e6ef90740 20 client.25706 trim_cache size 0 max 16384 2024-03-26T19:04:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.960+0000 7f9e6ef90740 1 Processor -- start 2024-03-26T19:04:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.960+0000 7f9e6ef90740 1 -- start start 2024-03-26T19:04:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.960+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f908da0 0x562d0f908040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.960+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f91a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.960+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f90e900 0x562d0f912390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.960+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641af2a0 con 0x562d0f908da0 2024-03-26T19:04:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.960+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641b0de0 con 0x562d0f90e900 2024-03-26T19:04:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.960+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641af970 con 0x562d0f909790 2024-03-26T19:04:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.961+0000 7f9e68e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f91a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.962+0000 7f9e68e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f91a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:48928/0 (socket says 172.21.15.97:48928) 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.962+0000 7f9e68e00640 1 -- 172.21.15.97:0/4230028398 learned_addr learned my addr 172.21.15.97:0/4230028398 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.962+0000 7f9e68e00640 1 -- 172.21.15.97:0/4230028398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f90e900 msgr2=0x562d0f912390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.962+0000 7f9e6a200640 1 --2- 172.21.15.97:0/4230028398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f90e900 0x562d0f912390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.962+0000 7f9e68e00640 1 --2- 172.21.15.97:0/4230028398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f90e900 0x562d0f912390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.962+0000 7f9e68e00640 1 -- 172.21.15.97:0/4230028398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f908da0 msgr2=0x562d0f908040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.962+0000 7f9e68e00640 1 --2- 172.21.15.97:0/4230028398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f908da0 0x562d0f908040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.962+0000 7f9e68e00640 1 -- 172.21.15.97:0/4230028398 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d0f902ab0 con 0x562d0f909790 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.963+0000 7f9e68e00640 1 --2- 172.21.15.97:0/4230028398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f91a280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e50005d00 tx=0x7f9e50016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2928418f7f6b4c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.963+0000 7f9e56000640 1 -- 172.21.15.97:0/4230028398 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5002f070 con 0x562d0f909790 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.963+0000 7f9e56000640 1 -- 172.21.15.97:0/4230028398 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e50010e60 con 0x562d0f909790 2024-03-26T19:04:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.963+0000 7f9e56000640 1 -- 172.21.15.97:0/4230028398 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5001cb80 con 0x562d0f909790 2024-03-26T19:04:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.964+0000 7f9e6ef90740 1 -- 172.21.15.97:0/4230028398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 msgr2=0x562d0f91a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.964+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/4230028398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f91a280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e50005d00 tx=0x7f9e50016000 comp rx=0 tx=0).stop 2024-03-26T19:04:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.964+0000 7f9e6ef90740 1 -- 172.21.15.97:0/4230028398 shutdown_connections 2024-03-26T19:04:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.964+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/4230028398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f90e900 0x562d0f912390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.964+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/4230028398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f91a280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.964+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/4230028398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f908da0 0x562d0f908040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.964+0000 7f9e6ef90740 1 -- 172.21.15.97:0/4230028398 >> 172.21.15.97:0/4230028398 conn(0x562d0f904310 msgr2=0x562d0f90aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.965+0000 7f9e6ef90740 1 -- 172.21.15.97:0/4230028398 shutdown_connections 2024-03-26T19:04:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.965+0000 7f9e6ef90740 1 -- 172.21.15.97:0/4230028398 wait complete. 2024-03-26T19:04:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.965+0000 7f9e6ef90740 1 -- start start 2024-03-26T19:04:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.965+0000 7f9e6ef90740 1 Processor -- start 2024-03-26T19:04:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.966+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f908da0 0x562d0f917370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.966+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f9178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.966+0000 7f9e6ef90740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f90e900 0x562d0f9188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.966+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641b0710 con 0x562d0f908da0 2024-03-26T19:04:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.966+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641b0040 con 0x562d0f90e900 2024-03-26T19:04:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.966+0000 7f9e6ef90740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641ad280 con 0x562d0f909790 2024-03-26T19:04:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.966+0000 7f9e6ef90740 10 client.0 _ll_register_callbacks cb 0x7f9e58034840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:04:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.966+0000 7f9e68e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f9178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.966+0000 7f9e68e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f9178b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:48934/0 (socket says 172.21.15.97:48934) 2024-03-26T19:04:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.966+0000 7f9e68e00640 1 -- 172.21.15.97:0/3264363424 learned_addr learned my addr 172.21.15.97:0/3264363424 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.967+0000 7f9e69800640 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f908da0 0x562d0f917370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.967+0000 7f9e6a200640 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f90e900 0x562d0f9188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.968+0000 7f9e68e00640 1 -- 172.21.15.97:0/3264363424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f90e900 msgr2=0x562d0f9188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.968+0000 7f9e68e00640 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f90e900 0x562d0f9188c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.968+0000 7f9e68e00640 1 -- 172.21.15.97:0/3264363424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f908da0 msgr2=0x562d0f917370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.968+0000 7f9e68e00640 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f908da0 0x562d0f917370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.968+0000 7f9e68e00640 1 -- 172.21.15.97:0/3264363424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d0f8ff730 con 0x562d0f909790 2024-03-26T19:04:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.968+0000 7f9e69800640 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f908da0 0x562d0f917370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:04:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.968+0000 7f9e68e00640 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f9178b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009e80 tx=0x7f9e5001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c1c8557787b12c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.968+0000 7f9e4b400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:04:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.969+0000 7f9e4b400640 1 -- 172.21.15.97:0/3264363424 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5002f040 con 0x562d0f909790 2024-03-26T19:04:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.969+0000 7f9e4b400640 1 -- 172.21.15.97:0/3264363424 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9e5001cac0 con 0x562d0f909790 2024-03-26T19:04:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.969+0000 7f9e6ef90740 1 -- 172.21.15.97:0/3264363424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e50010e60 con 0x562d0f909790 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.969+0000 7f9e4b400640 1 -- 172.21.15.97:0/3264363424 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e5001ccc0 con 0x562d0f909790 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.969+0000 7f9e6ef90740 10 client.25666 fetch_fsmap learned FSMap version 27 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.969+0000 7f9e6ef90740 1 -- 172.21.15.97:0/3264363424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d0f91a020 con 0x562d0f909790 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.970+0000 7f9e4b400640 1 -- 172.21.15.97:0/3264363424 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e50010e60 con 0x562d0f909790 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.970+0000 7f9e4b400640 1 -- 172.21.15.97:0/3264363424 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9e5003b030 con 0x562d0f909790 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.970+0000 7f9e6ef90740 10 client.25666 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.970+0000 7f9e6ef90740 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.970+0000 7f9e6ef90740 20 client.25666 populate_metadata read hostname 'smithi097' 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.970+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d0f995df0 0x562d0f998210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.970+0000 7f9e6ef90740 4 client.25666 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:04:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.970+0000 7f9e6ef90740 1 -- 172.21.15.97:0/3264363424 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d0f875710 con 0x562d0f995df0 2024-03-26T19:04:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.971+0000 7f9e69800640 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d0f995df0 0x562d0f998210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.971+0000 7f9e69800640 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d0f995df0 0x562d0f998210 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f3a6f596de16b3e server_cookie=6001a024956d262 in_seq=0 out_seq=0 2024-03-26T19:04:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.972+0000 7f9e4b400640 10 client.25666 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.984+0000 7f9e4b400640 1 -- 172.21.15.97:0/3264363424 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d0f875710 con 0x562d0f995df0 2024-03-26T19:04:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.984+0000 7f9e4b400640 10 client.25666 handle_command_reply: tid=0 2024-03-26T19:04:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.984+0000 7f9e6ef90740 1 client.25666 shutdown 2024-03-26T19:04:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.984+0000 7f9e6ef90740 1 -- 172.21.15.97:0/3264363424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 msgr2=0x562d0f9178b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.984+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f9178b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009e80 tx=0x7f9e5001c470 comp rx=0 tx=0).stop 2024-03-26T19:04:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.985+0000 7f9e6ef90740 1 -- 172.21.15.97:0/3264363424 shutdown_connections 2024-03-26T19:04:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.985+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562d0f995df0 0x562d0f998210 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.985+0000 7f9e4b400640 0 client.25666 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.985+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d0f90e900 0x562d0f9188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.985+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d0f909790 0x562d0f9178b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.985+0000 7f9e6ef90740 1 --2- 172.21.15.97:0/3264363424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562d0f908da0 0x562d0f917370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.985+0000 7f9e6ef90740 1 -- 172.21.15.97:0/3264363424 >> 172.21.15.97:0/3264363424 conn(0x562d0f904310 msgr2=0x562d0f90a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.985+0000 7f9e6ef90740 1 -- 172.21.15.97:0/3264363424 shutdown_connections 2024-03-26T19:04:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.985+0000 7f9e6ef90740 1 -- 172.21.15.97:0/3264363424 wait complete. 2024-03-26T19:04:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.985+0000 7f9e6ef90740 20 client.25666 trim_cache size 0 max 16384 2024-03-26T19:04:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:04:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:04:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:04:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:04:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 -- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e4c0a3040 msgr2=0x7f9e4c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e4c0a3040 0x7f9e4c0a5500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9e50004650 tx=0x7f9e500095d0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 -- 172.21.15.97:0/30861224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e641056d0 msgr2=0x7f9e641a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e641056d0 0x7f9e641a5720 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00cad0 tx=0x7f9e5c00cf90 comp rx=0 tx=0).stop 2024-03-26T19:04:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 -- 172.21.15.97:0/30861224 shutdown_connections 2024-03-26T19:04:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9e4c0a3040 0x7f9e4c0a5500 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e641056d0 0x7f9e641a5720 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e64104d10 0x7f9e6419e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 --2- 172.21.15.97:0/30861224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e64103b20 0x7f9e6419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 -- 172.21.15.97:0/30861224 >> 172.21.15.97:0/30861224 conn(0x7f9e640ff0b0 msgr2=0x7f9e640ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.988+0000 7f9e6c200640 1 -- 172.21.15.97:0/30861224 shutdown_connections 2024-03-26T19:04:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:42.989+0000 7f9e6c200640 1 -- 172.21.15.97:0/30861224 wait complete. 2024-03-26T19:04:44.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:44 smithi097 ceph-mon[31955]: pgmap v564: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 MiB/s wr, 750 op/s 2024-03-26T19:04:44.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:44 smithi144 ceph-mon[36053]: pgmap v564: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 MiB/s wr, 750 op/s 2024-03-26T19:04:44.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:44 smithi105 ceph-mon[36270]: pgmap v564: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 MiB/s wr, 750 op/s 2024-03-26T19:04:45.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 320.0 s, 1775.0 tps, lat 56.842 ms stddev 65.212 2024-03-26T19:04:46.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:46 smithi097 ceph-mon[31955]: pgmap v565: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 802 op/s 2024-03-26T19:04:46.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:46 smithi144 ceph-mon[36053]: pgmap v565: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 802 op/s 2024-03-26T19:04:46.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:46 smithi105 ceph-mon[36270]: pgmap v565: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 802 op/s 2024-03-26T19:04:48.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:48 smithi097 ceph-mon[31955]: pgmap v566: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-03-26T19:04:48.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:48 smithi144 ceph-mon[36053]: pgmap v566: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-03-26T19:04:48.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:48 smithi105 ceph-mon[36270]: pgmap v566: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-03-26T19:04:50.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 325.0 s, 2853.0 tps, lat 34.968 ms stddev 39.163 2024-03-26T19:04:50.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:50 smithi097 ceph-mon[31955]: pgmap v567: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 767 op/s 2024-03-26T19:04:50.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:50 smithi144 ceph-mon[36053]: pgmap v567: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 767 op/s 2024-03-26T19:04:50.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:50 smithi105 ceph-mon[36270]: pgmap v567: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 767 op/s 2024-03-26T19:04:52.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:52 smithi097 ceph-mon[31955]: pgmap v568: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.8 MiB/s wr, 678 op/s 2024-03-26T19:04:52.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:52 smithi144 ceph-mon[36053]: pgmap v568: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.8 MiB/s wr, 678 op/s 2024-03-26T19:04:52.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:52 smithi105 ceph-mon[36270]: pgmap v568: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.8 MiB/s wr, 678 op/s 2024-03-26T19:04:54.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:54 smithi097 ceph-mon[31955]: pgmap v569: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.0 MiB/s wr, 665 op/s 2024-03-26T19:04:54.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:54 smithi144 ceph-mon[36053]: pgmap v569: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.0 MiB/s wr, 665 op/s 2024-03-26T19:04:54.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:54 smithi105 ceph-mon[36270]: pgmap v569: 97 pgs: 97 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.0 MiB/s wr, 665 op/s 2024-03-26T19:04:55.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 330.0 s, 2711.9 tps, lat 36.699 ms stddev 44.173 2024-03-26T19:04:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.189+0000 7f7d14600640 1 -- 172.21.15.97:0/2870631533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d0c1088c0 msgr2=0x7f7d0c108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.189+0000 7f7d14600640 1 --2- 172.21.15.97:0/2870631533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d0c1088c0 0x7f7d0c108ce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009be0 tx=0x7f7d0402bef0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.190+0000 7f7d14600640 1 -- 172.21.15.97:0/2870631533 shutdown_connections 2024-03-26T19:04:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.190+0000 7f7d14600640 1 --2- 172.21.15.97:0/2870631533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d0c110140 0x7f7d0c112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.190+0000 7f7d14600640 1 --2- 172.21.15.97:0/2870631533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c109220 0x7f7d0c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.190+0000 7f7d14600640 1 --2- 172.21.15.97:0/2870631533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d0c1088c0 0x7f7d0c108ce0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.190+0000 7f7d14600640 1 -- 172.21.15.97:0/2870631533 >> 172.21.15.97:0/2870631533 conn(0x7f7d0c06e900 msgr2=0x7f7d0c070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.190+0000 7f7d14600640 1 -- 172.21.15.97:0/2870631533 shutdown_connections 2024-03-26T19:04:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.190+0000 7f7d14600640 1 -- 172.21.15.97:0/2870631533 wait complete. 2024-03-26T19:04:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.191+0000 7f7d14600640 1 Processor -- start 2024-03-26T19:04:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.191+0000 7f7d14600640 1 -- start start 2024-03-26T19:04:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d14600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d0c1088c0 0x7f7d0c1a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d14600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c109220 0x7f7d0c1a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d14600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d0c110140 0x7f7d0c1a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d14600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c10acd0 con 0x7f7d0c110140 2024-03-26T19:04:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d14600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c10ab50 con 0x7f7d0c1088c0 2024-03-26T19:04:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d14600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c10ae50 con 0x7f7d0c109220 2024-03-26T19:04:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d11c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d0c1088c0 0x7f7d0c1a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d11200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c109220 0x7f7d0c1a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d11c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d0c1088c0 0x7f7d0c1a2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53834/0 (socket says 172.21.15.97:53834) 2024-03-26T19:04:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d11c00640 1 -- 172.21.15.97:0/4166965431 learned_addr learned my addr 172.21.15.97:0/4166965431 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.192+0000 7f7d12600640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d0c110140 0x7f7d0c1a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.193+0000 7f7d11c00640 1 -- 172.21.15.97:0/4166965431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c109220 msgr2=0x7f7d0c1a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.193+0000 7f7d11c00640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c109220 0x7f7d0c1a2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.193+0000 7f7d11c00640 1 -- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d0c110140 msgr2=0x7f7d0c1a9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.193+0000 7f7d11c00640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d0c110140 0x7f7d0c1a9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.193+0000 7f7d11c00640 1 -- 172.21.15.97:0/4166965431 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c1aa150 con 0x7f7d0c1088c0 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.193+0000 7f7d12600640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d0c110140 0x7f7d0c1a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.193+0000 7f7d11c00640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d0c1088c0 0x7f7d0c1a2530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7d04000f80 tx=0x7f7d040337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.194+0000 7f7d0a000640 1 -- 172.21.15.97:0/4166965431 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04043020 con 0x7f7d0c1088c0 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.194+0000 7f7d14600640 1 -- 172.21.15.97:0/4166965431 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0c1aa3e0 con 0x7f7d0c1088c0 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.194+0000 7f7d0a000640 1 -- 172.21.15.97:0/4166965431 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0402ca60 con 0x7f7d0c1088c0 2024-03-26T19:04:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.194+0000 7f7d0a000640 1 -- 172.21.15.97:0/4166965431 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04034690 con 0x7f7d0c1088c0 2024-03-26T19:04:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.194+0000 7f7d14600640 1 -- 172.21.15.97:0/4166965431 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0c1aa980 con 0x7f7d0c1088c0 2024-03-26T19:04:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.197+0000 7f7d171f3740 1 Processor -- start 2024-03-26T19:04:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.197+0000 7f7d0a000640 1 -- 172.21.15.97:0/4166965431 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f7d04046020 con 0x7f7d0c1088c0 2024-03-26T19:04:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.198+0000 7f7d171f3740 1 -- start start 2024-03-26T19:04:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.198+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bcda0 0x562dc58bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.198+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bd790 0x562dc58c2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.198+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 0x562dc58c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.198+0000 7f7d171f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562dc5821e80 con 0x562dc58bcda0 2024-03-26T19:04:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.198+0000 7f7d171f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562dc57ca660 con 0x562dc58bd790 2024-03-26T19:04:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.198+0000 7f7d171f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562dc581de30 con 0x562dc58c2900 2024-03-26T19:04:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.199+0000 7f7d11200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 0x562dc58c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.199+0000 7f7d11200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 0x562dc58c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:41882/0 (socket says 172.21.15.97:41882) 2024-03-26T19:04:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.199+0000 7f7d11200640 1 -- 172.21.15.97:0/2207350712 learned_addr learned my addr 172.21.15.97:0/2207350712 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.199+0000 7f7d12600640 1 --2- 172.21.15.97:0/2207350712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bcda0 0x562dc58bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.200+0000 7f7d11c00640 1 --2- 172.21.15.97:0/2207350712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bd790 0x562dc58c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.200+0000 7f7d0a000640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7cf40a3090 0x7f7cf40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.200+0000 7f7d11200640 1 -- 172.21.15.97:0/2207350712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bd790 msgr2=0x562dc58c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.200+0000 7f7d11200640 1 --2- 172.21.15.97:0/2207350712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bd790 0x562dc58c2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.200+0000 7f7d11200640 1 -- 172.21.15.97:0/2207350712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bcda0 msgr2=0x562dc58bd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.200+0000 7f7d11200640 1 --2- 172.21.15.97:0/2207350712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bcda0 0x562dc58bd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.200+0000 7f7d11200640 1 -- 172.21.15.97:0/2207350712 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc5536210 con 0x562dc58c2900 2024-03-26T19:04:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.200+0000 7f7d0a000640 1 -- 172.21.15.97:0/4166965431 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7d040f1ab0 con 0x7f7d0c1088c0 2024-03-26T19:04:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.201+0000 7f7d11200640 1 --2- 172.21.15.97:0/2207350712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 0x562dc58c4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009c30 tx=0x7f7cfc0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f03b72c05ceb75b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.202+0000 7f7cfaa00640 1 -- 172.21.15.97:0/2207350712 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc012450 con 0x562dc58c2900 2024-03-26T19:04:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.202+0000 7f7cfaa00640 1 -- 172.21.15.97:0/2207350712 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cfc012ad0 con 0x562dc58c2900 2024-03-26T19:04:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.202+0000 7f7cfaa00640 1 -- 172.21.15.97:0/2207350712 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00fbf0 con 0x562dc58c2900 2024-03-26T19:04:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.202+0000 7f7d12600640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7cf40a3090 0x7f7cf40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.204+0000 7f7d171f3740 1 -- 172.21.15.97:0/2207350712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 msgr2=0x562dc58c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.204+0000 7f7d171f3740 1 --2- 172.21.15.97:0/2207350712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 0x562dc58c4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009c30 tx=0x7f7cfc0117b0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.204+0000 7f7d12600640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7cf40a3090 0x7f7cf40a5550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x562dc58bdff0 tx=0x7f7d000046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.204+0000 7f7d171f3740 1 -- 172.21.15.97:0/2207350712 shutdown_connections 2024-03-26T19:04:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.204+0000 7f7d171f3740 1 --2- 172.21.15.97:0/2207350712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 0x562dc58c4d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.204+0000 7f7d171f3740 1 --2- 172.21.15.97:0/2207350712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bd790 0x562dc58c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.204+0000 7f7d171f3740 1 --2- 172.21.15.97:0/2207350712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bcda0 0x562dc58bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.204+0000 7f7d171f3740 1 -- 172.21.15.97:0/2207350712 >> 172.21.15.97:0/2207350712 conn(0x562dc58b8310 msgr2=0x562dc58b8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.205+0000 7f7d171f3740 1 -- 172.21.15.97:0/2207350712 shutdown_connections 2024-03-26T19:04:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.205+0000 7f7d171f3740 1 -- 172.21.15.97:0/2207350712 wait complete. 2024-03-26T19:04:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.206+0000 7f7d171f3740 1 -- start start 2024-03-26T19:04:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.206+0000 7f7d171f3740 1 Processor -- start 2024-03-26T19:04:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.207+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bcda0 0x562dc58cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.207+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 0x562dc58cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.207+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58c2900 0x562dc58d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.207+0000 7f7d171f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562dc58c58b0 con 0x562dc58bd790 2024-03-26T19:04:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.207+0000 7f7d171f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562dc57ddac0 con 0x562dc58c2900 2024-03-26T19:04:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.207+0000 7f7d171f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562dc58c5a30 con 0x562dc58bcda0 2024-03-26T19:04:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.207+0000 7f7d171f3740 10 client.0 _ll_register_callbacks cb 0x562dc57dd490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:04:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.207+0000 7f7d12600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58c2900 0x562dc58d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.207+0000 7f7d12600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58c2900 0x562dc58d6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53856/0 (socket says 172.21.15.97:53856) 2024-03-26T19:04:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.207+0000 7f7d12600640 1 -- 172.21.15.97:0/1297329597 learned_addr learned my addr 172.21.15.97:0/1297329597 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.208+0000 7f7d11200640 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 0x562dc58cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.208+0000 7f7d11c00640 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bcda0 0x562dc58cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.209+0000 7f7d11200640 1 -- 172.21.15.97:0/1297329597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bcda0 msgr2=0x562dc58cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.209+0000 7f7d11200640 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bcda0 0x562dc58cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.209+0000 7f7d11200640 1 -- 172.21.15.97:0/1297329597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58c2900 msgr2=0x562dc58d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.209+0000 7f7d11200640 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58c2900 0x562dc58d6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.209+0000 7f7d11200640 1 -- 172.21.15.97:0/1297329597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc5602620 con 0x562dc58bd790 2024-03-26T19:04:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.209+0000 7f7d12600640 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58c2900 0x562dc58d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:04:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.210+0000 7f7d11200640 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 0x562dc58cfb20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc01c040 tx=0x7f7cfc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6c252ec44b30f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.210+0000 7f7cef400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T19:04:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.211+0000 7f7cef400640 1 -- 172.21.15.97:0/1297329597 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc028020 con 0x562dc58bd790 2024-03-26T19:04:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.211+0000 7f7cef400640 1 -- 172.21.15.97:0/1297329597 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7cfc012450 con 0x562dc58bd790 2024-03-26T19:04:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.211+0000 7f7d171f3740 1 -- 172.21.15.97:0/1297329597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cfc012ad0 con 0x562dc58bd790 2024-03-26T19:04:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.211+0000 7f7cef400640 1 -- 172.21.15.97:0/1297329597 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7cfc012650 con 0x562dc58bd790 2024-03-26T19:04:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.212+0000 7f7d171f3740 10 client.15441 fetch_fsmap learned FSMap version 27 2024-03-26T19:04:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.212+0000 7f7d171f3740 1 -- 172.21.15.97:0/1297329597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562dc58d9ed0 con 0x562dc58bd790 2024-03-26T19:04:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.216+0000 7f7cef400640 1 -- 172.21.15.97:0/1297329597 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cfc00fe10 con 0x562dc58bd790 2024-03-26T19:04:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.219+0000 7f7cef400640 1 -- 172.21.15.97:0/1297329597 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7cfc0054b0 con 0x562dc58bd790 2024-03-26T19:04:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.219+0000 7f7d171f3740 10 client.15441 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:04:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.219+0000 7f7d171f3740 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:04:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.220+0000 7f7d171f3740 20 client.15441 populate_metadata read hostname 'smithi097' 2024-03-26T19:04:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.220+0000 7f7d171f3740 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562dc58db2e0 0x562dc58dd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.220+0000 7f7d171f3740 4 client.15441 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:04:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.220+0000 7f7d171f3740 1 -- 172.21.15.97:0/1297329597 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7cfc012ad0 con 0x562dc58db2e0 2024-03-26T19:04:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.220+0000 7f7d11c00640 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562dc58db2e0 0x562dc58dd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.222+0000 7f7cef400640 10 client.15441 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.222+0000 7f7d11c00640 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562dc58db2e0 0x562dc58dd6d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1610182a9d39e7b server_cookie=73e1e9dc99fb2e59 in_seq=0 out_seq=0 2024-03-26T19:04:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.223+0000 7f7cef400640 1 -- 172.21.15.97:0/1297329597 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f7cfc012ad0 con 0x562dc58db2e0 2024-03-26T19:04:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.223+0000 7f7cef400640 10 client.15441 handle_command_reply: tid=0 2024-03-26T19:04:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.223+0000 7f7d171f3740 1 client.15441 shutdown 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.225+0000 7f7d171f3740 1 -- 172.21.15.97:0/1297329597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 msgr2=0x562dc58cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.225+0000 7f7d171f3740 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 0x562dc58cfb20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc01c040 tx=0x7f7cfc004270 comp rx=0 tx=0).stop 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.226+0000 7f7d171f3740 1 -- 172.21.15.97:0/1297329597 shutdown_connections 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.226+0000 7f7d171f3740 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562dc58db2e0 0x562dc58dd6d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.226+0000 7f7d171f3740 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58c2900 0x562dc58d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.226+0000 7f7cef400640 0 client.15441 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.226+0000 7f7d171f3740 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 0x562dc58cfb20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.226+0000 7f7d171f3740 1 --2- 172.21.15.97:0/1297329597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bcda0 0x562dc58cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.226+0000 7f7d171f3740 1 -- 172.21.15.97:0/1297329597 >> 172.21.15.97:0/1297329597 conn(0x562dc58b8310 msgr2=0x562dc58b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.227+0000 7f7d171f3740 1 -- 172.21.15.97:0/1297329597 shutdown_connections 2024-03-26T19:04:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.227+0000 7f7d171f3740 1 -- 172.21.15.97:0/1297329597 wait complete. 2024-03-26T19:04:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.229+0000 7f7d171f3740 20 client.15441 trim_cache size 0 max 16384 2024-03-26T19:04:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.264+0000 7f7d171f3740 1 Processor -- start 2024-03-26T19:04:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.265+0000 7f7d171f3740 1 -- start start 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.265+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 0x562dc58be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.265+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 0x562dc58d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.265+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 0x562dc58e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.265+0000 7f7d171f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1af190 con 0x562dc58bd790 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.265+0000 7f7d171f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1b0cd0 con 0x562dc58bcda0 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.265+0000 7f7d171f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1af860 con 0x562dc58c2900 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.265+0000 7f7d11c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 0x562dc58be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.265+0000 7f7d11c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 0x562dc58be940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53864/0 (socket says 172.21.15.97:53864) 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.265+0000 7f7d11c00640 1 -- 172.21.15.97:0/3341190574 learned_addr learned my addr 172.21.15.97:0/3341190574 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.266+0000 7f7d11200640 1 --2- 172.21.15.97:0/3341190574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 0x562dc58d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.266+0000 7f7d12600640 1 --2- 172.21.15.97:0/3341190574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 0x562dc58e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.266+0000 7f7d11c00640 1 -- 172.21.15.97:0/3341190574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 msgr2=0x562dc58e1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.266+0000 7f7d11c00640 1 --2- 172.21.15.97:0/3341190574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 0x562dc58e1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.266+0000 7f7d11c00640 1 -- 172.21.15.97:0/3341190574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 msgr2=0x562dc58d6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.266+0000 7f7d11c00640 1 --2- 172.21.15.97:0/3341190574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 0x562dc58d6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.266+0000 7f7d11c00640 1 -- 172.21.15.97:0/3341190574 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc58ce690 con 0x562dc58bcda0 2024-03-26T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.266+0000 7f7d11c00640 1 --2- 172.21.15.97:0/3341190574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 0x562dc58be940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7d040be140 tx=0x7f7d040de040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81c03990df31f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.267+0000 7f7cfa000640 1 -- 172.21.15.97:0/3341190574 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d040be8b0 con 0x562dc58bcda0 2024-03-26T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.267+0000 7f7cfa000640 1 -- 172.21.15.97:0/3341190574 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cfc012ad0 con 0x562dc58bcda0 2024-03-26T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.267+0000 7f7cfa000640 1 -- 172.21.15.97:0/3341190574 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d040ef690 con 0x562dc58bcda0 2024-03-26T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.267+0000 7f7d171f3740 1 -- 172.21.15.97:0/3341190574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 msgr2=0x562dc58be940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.268+0000 7f7d171f3740 1 --2- 172.21.15.97:0/3341190574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 0x562dc58be940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7d040be140 tx=0x7f7d040de040 comp rx=0 tx=0).stop 2024-03-26T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.268+0000 7f7d171f3740 1 -- 172.21.15.97:0/3341190574 shutdown_connections 2024-03-26T19:04:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.268+0000 7f7d171f3740 1 --2- 172.21.15.97:0/3341190574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58c2900 0x562dc58e1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.268+0000 7f7d171f3740 1 --2- 172.21.15.97:0/3341190574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58bd790 0x562dc58d6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.268+0000 7f7d171f3740 1 --2- 172.21.15.97:0/3341190574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 0x562dc58be940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.268+0000 7f7d171f3740 1 -- 172.21.15.97:0/3341190574 >> 172.21.15.97:0/3341190574 conn(0x562dc58b8310 msgr2=0x562dc58be230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.268+0000 7f7d171f3740 1 -- 172.21.15.97:0/3341190574 shutdown_connections 2024-03-26T19:04:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.268+0000 7f7d171f3740 1 -- 172.21.15.97:0/3341190574 wait complete. 2024-03-26T19:04:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.268+0000 7f7d171f3740 1 -- start start 2024-03-26T19:04:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.268+0000 7f7d171f3740 1 Processor -- start 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 0x562dc58d4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bd790 0x562dc58d4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d171f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58c2900 0x562dc58d5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d171f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1b0600 con 0x562dc58c2900 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d171f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1aff30 con 0x562dc58bcda0 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d171f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1ad1b0 con 0x562dc58bd790 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d171f3740 10 client.0 _ll_register_callbacks cb 0x7f7cfc012eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d11200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bd790 0x562dc58d4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d11200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bd790 0x562dc58d4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:41910/0 (socket says 172.21.15.97:41910) 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d11200640 1 -- 172.21.15.97:0/4065274072 learned_addr learned my addr 172.21.15.97:0/4065274072 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:04:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.269+0000 7f7d11c00640 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 0x562dc58d4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.270+0000 7f7d12600640 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58c2900 0x562dc58d5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.271+0000 7f7d11200640 1 -- 172.21.15.97:0/4065274072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 msgr2=0x562dc58d4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.271+0000 7f7d11200640 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 0x562dc58d4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.271+0000 7f7d11200640 1 -- 172.21.15.97:0/4065274072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58c2900 msgr2=0x562dc58d5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.271+0000 7f7d11200640 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58c2900 0x562dc58d5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.271+0000 7f7d11200640 1 -- 172.21.15.97:0/4065274072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc58d6360 con 0x562dc58bd790 2024-03-26T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.271+0000 7f7d11200640 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bd790 0x562dc58d4db0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00f4c0 tx=0x7f7cfc012590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535355bcbaf7dbb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.271+0000 7f7cef400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.272+0000 7f7cef400640 1 -- 172.21.15.97:0/4065274072 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc009730 con 0x562dc58bd790 2024-03-26T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.272+0000 7f7cef400640 1 -- 172.21.15.97:0/4065274072 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7cfc0035f0 con 0x562dc58bd790 2024-03-26T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.273+0000 7f7d171f3740 1 -- 172.21.15.97:0/4065274072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cfc012ad0 con 0x562dc58bd790 2024-03-26T19:04:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.273+0000 7f7cef400640 1 -- 172.21.15.97:0/4065274072 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7cfc0037f0 con 0x562dc58bd790 2024-03-26T19:04:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.273+0000 7f7d171f3740 10 client.25684 fetch_fsmap learned FSMap version 27 2024-03-26T19:04:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.273+0000 7f7d171f3740 1 -- 172.21.15.97:0/4065274072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562dc5941960 con 0x562dc58bd790 2024-03-26T19:04:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.273+0000 7f7cef400640 1 -- 172.21.15.97:0/4065274072 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cfc012ad0 con 0x562dc58bd790 2024-03-26T19:04:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.274+0000 7f7cef400640 1 -- 172.21.15.97:0/4065274072 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7cfc00fc20 con 0x562dc58bd790 2024-03-26T19:04:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.274+0000 7f7d171f3740 10 client.25684 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:04:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.274+0000 7f7d171f3740 10 client.25684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:04:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.274+0000 7f7d171f3740 20 client.25684 populate_metadata read hostname 'smithi097' 2024-03-26T19:04:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.274+0000 7f7d171f3740 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562dc58db2e0 0x562dc5944290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:04:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.274+0000 7f7d171f3740 4 client.25684 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:04:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.274+0000 7f7d171f3740 1 -- 172.21.15.97:0/4065274072 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562dc582f3c0 con 0x562dc58db2e0 2024-03-26T19:04:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.274+0000 7f7d11c00640 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562dc58db2e0 0x562dc5944290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:04:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.275+0000 7f7cef400640 10 client.25684 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.275+0000 7f7d11c00640 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562dc58db2e0 0x562dc5944290 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6284bf73e8591f2f server_cookie=2999cae639490674 in_seq=0 out_seq=0 2024-03-26T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.293+0000 7f7cef400640 1 -- 172.21.15.97:0/4065274072 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562dc582f3c0 con 0x562dc58db2e0 2024-03-26T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.293+0000 7f7cef400640 10 client.25684 handle_command_reply: tid=0 2024-03-26T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.293+0000 7f7d171f3740 1 client.25684 shutdown 2024-03-26T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.296+0000 7f7d171f3740 1 -- 172.21.15.97:0/4065274072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bd790 msgr2=0x562dc58d4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.296+0000 7f7d171f3740 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bd790 0x562dc58d4db0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00f4c0 tx=0x7f7cfc012590 comp rx=0 tx=0).stop 2024-03-26T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.296+0000 7f7d171f3740 1 -- 172.21.15.97:0/4065274072 shutdown_connections 2024-03-26T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.296+0000 7f7d171f3740 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562dc58db2e0 0x562dc5944290 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.296+0000 7f7d171f3740 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562dc58c2900 0x562dc58d5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.296+0000 7f7d171f3740 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dc58bd790 0x562dc58d4db0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.296+0000 7f7d171f3740 1 --2- 172.21.15.97:0/4065274072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562dc58bcda0 0x562dc58d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.296+0000 7f7d171f3740 1 -- 172.21.15.97:0/4065274072 >> 172.21.15.97:0/4065274072 conn(0x562dc58b8310 msgr2=0x562dc58b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.296+0000 7f7cef400640 0 client.25684 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.296+0000 7f7d171f3740 1 -- 172.21.15.97:0/4065274072 shutdown_connections 2024-03-26T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.297+0000 7f7d171f3740 1 -- 172.21.15.97:0/4065274072 wait complete. 2024-03-26T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.297+0000 7f7d171f3740 20 client.25684 trim_cache size 0 max 16384 2024-03-26T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.298+0000 7f7d14600640 1 -- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7cf40a3090 msgr2=0x7f7cf40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.298+0000 7f7d14600640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7cf40a3090 0x7f7cf40a5550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x562dc58bdff0 tx=0x7f7d000046c0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.298+0000 7f7d14600640 1 -- 172.21.15.97:0/4166965431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d0c1088c0 msgr2=0x7f7d0c1a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.298+0000 7f7d14600640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d0c1088c0 0x7f7d0c1a2530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7d04000f80 tx=0x7f7d040337b0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.298+0000 7f7d14600640 1 -- 172.21.15.97:0/4166965431 shutdown_connections 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.298+0000 7f7d14600640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7cf40a3090 0x7f7cf40a5550 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.298+0000 7f7d14600640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d0c110140 0x7f7d0c1a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.298+0000 7f7d14600640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c109220 0x7f7d0c1a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.298+0000 7f7d14600640 1 --2- 172.21.15.97:0/4166965431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d0c1088c0 0x7f7d0c1a2530 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.298+0000 7f7d14600640 1 -- 172.21.15.97:0/4166965431 >> 172.21.15.97:0/4166965431 conn(0x7f7d0c06e900 msgr2=0x7f7d0c070490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.299+0000 7f7d14600640 1 -- 172.21.15.97:0/4166965431 shutdown_connections 2024-03-26T19:04:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:04:56.299+0000 7f7d14600640 1 -- 172.21.15.97:0/4166965431 wait complete. 2024-03-26T19:04:56.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:56 smithi097 ceph-mon[31955]: pgmap v570: 97 pgs: 97 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 736 op/s 2024-03-26T19:04:56.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:56 smithi144 ceph-mon[36053]: pgmap v570: 97 pgs: 97 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 736 op/s 2024-03-26T19:04:56.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:56 smithi105 ceph-mon[36270]: pgmap v570: 97 pgs: 97 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 736 op/s 2024-03-26T19:04:58.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:04:58 smithi097 ceph-mon[31955]: pgmap v571: 97 pgs: 97 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 695 op/s 2024-03-26T19:04:58.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:04:58 smithi144 ceph-mon[36053]: pgmap v571: 97 pgs: 97 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 695 op/s 2024-03-26T19:04:58.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:04:58 smithi105 ceph-mon[36270]: pgmap v571: 97 pgs: 97 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 695 op/s 2024-03-26T19:05:00.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 335.0 s, 1729.6 tps, lat 58.234 ms stddev 70.473 2024-03-26T19:05:00.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:00 smithi097 ceph-mon[31955]: pgmap v572: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 792 op/s 2024-03-26T19:05:00.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:00 smithi144 ceph-mon[36053]: pgmap v572: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 792 op/s 2024-03-26T19:05:00.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:00 smithi105 ceph-mon[36270]: pgmap v572: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 792 op/s 2024-03-26T19:05:02.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:02 smithi097 ceph-mon[31955]: pgmap v573: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 673 op/s 2024-03-26T19:05:02.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:02 smithi144 ceph-mon[36053]: pgmap v573: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 673 op/s 2024-03-26T19:05:02.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:02 smithi105 ceph-mon[36270]: pgmap v573: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 673 op/s 2024-03-26T19:05:04.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:04 smithi097 ceph-mon[31955]: pgmap v574: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 661 op/s 2024-03-26T19:05:04.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:04 smithi144 ceph-mon[36053]: pgmap v574: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 661 op/s 2024-03-26T19:05:04.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:04 smithi105 ceph-mon[36270]: pgmap v574: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 661 op/s 2024-03-26T19:05:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 340.0 s, 2318.8 tps, lat 42.729 ms stddev 47.631 2024-03-26T19:05:05.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:05 smithi144 ceph-mon[36053]: pgmap v575: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 784 op/s 2024-03-26T19:05:05.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:05 smithi105 ceph-mon[36270]: pgmap v575: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 784 op/s 2024-03-26T19:05:05.856 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:05 smithi097 ceph-mon[31955]: pgmap v575: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 784 op/s 2024-03-26T19:05:08.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:08 smithi105 ceph-mon[36270]: pgmap v576: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 655 op/s 2024-03-26T19:05:08.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:08 smithi097 ceph-mon[31955]: pgmap v576: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 655 op/s 2024-03-26T19:05:08.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:08 smithi144 ceph-mon[36053]: pgmap v576: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 655 op/s 2024-03-26T19:05:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.503+0000 7fd39b200640 1 -- 172.21.15.97:0/2560015164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd39c0fff80 msgr2=0x7fd39c1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.503+0000 7fd39b200640 1 --2- 172.21.15.97:0/2560015164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd39c0fff80 0x7fd39c1023b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd38c009900 tx=0x7fd38c02bc50 comp rx=0 tx=0).stop 2024-03-26T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.506+0000 7fd39b200640 1 -- 172.21.15.97:0/2560015164 shutdown_connections 2024-03-26T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.506+0000 7fd39b200640 1 --2- 172.21.15.97:0/2560015164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39c1052a0 0x7fd39c105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.506+0000 7fd39b200640 1 --2- 172.21.15.97:0/2560015164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c1028f0 0x7fd39c104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.506+0000 7fd39b200640 1 --2- 172.21.15.97:0/2560015164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd39c0fff80 0x7fd39c1023b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.506+0000 7fd39b200640 1 -- 172.21.15.97:0/2560015164 >> 172.21.15.97:0/2560015164 conn(0x7fd39c0fbba0 msgr2=0x7fd39c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.506+0000 7fd39b200640 1 -- 172.21.15.97:0/2560015164 shutdown_connections 2024-03-26T19:05:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.507+0000 7fd39b200640 1 -- 172.21.15.97:0/2560015164 wait complete. 2024-03-26T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.510+0000 7fd39b200640 1 Processor -- start 2024-03-26T19:05:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.511+0000 7fd39b200640 1 -- start start 2024-03-26T19:05:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd39b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd39c0fff80 0x7fd39c19e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd39b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39c1028f0 0x7fd39c19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd39b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c1052a0 0x7fd39c1a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd39b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c113a30 con 0x7fd39c1028f0 2024-03-26T19:05:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd39b200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c1138b0 con 0x7fd39c0fff80 2024-03-26T19:05:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd39b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c113bb0 con 0x7fd39c1052a0 2024-03-26T19:05:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd39a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c1052a0 0x7fd39c1a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd399e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd39c0fff80 0x7fd39c19e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd39a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c1052a0 0x7fd39c1a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34790/0 (socket says 172.21.15.97:34790) 2024-03-26T19:05:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd39a800640 1 -- 172.21.15.97:0/1993231319 learned_addr learned my addr 172.21.15.97:0/1993231319 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.512+0000 7fd399400640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39c1028f0 0x7fd39c19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.513+0000 7fd39a800640 1 -- 172.21.15.97:0/1993231319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd39c0fff80 msgr2=0x7fd39c19e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.513+0000 7fd39a800640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd39c0fff80 0x7fd39c19e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.513+0000 7fd39a800640 1 -- 172.21.15.97:0/1993231319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39c1028f0 msgr2=0x7fd39c19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.513+0000 7fd39a800640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39c1028f0 0x7fd39c19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.513+0000 7fd39a800640 1 -- 172.21.15.97:0/1993231319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c1a5c70 con 0x7fd39c1052a0 2024-03-26T19:05:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.513+0000 7fd39a800640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c1052a0 0x7fd39c1a5600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd39400b970 tx=0x7fd39400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.514+0000 7fd392a00640 1 -- 172.21.15.97:0/1993231319 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394019070 con 0x7fd39c1052a0 2024-03-26T19:05:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.514+0000 7fd392a00640 1 -- 172.21.15.97:0/1993231319 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39400cb40 con 0x7fd39c1052a0 2024-03-26T19:05:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.514+0000 7fd392a00640 1 -- 172.21.15.97:0/1993231319 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394007650 con 0x7fd39c1052a0 2024-03-26T19:05:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.514+0000 7fd39b200640 1 -- 172.21.15.97:0/1993231319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39c1a5f60 con 0x7fd39c1052a0 2024-03-26T19:05:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.514+0000 7fd39b200640 1 -- 172.21.15.97:0/1993231319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39c1a63d0 con 0x7fd39c1052a0 2024-03-26T19:05:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.520+0000 7fd3a3da8740 1 Processor -- start 2024-03-26T19:05:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.521+0000 7fd392a00640 1 -- 172.21.15.97:0/1993231319 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fd39401c070 con 0x7fd39c1052a0 2024-03-26T19:05:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.522+0000 7fd3a3da8740 1 -- start start 2024-03-26T19:05:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.522+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x56090500a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.522+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x56090500f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.522+0000 7fd399400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x56090500a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.522+0000 7fd399e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x56090500f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.522+0000 7fd399400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x56090500a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34792/0 (socket says 172.21.15.97:34792) 2024-03-26T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.522+0000 7fd399e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x56090500f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:59152/0 (socket says 172.21.15.97:59152) 2024-03-26T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.523+0000 7fd399400640 1 -- 172.21.15.97:0/4037339626 learned_addr learned my addr 172.21.15.97:0/4037339626 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.523+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/4037339626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500fac0 0x560905011f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.523+0000 7fd3a3da8740 1 -- 172.21.15.97:0/4037339626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560904f6ee80 con 0x56090500fac0 2024-03-26T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.523+0000 7fd3a3da8740 1 -- 172.21.15.97:0/4037339626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560904f17660 con 0x56090500a950 2024-03-26T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.523+0000 7fd3a3da8740 1 -- 172.21.15.97:0/4037339626 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560904f6ae30 con 0x560905009f60 2024-03-26T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.523+0000 7fd399e00640 1 -- 172.21.15.97:0/4037339626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 msgr2=0x56090500a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.523+0000 7fd399e00640 1 --2- 172.21.15.97:0/4037339626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x56090500a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.523+0000 7fd399e00640 1 -- 172.21.15.97:0/4037339626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500fac0 msgr2=0x560905011f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.523+0000 7fd399e00640 1 --2- 172.21.15.97:0/4037339626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500fac0 0x560905011f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.523+0000 7fd399e00640 1 -- 172.21.15.97:0/4037339626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560904c83210 con 0x56090500a950 2024-03-26T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.524+0000 7fd392a00640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3800a2f90 0x7fd3800a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.524+0000 7fd399e00640 1 --2- 172.21.15.97:0/4037339626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x56090500f450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd38c02c190 tx=0x7fd38c004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48d25866300b10c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.524+0000 7fd392a00640 1 -- 172.21.15.97:0/1993231319 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd3940ce930 con 0x7fd39c1052a0 2024-03-26T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.524+0000 7fd387400640 1 -- 172.21.15.97:0/4037339626 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c043070 con 0x56090500a950 2024-03-26T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.525+0000 7fd387400640 1 -- 172.21.15.97:0/4037339626 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38c005270 con 0x56090500a950 2024-03-26T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.525+0000 7fd387400640 1 -- 172.21.15.97:0/4037339626 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c02cc70 con 0x56090500a950 2024-03-26T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.526+0000 7fd3a3da8740 1 -- 172.21.15.97:0/4037339626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 msgr2=0x56090500f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.526+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/4037339626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x56090500f450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd38c02c190 tx=0x7fd38c004740 comp rx=0 tx=0).stop 2024-03-26T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.526+0000 7fd399400640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3800a2f90 0x7fd3800a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.526+0000 7fd3a3da8740 1 -- 172.21.15.97:0/4037339626 shutdown_connections 2024-03-26T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.526+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/4037339626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500fac0 0x560905011f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.526+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/4037339626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x56090500f450 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.526+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/4037339626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x56090500a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.526+0000 7fd3a3da8740 1 -- 172.21.15.97:0/4037339626 >> 172.21.15.97:0/4037339626 conn(0x5609050054d0 msgr2=0x5609050058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.526+0000 7fd3a3da8740 1 -- 172.21.15.97:0/4037339626 shutdown_connections 2024-03-26T19:05:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.526+0000 7fd3a3da8740 1 -- 172.21.15.97:0/4037339626 wait complete. 2024-03-26T19:05:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.527+0000 7fd3a3da8740 1 -- start start 2024-03-26T19:05:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.527+0000 7fd3a3da8740 1 Processor -- start 2024-03-26T19:05:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905013ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500a950 0x5609050143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 0x56090501b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd3a3da8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560905012a10 con 0x56090500a950 2024-03-26T19:05:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd3a3da8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560904f2aac0 con 0x56090500fac0 2024-03-26T19:05:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd3a3da8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560905012b90 con 0x560905009f60 2024-03-26T19:05:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd3a3da8740 10 client.0 _ll_register_callbacks cb 0x560904f2a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:05:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd399e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905013ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd399e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905013ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34808/0 (socket says 172.21.15.97:34808) 2024-03-26T19:05:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd399e00640 1 -- 172.21.15.97:0/1867585723 learned_addr learned my addr 172.21.15.97:0/1867585723 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.528+0000 7fd39a800640 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 0x56090501b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.529+0000 7fd399e00640 1 -- 172.21.15.97:0/1867585723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 msgr2=0x56090501b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.529+0000 7fd399e00640 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 0x56090501b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.529+0000 7fd399e00640 1 -- 172.21.15.97:0/1867585723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500a950 msgr2=0x5609050143e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.529+0000 7fd399e00640 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500a950 0x5609050143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.529+0000 7fd399e00640 1 -- 172.21.15.97:0/1867585723 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560904d4f620 con 0x560905009f60 2024-03-26T19:05:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.529+0000 7fd399e00640 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905013ea0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd38c006010 tx=0x7fd38c004a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff8aea7ff46f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.530+0000 7fd37be00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:05:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.530+0000 7fd37be00640 1 -- 172.21.15.97:0/1867585723 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c04c070 con 0x560905009f60 2024-03-26T19:05:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.530+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1867585723 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd38c005270 con 0x560905009f60 2024-03-26T19:05:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.531+0000 7fd37be00640 1 -- 172.21.15.97:0/1867585723 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd38c039040 con 0x560905009f60 2024-03-26T19:05:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.531+0000 7fd37be00640 1 -- 172.21.15.97:0/1867585723 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd38c058050 con 0x560905009f60 2024-03-26T19:05:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.531+0000 7fd3a3da8740 10 client.25696 fetch_fsmap learned FSMap version 27 2024-03-26T19:05:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.531+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1867585723 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560905012a60 con 0x560905009f60 2024-03-26T19:05:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.531+0000 7fd399400640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3800a2f90 0x7fd3800a5450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x560905015380 tx=0x7fd38800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.532+0000 7fd37be00640 1 -- 172.21.15.97:0/1867585723 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38c005270 con 0x560905009f60 2024-03-26T19:05:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.532+0000 7fd37be00640 1 -- 172.21.15.97:0/1867585723 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd38c03fa70 con 0x560905009f60 2024-03-26T19:05:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.532+0000 7fd3a3da8740 10 client.25696 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:05:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.532+0000 7fd3a3da8740 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:05:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.532+0000 7fd3a3da8740 20 client.25696 populate_metadata read hostname 'smithi097' 2024-03-26T19:05:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.532+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56090501fa30 0x560905021e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.532+0000 7fd3a3da8740 4 client.25696 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:05:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.532+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1867585723 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560904f7c3c0 con 0x56090501fa30 2024-03-26T19:05:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.533+0000 7fd399400640 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56090501fa30 0x560905021e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.536+0000 7fd399400640 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56090501fa30 0x560905021e20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903dbf414aba2cef server_cookie=62403c06c54a5a67 in_seq=0 out_seq=0 2024-03-26T19:05:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.536+0000 7fd37be00640 10 client.25696 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd37be00640 1 -- 172.21.15.97:0/1867585723 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560904f7c3c0 con 0x56090501fa30 2024-03-26T19:05:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd37be00640 10 client.25696 handle_command_reply: tid=0 2024-03-26T19:05:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd3a3da8740 1 client.25696 shutdown 2024-03-26T19:05:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1867585723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 msgr2=0x560905013ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905013ea0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd38c006010 tx=0x7fd38c004a50 comp rx=0 tx=0).stop 2024-03-26T19:05:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1867585723 shutdown_connections 2024-03-26T19:05:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56090501fa30 0x560905021e20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 0x56090501b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500a950 0x5609050143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/1867585723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905013ea0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd37be00640 0 client.25696 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.537+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1867585723 >> 172.21.15.97:0/1867585723 conn(0x5609050054d0 msgr2=0x5609050046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.538+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1867585723 shutdown_connections 2024-03-26T19:05:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.538+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1867585723 wait complete. 2024-03-26T19:05:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.539+0000 7fd3a3da8740 20 client.25696 trim_cache size 0 max 16384 2024-03-26T19:05:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.568+0000 7fd3a3da8740 1 Processor -- start 2024-03-26T19:05:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.568+0000 7fd3a3da8740 1 -- start start 2024-03-26T19:05:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.568+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905008730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.568+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500a950 0x56090501b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.568+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 0x5609050134c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.568+0000 7fd3a3da8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c1af090 con 0x56090500a950 2024-03-26T19:05:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.568+0000 7fd3a3da8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c1b0bd0 con 0x56090500fac0 2024-03-26T19:05:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.568+0000 7fd3a3da8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c1af760 con 0x560905009f60 2024-03-26T19:05:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.569+0000 7fd399e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.569+0000 7fd399e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905008730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34818/0 (socket says 172.21.15.97:34818) 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.569+0000 7fd399e00640 1 -- 172.21.15.97:0/1476272852 learned_addr learned my addr 172.21.15.97:0/1476272852 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.569+0000 7fd39a800640 1 --2- 172.21.15.97:0/1476272852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 0x5609050134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.569+0000 7fd399400640 1 --2- 172.21.15.97:0/1476272852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500a950 0x56090501b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.569+0000 7fd399e00640 1 -- 172.21.15.97:0/1476272852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 msgr2=0x5609050134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.569+0000 7fd399e00640 1 --2- 172.21.15.97:0/1476272852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 0x5609050134c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.569+0000 7fd399e00640 1 -- 172.21.15.97:0/1476272852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500a950 msgr2=0x56090501b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.569+0000 7fd399e00640 1 --2- 172.21.15.97:0/1476272852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500a950 0x56090501b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.569+0000 7fd399e00640 1 -- 172.21.15.97:0/1476272852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560905012f60 con 0x560905009f60 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.570+0000 7fd39a800640 1 --2- 172.21.15.97:0/1476272852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 0x5609050134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.570+0000 7fd399e00640 1 --2- 172.21.15.97:0/1476272852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905008730 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd38c042000 tx=0x7fd38c004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b78524f82f19c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.570+0000 7fd386a00640 1 -- 172.21.15.97:0/1476272852 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c055070 con 0x560905009f60 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.570+0000 7fd386a00640 1 -- 172.21.15.97:0/1476272852 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38c048d80 con 0x560905009f60 2024-03-26T19:05:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.570+0000 7fd386a00640 1 -- 172.21.15.97:0/1476272852 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c04c070 con 0x560905009f60 2024-03-26T19:05:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.571+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1476272852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 msgr2=0x560905008730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.571+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/1476272852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905008730 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd38c042000 tx=0x7fd38c004740 comp rx=0 tx=0).stop 2024-03-26T19:05:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.571+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1476272852 shutdown_connections 2024-03-26T19:05:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.571+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/1476272852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500fac0 0x5609050134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.571+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/1476272852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500a950 0x56090501b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.571+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/1476272852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x560905008730 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.571+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1476272852 >> 172.21.15.97:0/1476272852 conn(0x5609050054d0 msgr2=0x56090500bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.571+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1476272852 shutdown_connections 2024-03-26T19:05:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.571+0000 7fd3a3da8740 1 -- 172.21.15.97:0/1476272852 wait complete. 2024-03-26T19:05:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.572+0000 7fd3a3da8740 1 -- start start 2024-03-26T19:05:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.573+0000 7fd3a3da8740 1 Processor -- start 2024-03-26T19:05:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.573+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x5609050189a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.573+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x560905018ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.574+0000 7fd3a3da8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500fac0 0x560905019ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.574+0000 7fd3a3da8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c1b0500 con 0x56090500fac0 2024-03-26T19:05:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.574+0000 7fd3a3da8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c1afe30 con 0x56090500a950 2024-03-26T19:05:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.574+0000 7fd3a3da8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c1ad070 con 0x560905009f60 2024-03-26T19:05:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.574+0000 7fd3a3da8740 10 client.0 _ll_register_callbacks cb 0x7fd38c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:05:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.574+0000 7fd399400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x560905018ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.574+0000 7fd399400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x560905018ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:59184/0 (socket says 172.21.15.97:59184) 2024-03-26T19:05:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.574+0000 7fd399400640 1 -- 172.21.15.97:0/3541545788 learned_addr learned my addr 172.21.15.97:0/3541545788 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.574+0000 7fd399e00640 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x5609050189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.575+0000 7fd39a800640 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500fac0 0x560905019ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.575+0000 7fd399400640 1 -- 172.21.15.97:0/3541545788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 msgr2=0x5609050189a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.575+0000 7fd399400640 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x5609050189a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.575+0000 7fd399400640 1 -- 172.21.15.97:0/3541545788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500fac0 msgr2=0x560905019ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.575+0000 7fd399400640 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500fac0 0x560905019ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.575+0000 7fd399400640 1 -- 172.21.15.97:0/3541545788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560905000ae0 con 0x56090500a950 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.575+0000 7fd39a800640 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500fac0 0x560905019ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.575+0000 7fd399400640 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x560905018ee0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd388005fd0 tx=0x7fd3880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c141062426aaed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.575+0000 7fd37be00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.576+0000 7fd37be00640 1 -- 172.21.15.97:0/3541545788 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388026c30 con 0x56090500a950 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.576+0000 7fd37be00640 1 -- 172.21.15.97:0/3541545788 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd388026dd0 con 0x56090500a950 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.576+0000 7fd3a3da8740 1 -- 172.21.15.97:0/3541545788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd38c048d80 con 0x56090500a950 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.576+0000 7fd37be00640 1 -- 172.21.15.97:0/3541545788 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd38801d4d0 con 0x56090500a950 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.576+0000 7fd3a3da8740 10 client.25745 fetch_fsmap learned FSMap version 27 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.576+0000 7fd3a3da8740 1 -- 172.21.15.97:0/3541545788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56090501b670 con 0x56090500a950 2024-03-26T19:05:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.577+0000 7fd37be00640 1 -- 172.21.15.97:0/3541545788 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38c048d80 con 0x56090500a950 2024-03-26T19:05:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.577+0000 7fd37be00640 1 -- 172.21.15.97:0/3541545788 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd388019040 con 0x56090500a950 2024-03-26T19:05:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.577+0000 7fd3a3da8740 10 client.25745 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:05:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.577+0000 7fd3a3da8740 10 client.25745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:05:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.577+0000 7fd3a3da8740 20 client.25745 populate_metadata read hostname 'smithi097' 2024-03-26T19:05:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.577+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560905086470 0x560905088890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.577+0000 7fd3a3da8740 4 client.25745 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:05:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.577+0000 7fd3a3da8740 1 -- 172.21.15.97:0/3541545788 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560904f76710 con 0x560905086470 2024-03-26T19:05:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.577+0000 7fd399e00640 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560905086470 0x560905088890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.578+0000 7fd37be00640 10 client.25745 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.578+0000 7fd399e00640 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560905086470 0x560905088890 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec8c2b741aca12f server_cookie=457519287fba4d0d in_seq=0 out_seq=0 2024-03-26T19:05:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.593+0000 7fd37be00640 1 -- 172.21.15.97:0/3541545788 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560904f76710 con 0x560905086470 2024-03-26T19:05:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.593+0000 7fd37be00640 10 client.25745 handle_command_reply: tid=0 2024-03-26T19:05:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.593+0000 7fd3a3da8740 1 client.25745 shutdown 2024-03-26T19:05:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 1 -- 172.21.15.97:0/3541545788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 msgr2=0x560905018ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x560905018ee0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd388005fd0 tx=0x7fd3880046c0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 1 -- 172.21.15.97:0/3541545788 shutdown_connections 2024-03-26T19:05:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560905086470 0x560905088890 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd37be00640 0 client.25745 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56090500fac0 0x560905019ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56090500a950 0x560905018ee0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 1 --2- 172.21.15.97:0/3541545788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560905009f60 0x5609050189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 1 -- 172.21.15.97:0/3541545788 >> 172.21.15.97:0/3541545788 conn(0x5609050054d0 msgr2=0x560905003e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 1 -- 172.21.15.97:0/3541545788 shutdown_connections 2024-03-26T19:05:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 1 -- 172.21.15.97:0/3541545788 wait complete. 2024-03-26T19:05:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.594+0000 7fd3a3da8740 20 client.25745 trim_cache size 0 max 16384 2024-03-26T19:05:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:05:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:05:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:05:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:05:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 -- 172.21.15.97:0/1993231319 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3800a2f90 msgr2=0x7fd3800a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3800a2f90 0x7fd3800a5450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x560905015380 tx=0x7fd38800b410 comp rx=0 tx=0).stop 2024-03-26T19:05:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 -- 172.21.15.97:0/1993231319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c1052a0 msgr2=0x7fd39c1a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c1052a0 0x7fd39c1a5600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd39400b970 tx=0x7fd39400be30 comp rx=0 tx=0).stop 2024-03-26T19:05:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 -- 172.21.15.97:0/1993231319 shutdown_connections 2024-03-26T19:05:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd3800a2f90 0x7fd3800a5450 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c1052a0 0x7fd39c1a5600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39c1028f0 0x7fd39c19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 --2- 172.21.15.97:0/1993231319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd39c0fff80 0x7fd39c19e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 -- 172.21.15.97:0/1993231319 >> 172.21.15.97:0/1993231319 conn(0x7fd39c0fbba0 msgr2=0x7fd39c103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.595+0000 7fd39b200640 1 -- 172.21.15.97:0/1993231319 shutdown_connections 2024-03-26T19:05:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:09.596+0000 7fd39b200640 1 -- 172.21.15.97:0/1993231319 wait complete. 2024-03-26T19:05:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 345.0 s, 1954.0 tps, lat 51.515 ms stddev 66.261 2024-03-26T19:05:10.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:10 smithi105 ceph-mon[36270]: pgmap v577: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.0 MiB/s wr, 725 op/s 2024-03-26T19:05:10.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:10 smithi097 ceph-mon[31955]: pgmap v577: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.0 MiB/s wr, 725 op/s 2024-03-26T19:05:10.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:10 smithi144 ceph-mon[36053]: pgmap v577: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.0 MiB/s wr, 725 op/s 2024-03-26T19:05:12.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:12 smithi105 ceph-mon[36270]: pgmap v578: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 653 op/s 2024-03-26T19:05:12.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:12 smithi097 ceph-mon[31955]: pgmap v578: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 653 op/s 2024-03-26T19:05:12.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:12 smithi144 ceph-mon[36053]: pgmap v578: 97 pgs: 97 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 653 op/s 2024-03-26T19:05:13.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:13 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:05:13.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:13 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:05:13.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:13 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:05:14.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:14 smithi105 ceph-mon[36270]: pgmap v579: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 671 op/s 2024-03-26T19:05:14.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:14 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:14 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:14 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:14 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:14 smithi097 ceph-mon[31955]: pgmap v579: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 671 op/s 2024-03-26T19:05:14.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:14 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:14 smithi144 ceph-mon[36053]: pgmap v579: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 671 op/s 2024-03-26T19:05:14.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:14 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:14 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:14 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:14.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:14 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 350.0 s, 1952.2 tps, lat 51.049 ms stddev 57.411 2024-03-26T19:05:15.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:15 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-26T19:05:15.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:15 smithi105 ceph-mon[36270]: Unable to set osd_memory_target on smithi105 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-26T19:05:15.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:05:15.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:05:15.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:05:15.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:15 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-26T19:05:15.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:15 smithi097 ceph-mon[31955]: Unable to set osd_memory_target on smithi105 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-26T19:05:15.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:05:15.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:05:15.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:05:15.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:15 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-26T19:05:15.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:15 smithi144 ceph-mon[36053]: Unable to set osd_memory_target on smithi105 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-26T19:05:15.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:05:15.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:05:15.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:05:15.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:05:16.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:16 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi097 to 176.7M 2024-03-26T19:05:16.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:16 smithi105 ceph-mon[36270]: Unable to set osd_memory_target on smithi097 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-26T19:05:16.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:16 smithi105 ceph-mon[36270]: pgmap v580: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 750 op/s 2024-03-26T19:05:16.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:16 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi097 to 176.7M 2024-03-26T19:05:16.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:16 smithi097 ceph-mon[31955]: Unable to set osd_memory_target on smithi097 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-26T19:05:16.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:16 smithi097 ceph-mon[31955]: pgmap v580: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 750 op/s 2024-03-26T19:05:16.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:16 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi097 to 176.7M 2024-03-26T19:05:16.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:16 smithi144 ceph-mon[36053]: Unable to set osd_memory_target on smithi097 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-26T19:05:16.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:16 smithi144 ceph-mon[36053]: pgmap v580: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 750 op/s 2024-03-26T19:05:18.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:18 smithi097 ceph-mon[31955]: pgmap v581: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-26T19:05:18.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:18 smithi144 ceph-mon[36053]: pgmap v581: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-26T19:05:18.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:18 smithi105 ceph-mon[36270]: pgmap v581: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-26T19:05:20.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 355.0 s, 1752.4 tps, lat 57.063 ms stddev 71.439 2024-03-26T19:05:20.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:20 smithi097 ceph-mon[31955]: pgmap v582: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-03-26T19:05:20.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:20 smithi144 ceph-mon[36053]: pgmap v582: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-03-26T19:05:20.713 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:20 smithi105 ceph-mon[36270]: pgmap v582: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-03-26T19:05:22.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:22 smithi097 ceph-mon[31955]: pgmap v583: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.1 MiB/s wr, 632 op/s 2024-03-26T19:05:22.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:22 smithi144 ceph-mon[36053]: pgmap v583: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.1 MiB/s wr, 632 op/s 2024-03-26T19:05:22.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:22 smithi105 ceph-mon[36270]: pgmap v583: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.1 MiB/s wr, 632 op/s 2024-03-26T19:05:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.799+0000 7f5f89400640 1 -- 172.21.15.97:0/979640631 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c032340 con 0x7f5f8c103b20 2024-03-26T19:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.800+0000 7f5f90c00640 1 -- 172.21.15.97:0/979640631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f8c103b20 msgr2=0x7f5f8c103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.800+0000 7f5f90c00640 1 --2- 172.21.15.97:0/979640631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f8c103b20 0x7f5f8c103f40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009860 tx=0x7f5f7c02bb70 comp rx=0 tx=0).stop 2024-03-26T19:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.801+0000 7f5f90c00640 1 -- 172.21.15.97:0/979640631 shutdown_connections 2024-03-26T19:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.801+0000 7f5f90c00640 1 --2- 172.21.15.97:0/979640631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8c1056d0 0x7f5f8c10bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.801+0000 7f5f90c00640 1 --2- 172.21.15.97:0/979640631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8c104d10 0x7f5f8c105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.801+0000 7f5f90c00640 1 --2- 172.21.15.97:0/979640631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f8c103b20 0x7f5f8c103f40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.801+0000 7f5f90c00640 1 -- 172.21.15.97:0/979640631 >> 172.21.15.97:0/979640631 conn(0x7f5f8c0ff0b0 msgr2=0x7f5f8c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.801+0000 7f5f90c00640 1 -- 172.21.15.97:0/979640631 shutdown_connections 2024-03-26T19:05:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.801+0000 7f5f90c00640 1 -- 172.21.15.97:0/979640631 wait complete. 2024-03-26T19:05:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.802+0000 7f5f90c00640 1 Processor -- start 2024-03-26T19:05:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.803+0000 7f5f90c00640 1 -- start start 2024-03-26T19:05:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.803+0000 7f5f90c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8c103b20 0x7f5f8c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.803+0000 7f5f90c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8c104d10 0x7f5f8c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.803+0000 7f5f90c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f8c1056d0 0x7f5f8c1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.803+0000 7f5f90c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10f7c0 con 0x7f5f8c103b20 2024-03-26T19:05:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.803+0000 7f5f90c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10f640 con 0x7f5f8c104d10 2024-03-26T19:05:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.803+0000 7f5f90c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10f940 con 0x7f5f8c1056d0 2024-03-26T19:05:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.803+0000 7f5f8a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f8c1056d0 0x7f5f8c1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.803+0000 7f5f8a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f8c1056d0 0x7f5f8c1a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53582/0 (socket says 172.21.15.97:53582) 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.803+0000 7f5f8a800640 1 -- 172.21.15.97:0/2385618144 learned_addr learned my addr 172.21.15.97:0/2385618144 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.804+0000 7f5f8a800640 1 -- 172.21.15.97:0/2385618144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8c104d10 msgr2=0x7f5f8c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.804+0000 7f5f8a800640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8c104d10 0x7f5f8c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.804+0000 7f5f8a800640 1 -- 172.21.15.97:0/2385618144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8c103b20 msgr2=0x7f5f8c199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.804+0000 7f5f8a800640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8c103b20 0x7f5f8c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.804+0000 7f5f8a800640 1 -- 172.21.15.97:0/2385618144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c06b410 con 0x7f5f8c1056d0 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.804+0000 7f5f8a800640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f8c1056d0 0x7f5f8c1a10f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400eda0 tx=0x7f5f8400c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.805+0000 7f5f83400640 1 -- 172.21.15.97:0/2385618144 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84003b70 con 0x7f5f8c1056d0 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.805+0000 7f5f90c00640 1 -- 172.21.15.97:0/2385618144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c06b700 con 0x7f5f8c1056d0 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.805+0000 7f5f83400640 1 -- 172.21.15.97:0/2385618144 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f84003d10 con 0x7f5f8c1056d0 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.805+0000 7f5f83400640 1 -- 172.21.15.97:0/2385618144 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f840179b0 con 0x7f5f8c1056d0 2024-03-26T19:05:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.805+0000 7f5f90c00640 1 -- 172.21.15.97:0/2385618144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c06bc50 con 0x7f5f8c1056d0 2024-03-26T19:05:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.811+0000 7f5f9384a740 1 Processor -- start 2024-03-26T19:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.811+0000 7f5f83400640 1 -- 172.21.15.97:0/2385618144 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f5f84017b50 con 0x7f5f8c1056d0 2024-03-26T19:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.811+0000 7f5f9384a740 1 -- start start 2024-03-26T19:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.811+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb685b0 0x5614dbb689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.811+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb68fa0 0x5614dbb6daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.811+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.811+0000 7f5f9384a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5614dbac4e80 con 0x5614dbb68fa0 2024-03-26T19:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.812+0000 7f5f9384a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5614dba6d660 con 0x5614dbb685b0 2024-03-26T19:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.812+0000 7f5f9384a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614dbac0e30 con 0x5614dbb6e110 2024-03-26T19:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.812+0000 7f5f81400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb685b0 0x5614dbb689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.812+0000 7f5f81400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb685b0 0x5614dbb689d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:41182/0 (socket says 172.21.15.97:41182) 2024-03-26T19:05:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.812+0000 7f5f81400640 1 -- 172.21.15.97:0/3915657468 learned_addr learned my addr 172.21.15.97:0/3915657468 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.812+0000 7f5f81400640 1 --2- 172.21.15.97:0/3915657468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.813+0000 7f5f83400640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f700a30c0 0x7f5f700a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.813+0000 7f5f81400640 1 -- 172.21.15.97:0/3915657468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 msgr2=0x5614dbb705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.813+0000 7f5f81400640 1 --2- 172.21.15.97:0/3915657468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.813+0000 7f5f81400640 1 -- 172.21.15.97:0/3915657468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb68fa0 msgr2=0x5614dbb6daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.813+0000 7f5f83400640 1 -- 172.21.15.97:0/2385618144 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5f840cf790 con 0x7f5f8c1056d0 2024-03-26T19:05:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.814+0000 7f5f81400640 1 --2- 172.21.15.97:0/3915657468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb68fa0 0x5614dbb6daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.814+0000 7f5f8a800640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f700a30c0 0x7f5f700a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.814+0000 7f5f81400640 1 -- 172.21.15.97:0/3915657468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614db7d9210 con 0x5614dbb685b0 2024-03-26T19:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.815+0000 7f5f81400640 1 --2- 172.21.15.97:0/3915657468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb685b0 0x5614dbb689d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5f740127b0 tx=0x7f5f7400d3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b43fe10bcaa2f78d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.815+0000 7f5f8a800640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f700a30c0 0x7f5f700a5580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400ce20 tx=0x7f5f8400cd90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.817+0000 7f5f7b000640 1 -- 172.21.15.97:0/3915657468 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7400a6b0 con 0x5614dbb685b0 2024-03-26T19:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.817+0000 7f5f7b000640 1 -- 172.21.15.97:0/3915657468 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7400ad30 con 0x5614dbb685b0 2024-03-26T19:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.817+0000 7f5f7b000640 1 -- 172.21.15.97:0/3915657468 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74010ce0 con 0x5614dbb685b0 2024-03-26T19:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.818+0000 7f5f9384a740 1 -- 172.21.15.97:0/3915657468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb685b0 msgr2=0x5614dbb689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.818+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3915657468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb685b0 0x5614dbb689d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5f740127b0 tx=0x7f5f7400d3c0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.818+0000 7f5f9384a740 1 -- 172.21.15.97:0/3915657468 shutdown_connections 2024-03-26T19:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.818+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3915657468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.818+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3915657468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb68fa0 0x5614dbb6daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.818+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3915657468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb685b0 0x5614dbb689d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.818+0000 7f5f9384a740 1 -- 172.21.15.97:0/3915657468 >> 172.21.15.97:0/3915657468 conn(0x5614dbb5b310 msgr2=0x5614dbb5b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.818+0000 7f5f9384a740 1 -- 172.21.15.97:0/3915657468 shutdown_connections 2024-03-26T19:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.818+0000 7f5f9384a740 1 -- 172.21.15.97:0/3915657468 wait complete. 2024-03-26T19:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.819+0000 7f5f9384a740 1 -- start start 2024-03-26T19:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.819+0000 7f5f9384a740 1 Processor -- start 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.820+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 0x5614dbb61550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.820+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.820+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb66a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.820+0000 7f5f9384a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5614dbb5fe80 con 0x5614dbb685b0 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.820+0000 7f5f9384a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5614dba80ac0 con 0x5614dbb68fa0 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.820+0000 7f5f9384a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614dbb60000 con 0x5614dbb6e110 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.820+0000 7f5f9384a740 10 client.0 _ll_register_callbacks cb 0x5614dba80490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.821+0000 7f5f89e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.821+0000 7f5f89e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:41186/0 (socket says 172.21.15.97:41186) 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.821+0000 7f5f89e00640 1 -- 172.21.15.97:0/3471517925 learned_addr learned my addr 172.21.15.97:0/3471517925 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.821+0000 7f5f81400640 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 0x5614dbb61550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.822+0000 7f5f81400640 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb66a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.822+0000 7f5f89e00640 1 -- 172.21.15.97:0/3471517925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 msgr2=0x5614dbb66a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.822+0000 7f5f89e00640 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb66a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.822+0000 7f5f89e00640 1 -- 172.21.15.97:0/3471517925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 msgr2=0x5614dbb61550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.822+0000 7f5f89e00640 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 0x5614dbb61550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.822+0000 7f5f89e00640 1 -- 172.21.15.97:0/3471517925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614db8a5620 con 0x5614dbb68fa0 2024-03-26T19:05:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.822+0000 7f5f81400640 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb66a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:05:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.823+0000 7f5f89e00640 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c037040 tx=0x7f5f7c034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7da92fb5e68ad83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.823+0000 7f5f6f400640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:05:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.824+0000 7f5f6f400640 1 -- 172.21.15.97:0/3471517925 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c041020 con 0x5614dbb68fa0 2024-03-26T19:05:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.824+0000 7f5f6f400640 1 -- 172.21.15.97:0/3471517925 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5f7c004040 con 0x5614dbb68fa0 2024-03-26T19:05:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.824+0000 7f5f9384a740 1 -- 172.21.15.97:0/3471517925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f7400ad30 con 0x5614dbb68fa0 2024-03-26T19:05:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.827+0000 7f5f6f400640 1 -- 172.21.15.97:0/3471517925 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f7c039040 con 0x5614dbb68fa0 2024-03-26T19:05:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.827+0000 7f5f9384a740 10 client.25754 fetch_fsmap learned FSMap version 27 2024-03-26T19:05:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.827+0000 7f5f9384a740 1 -- 172.21.15.97:0/3471517925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614dbb857f0 con 0x5614dbb68fa0 2024-03-26T19:05:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.827+0000 7f5f6f400640 1 -- 172.21.15.97:0/3471517925 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7400ad30 con 0x5614dbb68fa0 2024-03-26T19:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.828+0000 7f5f6f400640 1 -- 172.21.15.97:0/3471517925 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5f7c03c050 con 0x5614dbb68fa0 2024-03-26T19:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.828+0000 7f5f9384a740 10 client.25754 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.828+0000 7f5f9384a740 10 client.25754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.828+0000 7f5f9384a740 20 client.25754 populate_metadata read hostname 'smithi097' 2024-03-26T19:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.828+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5614dbb86e10 0x5614dbb89200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.828+0000 7f5f9384a740 4 client.25754 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.828+0000 7f5f9384a740 1 -- 172.21.15.97:0/3471517925 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614dbad23c0 con 0x5614dbb86e10 2024-03-26T19:05:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.829+0000 7f5f81400640 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5614dbb86e10 0x5614dbb89200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.829+0000 7f5f6f400640 10 client.25754 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.830+0000 7f5f81400640 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5614dbb86e10 0x5614dbb89200 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56714c98f72dc31 server_cookie=af07e116a07854bb in_seq=0 out_seq=0 2024-03-26T19:05:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.831+0000 7f5f6f400640 1 -- 172.21.15.97:0/3471517925 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5614dbad23c0 con 0x5614dbb86e10 2024-03-26T19:05:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.832+0000 7f5f6f400640 10 client.25754 handle_command_reply: tid=0 2024-03-26T19:05:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.832+0000 7f5f9384a740 1 client.25754 shutdown 2024-03-26T19:05:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.833+0000 7f5f9384a740 1 -- 172.21.15.97:0/3471517925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 msgr2=0x5614dbb61a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.833+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c037040 tx=0x7f5f7c034670 comp rx=0 tx=0).stop 2024-03-26T19:05:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.836+0000 7f5f9384a740 1 -- 172.21.15.97:0/3471517925 shutdown_connections 2024-03-26T19:05:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.836+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5614dbb86e10 0x5614dbb89200 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.836+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb66a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.836+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.836+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3471517925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 0x5614dbb61550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.836+0000 7f5f9384a740 1 -- 172.21.15.97:0/3471517925 >> 172.21.15.97:0/3471517925 conn(0x5614dbb5b310 msgr2=0x5614dbb5a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.836+0000 7f5f6f400640 0 client.25754 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.836+0000 7f5f9384a740 1 -- 172.21.15.97:0/3471517925 shutdown_connections 2024-03-26T19:05:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.837+0000 7f5f9384a740 1 -- 172.21.15.97:0/3471517925 wait complete. 2024-03-26T19:05:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.838+0000 7f5f9384a740 20 client.25754 trim_cache size 0 max 16384 2024-03-26T19:05:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.871+0000 7f5f9384a740 1 Processor -- start 2024-03-26T19:05:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.872+0000 7f5f9384a740 1 -- start start 2024-03-26T19:05:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.872+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 0x5614dbb61550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.872+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.872+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb69910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.872+0000 7f5f9384a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c0705c0 con 0x5614dbb685b0 2024-03-26T19:05:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.872+0000 7f5f9384a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c072100 con 0x5614dbb68fa0 2024-03-26T19:05:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.872+0000 7f5f9384a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c070c90 con 0x5614dbb6e110 2024-03-26T19:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.873+0000 7f5f81400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 0x5614dbb61550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.873+0000 7f5f81400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 0x5614dbb61550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41002/0 (socket says 172.21.15.97:41002) 2024-03-26T19:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.873+0000 7f5f81400640 1 -- 172.21.15.97:0/4158002814 learned_addr learned my addr 172.21.15.97:0/4158002814 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.873+0000 7f5f89e00640 1 --2- 172.21.15.97:0/4158002814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.873+0000 7f5f81400640 1 --2- 172.21.15.97:0/4158002814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb69910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.875+0000 7f5f89e00640 1 -- 172.21.15.97:0/4158002814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 msgr2=0x5614dbb69910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.875+0000 7f5f89e00640 1 --2- 172.21.15.97:0/4158002814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb69910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.875+0000 7f5f89e00640 1 -- 172.21.15.97:0/4158002814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 msgr2=0x5614dbb61550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.875+0000 7f5f89e00640 1 --2- 172.21.15.97:0/4158002814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 0x5614dbb61550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.875+0000 7f5f89e00640 1 -- 172.21.15.97:0/4158002814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614dbb59ab0 con 0x5614dbb68fa0 2024-03-26T19:05:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.876+0000 7f5f89e00640 1 --2- 172.21.15.97:0/4158002814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c032f10 tx=0x7f5f7c0097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e118e8bad07049 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.877+0000 7f5f7a600640 1 -- 172.21.15.97:0/4158002814 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c052020 con 0x5614dbb68fa0 2024-03-26T19:05:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.877+0000 7f5f7a600640 1 -- 172.21.15.97:0/4158002814 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7c032b20 con 0x5614dbb68fa0 2024-03-26T19:05:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.877+0000 7f5f7a600640 1 -- 172.21.15.97:0/4158002814 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c041050 con 0x5614dbb68fa0 2024-03-26T19:05:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.878+0000 7f5f9384a740 1 -- 172.21.15.97:0/4158002814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 msgr2=0x5614dbb61a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.878+0000 7f5f9384a740 1 --2- 172.21.15.97:0/4158002814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c032f10 tx=0x7f5f7c0097f0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.878+0000 7f5f9384a740 1 -- 172.21.15.97:0/4158002814 shutdown_connections 2024-03-26T19:05:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.878+0000 7f5f9384a740 1 --2- 172.21.15.97:0/4158002814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb6e110 0x5614dbb69910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.878+0000 7f5f9384a740 1 --2- 172.21.15.97:0/4158002814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb68fa0 0x5614dbb61a90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.878+0000 7f5f9384a740 1 --2- 172.21.15.97:0/4158002814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb685b0 0x5614dbb61550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.878+0000 7f5f9384a740 1 -- 172.21.15.97:0/4158002814 >> 172.21.15.97:0/4158002814 conn(0x5614dbb5b310 msgr2=0x5614dbb6d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.878+0000 7f5f9384a740 1 -- 172.21.15.97:0/4158002814 shutdown_connections 2024-03-26T19:05:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.878+0000 7f5f9384a740 1 -- 172.21.15.97:0/4158002814 wait complete. 2024-03-26T19:05:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.879+0000 7f5f9384a740 1 -- start start 2024-03-26T19:05:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.879+0000 7f5f9384a740 1 Processor -- start 2024-03-26T19:05:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.880+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb685b0 0x5614dbb981b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.880+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb68fa0 0x5614dbb986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.880+0000 7f5f9384a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb6e110 0x5614dbb99690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.880+0000 7f5f9384a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c071a30 con 0x5614dbb68fa0 2024-03-26T19:05:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.880+0000 7f5f9384a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c071360 con 0x5614dbb6e110 2024-03-26T19:05:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.880+0000 7f5f9384a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c06e5a0 con 0x5614dbb685b0 2024-03-26T19:05:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.880+0000 7f5f9384a740 10 client.0 _ll_register_callbacks cb 0x7f5f7c0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:05:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.881+0000 7f5f81400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb685b0 0x5614dbb981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.881+0000 7f5f81400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb685b0 0x5614dbb981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53622/0 (socket says 172.21.15.97:53622) 2024-03-26T19:05:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.881+0000 7f5f81400640 1 -- 172.21.15.97:0/3540483871 learned_addr learned my addr 172.21.15.97:0/3540483871 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.881+0000 7f5f81400640 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb6e110 0x5614dbb99690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.882+0000 7f5f89e00640 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb68fa0 0x5614dbb986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.882+0000 7f5f81400640 1 -- 172.21.15.97:0/3540483871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb6e110 msgr2=0x5614dbb99690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.883+0000 7f5f81400640 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb6e110 0x5614dbb99690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.883+0000 7f5f81400640 1 -- 172.21.15.97:0/3540483871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb68fa0 msgr2=0x5614dbb986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.883+0000 7f5f81400640 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb68fa0 0x5614dbb986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.883+0000 7f5f81400640 1 -- 172.21.15.97:0/3540483871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614dbb99c80 con 0x5614dbb685b0 2024-03-26T19:05:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.883+0000 7f5f89e00640 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb68fa0 0x5614dbb986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:05:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.884+0000 7f5f81400640 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb685b0 0x5614dbb981b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5f740059f0 tx=0x7f5f74020040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc5fe9b30852f528 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.884+0000 7f5f6f400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:05:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.885+0000 7f5f6f400640 1 -- 172.21.15.97:0/3540483871 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7402ac00 con 0x5614dbb685b0 2024-03-26T19:05:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.885+0000 7f5f6f400640 1 -- 172.21.15.97:0/3540483871 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5f7402ada0 con 0x5614dbb685b0 2024-03-26T19:05:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.885+0000 7f5f9384a740 1 -- 172.21.15.97:0/3540483871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f7c032b20 con 0x5614dbb685b0 2024-03-26T19:05:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.887+0000 7f5f6f400640 1 -- 172.21.15.97:0/3540483871 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f74012070 con 0x5614dbb685b0 2024-03-26T19:05:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.887+0000 7f5f9384a740 10 client.25717 fetch_fsmap learned FSMap version 27 2024-03-26T19:05:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.887+0000 7f5f9384a740 1 -- 172.21.15.97:0/3540483871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614dbbed2a0 con 0x5614dbb685b0 2024-03-26T19:05:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.888+0000 7f5f6f400640 1 -- 172.21.15.97:0/3540483871 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7c032b20 con 0x5614dbb685b0 2024-03-26T19:05:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.888+0000 7f5f6f400640 1 -- 172.21.15.97:0/3540483871 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5f7402a3c0 con 0x5614dbb685b0 2024-03-26T19:05:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.888+0000 7f5f9384a740 10 client.25717 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:05:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.888+0000 7f5f9384a740 10 client.25717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:05:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.888+0000 7f5f9384a740 20 client.25717 populate_metadata read hostname 'smithi097' 2024-03-26T19:05:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.888+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5614dbbedaf0 0x5614dbbeff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.888+0000 7f5f9384a740 4 client.25717 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:05:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.888+0000 7f5f9384a740 1 -- 172.21.15.97:0/3540483871 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614dbacc710 con 0x5614dbbedaf0 2024-03-26T19:05:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.888+0000 7f5f89e00640 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5614dbbedaf0 0x5614dbbeff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.889+0000 7f5f89e00640 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5614dbbedaf0 0x5614dbbeff10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4852e84755099df server_cookie=413cec7c88a15b01 in_seq=0 out_seq=0 2024-03-26T19:05:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.890+0000 7f5f6f400640 10 client.25717 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.920+0000 7f5f6f400640 1 -- 172.21.15.97:0/3540483871 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614dbacc710 con 0x5614dbbedaf0 2024-03-26T19:05:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.920+0000 7f5f6f400640 10 client.25717 handle_command_reply: tid=0 2024-03-26T19:05:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.920+0000 7f5f9384a740 1 client.25717 shutdown 2024-03-26T19:05:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.920+0000 7f5f9384a740 1 -- 172.21.15.97:0/3540483871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb685b0 msgr2=0x5614dbb981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.920+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb685b0 0x5614dbb981b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5f740059f0 tx=0x7f5f74020040 comp rx=0 tx=0).stop 2024-03-26T19:05:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.920+0000 7f5f9384a740 1 -- 172.21.15.97:0/3540483871 shutdown_connections 2024-03-26T19:05:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.921+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5614dbbedaf0 0x5614dbbeff10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.921+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614dbb6e110 0x5614dbb99690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.921+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614dbb68fa0 0x5614dbb986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.921+0000 7f5f9384a740 1 --2- 172.21.15.97:0/3540483871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614dbb685b0 0x5614dbb981b0 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5f740059f0 tx=0x7f5f74020040 comp rx=0 tx=0).stop 2024-03-26T19:05:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.921+0000 7f5f9384a740 1 -- 172.21.15.97:0/3540483871 >> 172.21.15.97:0/3540483871 conn(0x5614dbb5b310 msgr2=0x5614dbb61530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.921+0000 7f5f6f400640 0 client.25717 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.921+0000 7f5f9384a740 1 -- 172.21.15.97:0/3540483871 shutdown_connections 2024-03-26T19:05:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.921+0000 7f5f9384a740 1 -- 172.21.15.97:0/3540483871 wait complete. 2024-03-26T19:05:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.921+0000 7f5f9384a740 20 client.25717 trim_cache size 0 max 16384 2024-03-26T19:05:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:05:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:05:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:05:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:05:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.923+0000 7f5f90c00640 1 -- 172.21.15.97:0/2385618144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f700a30c0 msgr2=0x7f5f700a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.923+0000 7f5f90c00640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f700a30c0 0x7f5f700a5580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400ce20 tx=0x7f5f8400cd90 comp rx=0 tx=0).stop 2024-03-26T19:05:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.923+0000 7f5f90c00640 1 -- 172.21.15.97:0/2385618144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f8c1056d0 msgr2=0x7f5f8c1a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.923+0000 7f5f90c00640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f8c1056d0 0x7f5f8c1a10f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400eda0 tx=0x7f5f8400c6e0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.923+0000 7f5f90c00640 1 -- 172.21.15.97:0/2385618144 shutdown_connections 2024-03-26T19:05:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.923+0000 7f5f90c00640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5f700a30c0 0x7f5f700a5580 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.923+0000 7f5f90c00640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f8c1056d0 0x7f5f8c1a10f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.923+0000 7f5f90c00640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8c104d10 0x7f5f8c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.923+0000 7f5f90c00640 1 --2- 172.21.15.97:0/2385618144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8c103b20 0x7f5f8c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.923+0000 7f5f90c00640 1 -- 172.21.15.97:0/2385618144 >> 172.21.15.97:0/2385618144 conn(0x7f5f8c0ff0b0 msgr2=0x7f5f8c107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.924+0000 7f5f90c00640 1 -- 172.21.15.97:0/2385618144 shutdown_connections 2024-03-26T19:05:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:22.924+0000 7f5f90c00640 1 -- 172.21.15.97:0/2385618144 wait complete. 2024-03-26T19:05:24.515 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:24 smithi105 ceph-mon[36270]: pgmap v584: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-03-26T19:05:24.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:24 smithi097 ceph-mon[31955]: pgmap v584: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-03-26T19:05:24.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:24 smithi144 ceph-mon[36053]: pgmap v584: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-03-26T19:05:25.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 360.0 s, 2225.2 tps, lat 45.052 ms stddev 52.142 2024-03-26T19:05:26.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:26 smithi097 ceph-mon[31955]: pgmap v585: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 690 op/s 2024-03-26T19:05:26.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:26 smithi144 ceph-mon[36053]: pgmap v585: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 690 op/s 2024-03-26T19:05:26.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:26 smithi105 ceph-mon[36270]: pgmap v585: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 690 op/s 2024-03-26T19:05:28.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:28 smithi097 ceph-mon[31955]: pgmap v586: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 612 op/s 2024-03-26T19:05:28.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:28 smithi144 ceph-mon[36053]: pgmap v586: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 612 op/s 2024-03-26T19:05:28.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:28 smithi105 ceph-mon[36270]: pgmap v586: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 612 op/s 2024-03-26T19:05:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 365.0 s, 1855.2 tps, lat 53.474 ms stddev 63.055 2024-03-26T19:05:30.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:30 smithi097 ceph-mon[31955]: pgmap v587: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 659 op/s 2024-03-26T19:05:30.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:30 smithi144 ceph-mon[36053]: pgmap v587: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 659 op/s 2024-03-26T19:05:30.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:30 smithi105 ceph-mon[36270]: pgmap v587: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 659 op/s 2024-03-26T19:05:32.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:32 smithi097 ceph-mon[31955]: pgmap v588: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-03-26T19:05:32.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:32 smithi144 ceph-mon[36053]: pgmap v588: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-03-26T19:05:32.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:32 smithi105 ceph-mon[36270]: pgmap v588: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-03-26T19:05:34.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:34 smithi097 ceph-mon[31955]: pgmap v589: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 631 op/s 2024-03-26T19:05:34.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:34 smithi144 ceph-mon[36053]: pgmap v589: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 631 op/s 2024-03-26T19:05:34.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:34 smithi105 ceph-mon[36270]: pgmap v589: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 631 op/s 2024-03-26T19:05:35.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 370.0 s, 2010.0 tps, lat 49.937 ms stddev 62.311 2024-03-26T19:05:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.119+0000 7f868e400640 1 -- 172.21.15.97:0/498850979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8688102390 msgr2=0x7f8688108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.119+0000 7f868e400640 1 --2- 172.21.15.97:0/498850979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8688102390 0x7f8688108c20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8680009900 tx=0x7f868002bc40 comp rx=0 tx=0).stop 2024-03-26T19:05:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.119+0000 7f868e400640 1 -- 172.21.15.97:0/498850979 shutdown_connections 2024-03-26T19:05:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.119+0000 7f868e400640 1 --2- 172.21.15.97:0/498850979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8688102390 0x7f8688108c20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.119+0000 7f868e400640 1 --2- 172.21.15.97:0/498850979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86881019d0 0x7f8688101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.119+0000 7f868e400640 1 --2- 172.21.15.97:0/498850979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86881007e0 0x7f8688100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.119+0000 7f868e400640 1 -- 172.21.15.97:0/498850979 >> 172.21.15.97:0/498850979 conn(0x7f86880fbd50 msgr2=0x7f86880fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.119+0000 7f868e400640 1 -- 172.21.15.97:0/498850979 shutdown_connections 2024-03-26T19:05:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.119+0000 7f868e400640 1 -- 172.21.15.97:0/498850979 wait complete. 2024-03-26T19:05:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.121+0000 7f868e400640 1 Processor -- start 2024-03-26T19:05:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.121+0000 7f868e400640 1 -- start start 2024-03-26T19:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f868e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86881007e0 0x7f8688199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f868e400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86881019d0 0x7f868819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f868e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688102390 0x7f86881a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f868e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f868810f780 con 0x7f8688102390 2024-03-26T19:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f868e400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f868810f600 con 0x7f86881019d0 2024-03-26T19:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f868e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f868810f900 con 0x7f86881007e0 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f8687400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86881007e0 0x7f8688199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f8687400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86881007e0 0x7f8688199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:47900/0 (socket says 172.21.15.97:47900) 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f8687400640 1 -- 172.21.15.97:0/388631881 learned_addr learned my addr 172.21.15.97:0/388631881 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f8686a00640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86881019d0 0x7f868819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f8687e00640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688102390 0x7f86881a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f8687400640 1 -- 172.21.15.97:0/388631881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86881019d0 msgr2=0x7f868819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f8687400640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86881019d0 0x7f868819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f8687400640 1 -- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688102390 msgr2=0x7f86881a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f8687400640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688102390 0x7f86881a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.122+0000 7f8687400640 1 -- 172.21.15.97:0/388631881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86881a1940 con 0x7f86881007e0 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.123+0000 7f8687e00640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688102390 0x7f86881a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.123+0000 7f8686a00640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86881019d0 0x7f868819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.123+0000 7f8687400640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86881007e0 0x7f8688199d30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8678009d20 tx=0x7f867800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.123+0000 7f867fe00640 1 -- 172.21.15.97:0/388631881 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678010630 con 0x7f86881007e0 2024-03-26T19:05:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.123+0000 7f867fe00640 1 -- 172.21.15.97:0/388631881 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8678010cb0 con 0x7f86881007e0 2024-03-26T19:05:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.123+0000 7f867fe00640 1 -- 172.21.15.97:0/388631881 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678018de0 con 0x7f86881007e0 2024-03-26T19:05:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.123+0000 7f868e400640 1 -- 172.21.15.97:0/388631881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86881a1c30 con 0x7f86881007e0 2024-03-26T19:05:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.123+0000 7f868e400640 1 -- 172.21.15.97:0/388631881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86881a2130 con 0x7f86881007e0 2024-03-26T19:05:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.127+0000 7f86911b2740 1 Processor -- start 2024-03-26T19:05:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.128+0000 7f86911b2740 1 -- start start 2024-03-26T19:05:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.128+0000 7f867fe00640 1 -- 172.21.15.97:0/388631881 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f867801f020 con 0x7f86881007e0 2024-03-26T19:05:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.128+0000 7f86911b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x5615937149d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.128+0000 7f86911b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x561593719aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.128+0000 7f86911b2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x56159371c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.128+0000 7f86911b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561593670e80 con 0x5615937145b0 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.128+0000 7f86911b2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561593619660 con 0x561593714fa0 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.128+0000 7f86911b2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56159366ce30 con 0x56159371a110 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.129+0000 7f8687400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x561593719aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.129+0000 7f8687e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x5615937149d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.129+0000 7f8687400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x561593719aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60776/0 (socket says 172.21.15.97:60776) 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.129+0000 7f8687400640 1 -- 172.21.15.97:0/4002330083 learned_addr learned my addr 172.21.15.97:0/4002330083 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.129+0000 7f8687400640 1 -- 172.21.15.97:0/4002330083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 msgr2=0x56159371c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.129+0000 7f8687400640 1 --2- 172.21.15.97:0/4002330083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x56159371c5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.129+0000 7f8687400640 1 -- 172.21.15.97:0/4002330083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 msgr2=0x5615937149d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.129+0000 7f8687400640 1 --2- 172.21.15.97:0/4002330083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x5615937149d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.129+0000 7f8687400640 1 -- 172.21.15.97:0/4002330083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561593385210 con 0x561593714fa0 2024-03-26T19:05:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.131+0000 7f867fe00640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f866c0a3040 0x7f866c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.131+0000 7f8687400640 1 --2- 172.21.15.97:0/4002330083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x561593719aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f86780d5ba0 tx=0x7f86780d5c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f923c863d3e6ea99 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.131+0000 7f8687e00640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f866c0a3040 0x7f866c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.131+0000 7f867fe00640 1 -- 172.21.15.97:0/388631881 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f86780cfa10 con 0x7f86881007e0 2024-03-26T19:05:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.133+0000 7f867cc00640 1 -- 172.21.15.97:0/4002330083 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678098320 con 0x561593714fa0 2024-03-26T19:05:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.133+0000 7f867cc00640 1 -- 172.21.15.97:0/4002330083 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867809c810 con 0x561593714fa0 2024-03-26T19:05:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.133+0000 7f867cc00640 1 -- 172.21.15.97:0/4002330083 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867809a250 con 0x561593714fa0 2024-03-26T19:05:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.133+0000 7f8687e00640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f866c0a3040 0x7f866c0a5500 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8680036040 tx=0x7f868002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.135+0000 7f86911b2740 1 -- 172.21.15.97:0/4002330083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 msgr2=0x561593719aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.135+0000 7f86911b2740 1 --2- 172.21.15.97:0/4002330083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x561593719aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f86780d5ba0 tx=0x7f86780d5c90 comp rx=0 tx=0).stop 2024-03-26T19:05:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.135+0000 7f86911b2740 1 -- 172.21.15.97:0/4002330083 shutdown_connections 2024-03-26T19:05:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.135+0000 7f86911b2740 1 --2- 172.21.15.97:0/4002330083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x56159371c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.135+0000 7f86911b2740 1 --2- 172.21.15.97:0/4002330083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x561593719aa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.135+0000 7f86911b2740 1 --2- 172.21.15.97:0/4002330083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x5615937149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.135+0000 7f86911b2740 1 -- 172.21.15.97:0/4002330083 >> 172.21.15.97:0/4002330083 conn(0x561593707310 msgr2=0x561593707710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.136+0000 7f86911b2740 1 -- 172.21.15.97:0/4002330083 shutdown_connections 2024-03-26T19:05:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.136+0000 7f86911b2740 1 -- 172.21.15.97:0/4002330083 wait complete. 2024-03-26T19:05:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.137+0000 7f86911b2740 1 -- start start 2024-03-26T19:05:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.137+0000 7f86911b2740 1 Processor -- start 2024-03-26T19:05:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.138+0000 7f86911b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x56159371e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.138+0000 7f86911b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x56159371eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.138+0000 7f86911b2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x561593725b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.138+0000 7f86911b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56159371d0c0 con 0x5615937145b0 2024-03-26T19:05:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.138+0000 7f86911b2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56159362cac0 con 0x561593714fa0 2024-03-26T19:05:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.139+0000 7f86911b2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56159371d240 con 0x56159371a110 2024-03-26T19:05:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.139+0000 7f8687e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x561593725b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.140+0000 7f8687e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x561593725b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:47918/0 (socket says 172.21.15.97:47918) 2024-03-26T19:05:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.140+0000 7f8687e00640 1 -- 172.21.15.97:0/3354389322 learned_addr learned my addr 172.21.15.97:0/3354389322 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.139+0000 7f8687400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x56159371e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.139+0000 7f8686a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x56159371eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.140+0000 7f86911b2740 10 client.0 _ll_register_callbacks cb 0x56159362c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:05:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.141+0000 7f8687e00640 1 -- 172.21.15.97:0/3354389322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 msgr2=0x56159371eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.142+0000 7f8687e00640 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x56159371eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.142+0000 7f8687e00640 1 -- 172.21.15.97:0/3354389322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 msgr2=0x56159371e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.142+0000 7f8687e00640 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x56159371e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.142+0000 7f8687e00640 1 -- 172.21.15.97:0/3354389322 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561593451620 con 0x56159371a110 2024-03-26T19:05:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.142+0000 7f8686a00640 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x56159371eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:05:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.142+0000 7f8687400640 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x56159371e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:05:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.144+0000 7f8687e00640 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x561593725b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f868002c250 tx=0x7f8680009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38b530181bfcec67 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.144+0000 7f8671600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:05:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.145+0000 7f8671600640 1 -- 172.21.15.97:0/3354389322 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868004f020 con 0x56159371a110 2024-03-26T19:05:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.145+0000 7f8671600640 1 -- 172.21.15.97:0/3354389322 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f86800046c0 con 0x56159371a110 2024-03-26T19:05:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.145+0000 7f86911b2740 1 -- 172.21.15.97:0/3354389322 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f867809c810 con 0x56159371a110 2024-03-26T19:05:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.146+0000 7f8671600640 1 -- 172.21.15.97:0/3354389322 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8680005650 con 0x56159371a110 2024-03-26T19:05:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.146+0000 7f86911b2740 10 client.25726 fetch_fsmap learned FSMap version 27 2024-03-26T19:05:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.146+0000 7f86911b2740 1 -- 172.21.15.97:0/3354389322 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561593739f80 con 0x56159371a110 2024-03-26T19:05:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.147+0000 7f8671600640 1 -- 172.21.15.97:0/3354389322 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867809c810 con 0x56159371a110 2024-03-26T19:05:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.147+0000 7f8671600640 1 -- 172.21.15.97:0/3354389322 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8680003ce0 con 0x56159371a110 2024-03-26T19:05:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.148+0000 7f86911b2740 10 client.25726 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:05:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.148+0000 7f86911b2740 10 client.25726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:05:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.148+0000 7f86911b2740 20 client.25726 populate_metadata read hostname 'smithi097' 2024-03-26T19:05:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.148+0000 7f86911b2740 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56159373b660 0x56159373dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.148+0000 7f86911b2740 4 client.25726 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:05:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.148+0000 7f86911b2740 1 -- 172.21.15.97:0/3354389322 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56159367e3c0 con 0x56159373b660 2024-03-26T19:05:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.148+0000 7f8687400640 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56159373b660 0x56159373dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.149+0000 7f8687400640 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56159373b660 0x56159373dab0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95d97042954edff9 server_cookie=ab4243c6e9d92962 in_seq=0 out_seq=0 2024-03-26T19:05:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.149+0000 7f8671600640 10 client.25726 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.150+0000 7f8671600640 1 -- 172.21.15.97:0/3354389322 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56159367e3c0 con 0x56159373b660 2024-03-26T19:05:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.150+0000 7f8671600640 10 client.25726 handle_command_reply: tid=0 2024-03-26T19:05:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.150+0000 7f86911b2740 1 client.25726 shutdown 2024-03-26T19:05:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.150+0000 7f86911b2740 1 -- 172.21.15.97:0/3354389322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 msgr2=0x561593725b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.150+0000 7f86911b2740 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x561593725b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f868002c250 tx=0x7f8680009260 comp rx=0 tx=0).stop 2024-03-26T19:05:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.151+0000 7f86911b2740 1 -- 172.21.15.97:0/3354389322 shutdown_connections 2024-03-26T19:05:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.151+0000 7f86911b2740 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56159373b660 0x56159373dab0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.151+0000 7f86911b2740 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x561593725b90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.151+0000 7f86911b2740 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561593714fa0 0x56159371eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.151+0000 7f86911b2740 1 --2- 172.21.15.97:0/3354389322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x56159371e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.151+0000 7f86911b2740 1 -- 172.21.15.97:0/3354389322 >> 172.21.15.97:0/3354389322 conn(0x561593707310 msgr2=0x561593706540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.151+0000 7f8671600640 0 client.25726 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.151+0000 7f86911b2740 1 -- 172.21.15.97:0/3354389322 shutdown_connections 2024-03-26T19:05:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.151+0000 7f86911b2740 1 -- 172.21.15.97:0/3354389322 wait complete. 2024-03-26T19:05:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.152+0000 7f86911b2740 20 client.25726 trim_cache size 0 max 16384 2024-03-26T19:05:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.186+0000 7f86911b2740 1 Processor -- start 2024-03-26T19:05:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.186+0000 7f86911b2740 1 -- start start 2024-03-26T19:05:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f86911b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x56159370a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f86911b2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561593714fa0 0x561593725b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f86911b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56159371a110 0x56159370aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f86911b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f868806fbb0 con 0x5615937145b0 2024-03-26T19:05:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f86911b2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86880716f0 con 0x56159371a110 2024-03-26T19:05:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f86911b2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8688070280 con 0x561593714fa0 2024-03-26T19:05:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f8687e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56159371a110 0x56159370aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f8687400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x56159370a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f8687e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56159371a110 0x56159370aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60794/0 (socket says 172.21.15.97:60794) 2024-03-26T19:05:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f8687e00640 1 -- 172.21.15.97:0/227054565 learned_addr learned my addr 172.21.15.97:0/227054565 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.187+0000 7f8686a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561593714fa0 0x561593725b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.188+0000 7f8686a00640 1 -- 172.21.15.97:0/227054565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56159371a110 msgr2=0x56159370aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.188+0000 7f8686a00640 1 --2- 172.21.15.97:0/227054565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56159371a110 0x56159370aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.188+0000 7f8686a00640 1 -- 172.21.15.97:0/227054565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 msgr2=0x56159370a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.188+0000 7f8686a00640 1 --2- 172.21.15.97:0/227054565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x56159370a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.188+0000 7f8686a00640 1 -- 172.21.15.97:0/227054565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561593708080 con 0x561593714fa0 2024-03-26T19:05:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.189+0000 7f8686a00640 1 --2- 172.21.15.97:0/227054565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561593714fa0 0x561593725b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8674009f40 tx=0x7f86740128e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc4a607e1ce3c076 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.189+0000 7f8673e00640 1 -- 172.21.15.97:0/227054565 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674004030 con 0x561593714fa0 2024-03-26T19:05:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.189+0000 7f8673e00640 1 -- 172.21.15.97:0/227054565 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86740041d0 con 0x561593714fa0 2024-03-26T19:05:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.190+0000 7f8673e00640 1 -- 172.21.15.97:0/227054565 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674004030 con 0x561593714fa0 2024-03-26T19:05:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.191+0000 7f86911b2740 1 -- 172.21.15.97:0/227054565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561593714fa0 msgr2=0x561593725b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.191+0000 7f86911b2740 1 --2- 172.21.15.97:0/227054565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561593714fa0 0x561593725b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8674009f40 tx=0x7f86740128e0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.191+0000 7f86911b2740 1 -- 172.21.15.97:0/227054565 shutdown_connections 2024-03-26T19:05:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.191+0000 7f86911b2740 1 --2- 172.21.15.97:0/227054565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56159371a110 0x56159370aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.191+0000 7f86911b2740 1 --2- 172.21.15.97:0/227054565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561593714fa0 0x561593725b50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.191+0000 7f86911b2740 1 --2- 172.21.15.97:0/227054565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615937145b0 0x56159370a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.191+0000 7f86911b2740 1 -- 172.21.15.97:0/227054565 >> 172.21.15.97:0/227054565 conn(0x561593707310 msgr2=0x56159371e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.192+0000 7f86911b2740 1 -- 172.21.15.97:0/227054565 shutdown_connections 2024-03-26T19:05:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.192+0000 7f86911b2740 1 -- 172.21.15.97:0/227054565 wait complete. 2024-03-26T19:05:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.192+0000 7f86911b2740 1 -- start start 2024-03-26T19:05:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.193+0000 7f86911b2740 1 Processor -- start 2024-03-26T19:05:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.194+0000 7f86911b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5615937145b0 0x561593722bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.194+0000 7f86911b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561593714fa0 0x5615937230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.195+0000 7f86911b2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x561593724100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.195+0000 7f86911b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8688071020 con 0x561593714fa0 2024-03-26T19:05:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.195+0000 7f86911b2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8688070950 con 0x5615937145b0 2024-03-26T19:05:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.195+0000 7f86911b2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f868806db90 con 0x56159371a110 2024-03-26T19:05:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.195+0000 7f86911b2740 10 client.0 _ll_register_callbacks cb 0x7f8674005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:05:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.195+0000 7f8687400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5615937145b0 0x561593722bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.195+0000 7f8687400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5615937145b0 0x561593722bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60806/0 (socket says 172.21.15.97:60806) 2024-03-26T19:05:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.195+0000 7f8687400640 1 -- 172.21.15.97:0/2500108216 learned_addr learned my addr 172.21.15.97:0/2500108216 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.195+0000 7f8687e00640 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x561593724100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.195+0000 7f8686a00640 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561593714fa0 0x5615937230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.196+0000 7f8687400640 1 -- 172.21.15.97:0/2500108216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 msgr2=0x561593724100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.196+0000 7f8687400640 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x561593724100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.196+0000 7f8687400640 1 -- 172.21.15.97:0/2500108216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561593714fa0 msgr2=0x5615937230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.196+0000 7f8687400640 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561593714fa0 0x5615937230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.196+0000 7f8687400640 1 -- 172.21.15.97:0/2500108216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615937246f0 con 0x5615937145b0 2024-03-26T19:05:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.196+0000 7f8687400640 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5615937145b0 0x561593722bb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8678005c40 tx=0x7f8678018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc205da7717f6ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.196+0000 7f8671600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:05:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.197+0000 7f8671600640 1 -- 172.21.15.97:0/2500108216 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678010e60 con 0x5615937145b0 2024-03-26T19:05:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.197+0000 7f8671600640 1 -- 172.21.15.97:0/2500108216 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f86780d52f0 con 0x5615937145b0 2024-03-26T19:05:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.197+0000 7f86911b2740 1 -- 172.21.15.97:0/2500108216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86740041d0 con 0x5615937145b0 2024-03-26T19:05:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.198+0000 7f8671600640 1 -- 172.21.15.97:0/2500108216 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f867809ba70 con 0x5615937145b0 2024-03-26T19:05:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.198+0000 7f86911b2740 10 client.25778 fetch_fsmap learned FSMap version 27 2024-03-26T19:05:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.198+0000 7f86911b2740 1 -- 172.21.15.97:0/2500108216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561593725a00 con 0x5615937145b0 2024-03-26T19:05:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.198+0000 7f8671600640 1 -- 172.21.15.97:0/2500108216 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86740041d0 con 0x5615937145b0 2024-03-26T19:05:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.198+0000 7f8671600640 1 -- 172.21.15.97:0/2500108216 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f86780cd560 con 0x5615937145b0 2024-03-26T19:05:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.199+0000 7f86911b2740 10 client.25778 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:05:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.199+0000 7f86911b2740 10 client.25778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:05:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.199+0000 7f86911b2740 20 client.25778 populate_metadata read hostname 'smithi097' 2024-03-26T19:05:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.199+0000 7f86911b2740 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5615937a19f0 0x5615937a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.199+0000 7f86911b2740 4 client.25778 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:05:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.199+0000 7f86911b2740 1 -- 172.21.15.97:0/2500108216 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561593678710 con 0x5615937a19f0 2024-03-26T19:05:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.199+0000 7f8686a00640 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5615937a19f0 0x5615937a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.200+0000 7f8671600640 10 client.25778 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.200+0000 7f8686a00640 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5615937a19f0 0x5615937a3de0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9be5c19da2803d21 server_cookie=c83c4de63d4e945d in_seq=0 out_seq=0 2024-03-26T19:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.229+0000 7f8671600640 1 -- 172.21.15.97:0/2500108216 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561593678710 con 0x5615937a19f0 2024-03-26T19:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.229+0000 7f8671600640 10 client.25778 handle_command_reply: tid=0 2024-03-26T19:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.229+0000 7f86911b2740 1 client.25778 shutdown 2024-03-26T19:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.230+0000 7f86911b2740 1 -- 172.21.15.97:0/2500108216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5615937145b0 msgr2=0x561593722bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.230+0000 7f86911b2740 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5615937145b0 0x561593722bb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8678005c40 tx=0x7f8678018420 comp rx=0 tx=0).stop 2024-03-26T19:05:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.230+0000 7f86911b2740 1 -- 172.21.15.97:0/2500108216 shutdown_connections 2024-03-26T19:05:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.230+0000 7f86911b2740 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5615937a19f0 0x5615937a3de0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.230+0000 7f86911b2740 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56159371a110 0x561593724100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.230+0000 7f86911b2740 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561593714fa0 0x5615937230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.230+0000 7f86911b2740 1 --2- 172.21.15.97:0/2500108216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5615937145b0 0x561593722bb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.230+0000 7f86911b2740 1 -- 172.21.15.97:0/2500108216 >> 172.21.15.97:0/2500108216 conn(0x561593707310 msgr2=0x56159370a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.231+0000 7f8671600640 0 client.25778 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.231+0000 7f86911b2740 1 -- 172.21.15.97:0/2500108216 shutdown_connections 2024-03-26T19:05:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.231+0000 7f86911b2740 1 -- 172.21.15.97:0/2500108216 wait complete. 2024-03-26T19:05:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.231+0000 7f86911b2740 20 client.25778 trim_cache size 0 max 16384 2024-03-26T19:05:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:05:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.233+0000 7f868e400640 1 -- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f866c0a3040 msgr2=0x7f866c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.233+0000 7f868e400640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f866c0a3040 0x7f866c0a5500 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8680036040 tx=0x7f868002cf90 comp rx=0 tx=0).stop 2024-03-26T19:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.233+0000 7f868e400640 1 -- 172.21.15.97:0/388631881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86881007e0 msgr2=0x7f8688199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.233+0000 7f868e400640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86881007e0 0x7f8688199d30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8678009d20 tx=0x7f867800c6a0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.233+0000 7f868e400640 1 -- 172.21.15.97:0/388631881 shutdown_connections 2024-03-26T19:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.233+0000 7f868e400640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f866c0a3040 0x7f866c0a5500 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.233+0000 7f868e400640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688102390 0x7f86881a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.233+0000 7f868e400640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86881019d0 0x7f868819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.233+0000 7f868e400640 1 --2- 172.21.15.97:0/388631881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86881007e0 0x7f8688199d30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.234+0000 7f868e400640 1 -- 172.21.15.97:0/388631881 >> 172.21.15.97:0/388631881 conn(0x7f86880fbd50 msgr2=0x7f8688104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.234+0000 7f868e400640 1 -- 172.21.15.97:0/388631881 shutdown_connections 2024-03-26T19:05:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:36.234+0000 7f868e400640 1 -- 172.21.15.97:0/388631881 wait complete. 2024-03-26T19:05:36.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:36 smithi097 ceph-mon[31955]: pgmap v590: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 728 op/s 2024-03-26T19:05:36.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:36 smithi144 ceph-mon[36053]: pgmap v590: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 728 op/s 2024-03-26T19:05:36.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:36 smithi105 ceph-mon[36270]: pgmap v590: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 728 op/s 2024-03-26T19:05:38.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:38 smithi097 ceph-mon[31955]: pgmap v591: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.3 MiB/s wr, 630 op/s 2024-03-26T19:05:38.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:38 smithi144 ceph-mon[36053]: pgmap v591: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.3 MiB/s wr, 630 op/s 2024-03-26T19:05:38.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:38 smithi105 ceph-mon[36270]: pgmap v591: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.3 MiB/s wr, 630 op/s 2024-03-26T19:05:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 375.0 s, 2098.6 tps, lat 47.478 ms stddev 50.453 2024-03-26T19:05:40.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:40 smithi097 ceph-mon[31955]: pgmap v592: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 684 op/s 2024-03-26T19:05:40.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:40 smithi144 ceph-mon[36053]: pgmap v592: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 684 op/s 2024-03-26T19:05:40.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:40 smithi105 ceph-mon[36270]: pgmap v592: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 684 op/s 2024-03-26T19:05:42.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:42 smithi097 ceph-mon[31955]: pgmap v593: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.1 MiB/s wr, 730 op/s 2024-03-26T19:05:42.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:42 smithi144 ceph-mon[36053]: pgmap v593: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.1 MiB/s wr, 730 op/s 2024-03-26T19:05:42.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:42 smithi105 ceph-mon[36270]: pgmap v593: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.1 MiB/s wr, 730 op/s 2024-03-26T19:05:44.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:44 smithi097 ceph-mon[31955]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 MiB/s wr, 655 op/s 2024-03-26T19:05:44.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:44 smithi144 ceph-mon[36053]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 MiB/s wr, 655 op/s 2024-03-26T19:05:44.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:44 smithi105 ceph-mon[36270]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 MiB/s wr, 655 op/s 2024-03-26T19:05:45.026 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 380.0 s, 1515.6 tps, lat 66.127 ms stddev 80.017 2024-03-26T19:05:46.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:46 smithi097 ceph-mon[31955]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 737 op/s 2024-03-26T19:05:46.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:46 smithi144 ceph-mon[36053]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 737 op/s 2024-03-26T19:05:46.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:46 smithi105 ceph-mon[36270]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 737 op/s 2024-03-26T19:05:48.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:48 smithi097 ceph-mon[31955]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.5 MiB/s wr, 603 op/s 2024-03-26T19:05:48.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:48 smithi144 ceph-mon[36053]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.5 MiB/s wr, 603 op/s 2024-03-26T19:05:48.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:48 smithi105 ceph-mon[36270]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.5 MiB/s wr, 603 op/s 2024-03-26T19:05:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.440+0000 7fe4bf000640 1 -- 172.21.15.97:0/2522776219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b8074320 msgr2=0x7fe4b80747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.440+0000 7fe4bf000640 1 --2- 172.21.15.97:0/2522776219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b8074320 0x7fe4b80747a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009930 tx=0x7fe4a402bc30 comp rx=0 tx=0).stop 2024-03-26T19:05:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.440+0000 7fe4bf000640 1 -- 172.21.15.97:0/2522776219 shutdown_connections 2024-03-26T19:05:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.440+0000 7fe4bf000640 1 --2- 172.21.15.97:0/2522776219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b810dec0 0x7fe4b81102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.440+0000 7fe4bf000640 1 --2- 172.21.15.97:0/2522776219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b8074320 0x7fe4b80747a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.440+0000 7fe4bf000640 1 --2- 172.21.15.97:0/2522776219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8075900 0x7fe4b8073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.440+0000 7fe4bf000640 1 -- 172.21.15.97:0/2522776219 >> 172.21.15.97:0/2522776219 conn(0x7fe4b80fce80 msgr2=0x7fe4b80ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.440+0000 7fe4bf000640 1 -- 172.21.15.97:0/2522776219 shutdown_connections 2024-03-26T19:05:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.440+0000 7fe4bf000640 1 -- 172.21.15.97:0/2522776219 wait complete. 2024-03-26T19:05:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.441+0000 7fe4bf000640 1 Processor -- start 2024-03-26T19:05:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4bf000640 1 -- start start 2024-03-26T19:05:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4bf000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b8074320 0x7fe4b819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4bf000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8075900 0x7fe4b819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4bf000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b810dec0 0x7fe4b81a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4bf000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8113bc0 con 0x7fe4b8074320 2024-03-26T19:05:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4bf000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8113a40 con 0x7fe4b810dec0 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4bf000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8113d40 con 0x7fe4b8075900 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8075900 0x7fe4b819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8075900 0x7fe4b819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:43412/0 (socket says 172.21.15.97:43412) 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4b7400640 1 -- 172.21.15.97:0/440208810 learned_addr learned my addr 172.21.15.97:0/440208810 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4b7e00640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b8074320 0x7fe4b819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.442+0000 7fe4bd000640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b810dec0 0x7fe4b81a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.443+0000 7fe4b7400640 1 -- 172.21.15.97:0/440208810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b810dec0 msgr2=0x7fe4b81a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.443+0000 7fe4b7400640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b810dec0 0x7fe4b81a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.443+0000 7fe4b7400640 1 -- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b8074320 msgr2=0x7fe4b819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.443+0000 7fe4b7400640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b8074320 0x7fe4b819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.443+0000 7fe4b7400640 1 -- 172.21.15.97:0/440208810 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b81a5dc0 con 0x7fe4b8075900 2024-03-26T19:05:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.443+0000 7fe4bd000640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b810dec0 0x7fe4b81a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:05:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.443+0000 7fe4b7e00640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b8074320 0x7fe4b819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:05:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.444+0000 7fe4b7400640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8075900 0x7fe4b819e6e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4005ab0 tx=0x7fe4a4005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.444+0000 7fe4b4c00640 1 -- 172.21.15.97:0/440208810 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4041070 con 0x7fe4b8075900 2024-03-26T19:05:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.444+0000 7fe4bf000640 1 -- 172.21.15.97:0/440208810 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b81a6050 con 0x7fe4b8075900 2024-03-26T19:05:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.444+0000 7fe4b4c00640 1 -- 172.21.15.97:0/440208810 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a40054e0 con 0x7fe4b8075900 2024-03-26T19:05:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.444+0000 7fe4b4c00640 1 -- 172.21.15.97:0/440208810 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4034440 con 0x7fe4b8075900 2024-03-26T19:05:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.444+0000 7fe4bf000640 1 -- 172.21.15.97:0/440208810 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b81a6590 con 0x7fe4b8075900 2024-03-26T19:05:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.449+0000 7fe4b4c00640 1 -- 172.21.15.97:0/440208810 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fe4a4044070 con 0x7fe4b8075900 2024-03-26T19:05:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.449+0000 7fe4c1c13740 1 Processor -- start 2024-03-26T19:05:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.452+0000 7fe4b4c00640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe49c0a30c0 0x7fe49c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.452+0000 7fe4b4c00640 1 -- 172.21.15.97:0/440208810 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe4a40f0fe0 con 0x7fe4b8075900 2024-03-26T19:05:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.453+0000 7fe4c1c13740 1 -- start start 2024-03-26T19:05:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.453+0000 7fe4bd000640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe49c0a30c0 0x7fe49c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.453+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x5581711761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.453+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 0x55817117b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.453+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55817117b900 0x55817117dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.453+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5581710dae80 con 0x55817117b900 2024-03-26T19:05:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.453+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558171083660 con 0x558171175da0 2024-03-26T19:05:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.453+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5581710d6e30 con 0x558171176790 2024-03-26T19:05:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.454+0000 7fe4b7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x5581711761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.454+0000 7fe4b7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x5581711761c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54902/0 (socket says 172.21.15.97:54902) 2024-03-26T19:05:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.454+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 0x55817117b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.454+0000 7fe4b7e00640 1 -- 172.21.15.97:0/3858489711 learned_addr learned my addr 172.21.15.97:0/3858489711 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.455+0000 7fe4bd000640 1 --2- 172.21.15.97:0/3858489711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55817117b900 0x55817117dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.455+0000 7fe4bd000640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe49c0a30c0 0x7fe49c0a5580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000a730 tx=0x7fe4b000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.455+0000 7fe4b7400640 1 -- 172.21.15.97:0/3858489711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 msgr2=0x5581711761c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.455+0000 7fe4b7400640 1 --2- 172.21.15.97:0/3858489711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x5581711761c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.455+0000 7fe4b7400640 1 -- 172.21.15.97:0/3858489711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55817117b900 msgr2=0x55817117dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.455+0000 7fe4b7400640 1 --2- 172.21.15.97:0/3858489711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55817117b900 0x55817117dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.455+0000 7fe4b7400640 1 -- 172.21.15.97:0/3858489711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558170def210 con 0x558171176790 2024-03-26T19:05:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.455+0000 7fe4b7400640 1 --2- 172.21.15.97:0/3858489711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 0x55817117b290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4006010 tx=0x7fe4a4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a36f3eaa2b3bcff9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.456+0000 7fe4ad600640 1 -- 172.21.15.97:0/3858489711 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a404b040 con 0x558171176790 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.456+0000 7fe4ad600640 1 -- 172.21.15.97:0/3858489711 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4004c80 con 0x558171176790 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.456+0000 7fe4ad600640 1 -- 172.21.15.97:0/3858489711 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40a8070 con 0x558171176790 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.456+0000 7fe4c1c13740 1 -- 172.21.15.97:0/3858489711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 msgr2=0x55817117b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.456+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/3858489711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 0x55817117b290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4006010 tx=0x7fe4a4009260 comp rx=0 tx=0).stop 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.457+0000 7fe4c1c13740 1 -- 172.21.15.97:0/3858489711 shutdown_connections 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.457+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/3858489711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55817117b900 0x55817117dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.457+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/3858489711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 0x55817117b290 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.457+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/3858489711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x5581711761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.457+0000 7fe4c1c13740 1 -- 172.21.15.97:0/3858489711 >> 172.21.15.97:0/3858489711 conn(0x558171171310 msgr2=0x558171171710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.457+0000 7fe4c1c13740 1 -- 172.21.15.97:0/3858489711 shutdown_connections 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.457+0000 7fe4c1c13740 1 -- 172.21.15.97:0/3858489711 wait complete. 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.457+0000 7fe4c1c13740 1 -- start start 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.457+0000 7fe4c1c13740 1 Processor -- start 2024-03-26T19:05:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.458+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 0x558171190b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.458+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 0x5581711910d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.458+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817117b900 0x558171198110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.458+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55817118f680 con 0x558171175da0 2024-03-26T19:05:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.458+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558171096ac0 con 0x55817117b900 2024-03-26T19:05:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.458+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55817118f800 con 0x558171176790 2024-03-26T19:05:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.458+0000 7fe4c1c13740 10 client.0 _ll_register_callbacks cb 0x558171096490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:05:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.458+0000 7fe4b7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 0x558171190b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.458+0000 7fe4b7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 0x558171190b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39094/0 (socket says 172.21.15.97:39094) 2024-03-26T19:05:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4b7e00640 1 -- 172.21.15.97:0/4149573663 learned_addr learned my addr 172.21.15.97:0/4149573663 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4b7400640 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 0x5581711910d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4bd000640 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817117b900 0x558171198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4b7400640 1 -- 172.21.15.97:0/4149573663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817117b900 msgr2=0x558171198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4b7400640 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817117b900 0x558171198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4b7400640 1 -- 172.21.15.97:0/4149573663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 msgr2=0x558171190b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4b7400640 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 0x558171190b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4b7400640 1 -- 172.21.15.97:0/4149573663 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558170ebb620 con 0x558171176790 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4b7400640 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 0x5581711910d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40bd950 tx=0x7fe4a40a82f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e263a33a6d68425 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4a2000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4a2000640 1 -- 172.21.15.97:0/4149573663 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40d5020 con 0x558171176790 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4a2000640 1 -- 172.21.15.97:0/4149573663 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe4a40c5c60 con 0x558171176790 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.459+0000 7fe4c1c13740 1 -- 172.21.15.97:0/4149573663 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a4004c80 con 0x558171176790 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.460+0000 7fe4a2000640 1 -- 172.21.15.97:0/4149573663 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4a404b030 con 0x558171176790 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.461+0000 7fe4c1c13740 10 client.25744 fetch_fsmap learned FSMap version 27 2024-03-26T19:05:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.461+0000 7fe4c1c13740 1 -- 172.21.15.97:0/4149573663 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55817118f680 con 0x558171176790 2024-03-26T19:05:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.461+0000 7fe4a2000640 1 -- 172.21.15.97:0/4149573663 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4004c80 con 0x558171176790 2024-03-26T19:05:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.461+0000 7fe4a2000640 1 -- 172.21.15.97:0/4149573663 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4a40bcbe0 con 0x558171176790 2024-03-26T19:05:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.461+0000 7fe4c1c13740 10 client.25744 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:05:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.461+0000 7fe4c1c13740 10 client.25744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:05:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.461+0000 7fe4c1c13740 20 client.25744 populate_metadata read hostname 'smithi097' 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.461+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55817119c730 0x55817119eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.461+0000 7fe4c1c13740 4 client.25744 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.461+0000 7fe4c1c13740 1 -- 172.21.15.97:0/4149573663 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581710e83c0 con 0x55817119c730 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.462+0000 7fe4b7e00640 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55817119c730 0x55817119eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.462+0000 7fe4a2000640 10 client.25744 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.462+0000 7fe4b7e00640 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55817119c730 0x55817119eb20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d11b59ab70e9857 server_cookie=69c17f3d160cb35b in_seq=0 out_seq=0 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.463+0000 7fe4a2000640 1 -- 172.21.15.97:0/4149573663 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5581710e83c0 con 0x55817119c730 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.463+0000 7fe4a2000640 10 client.25744 handle_command_reply: tid=0 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.463+0000 7fe4c1c13740 1 client.25744 shutdown 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.463+0000 7fe4c1c13740 1 -- 172.21.15.97:0/4149573663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 msgr2=0x5581711910d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.463+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 0x5581711910d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40bd950 tx=0x7fe4a40a82f0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.464+0000 7fe4c1c13740 1 -- 172.21.15.97:0/4149573663 shutdown_connections 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.464+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55817119c730 0x55817119eb20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.464+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817117b900 0x558171198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.464+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558171176790 0x5581711910d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.464+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/4149573663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 0x558171190b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.464+0000 7fe4c1c13740 1 -- 172.21.15.97:0/4149573663 >> 172.21.15.97:0/4149573663 conn(0x558171171310 msgr2=0x558171170540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.464+0000 7fe4a2000640 0 client.25744 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.464+0000 7fe4c1c13740 1 -- 172.21.15.97:0/4149573663 shutdown_connections 2024-03-26T19:05:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.464+0000 7fe4c1c13740 1 -- 172.21.15.97:0/4149573663 wait complete. 2024-03-26T19:05:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.464+0000 7fe4c1c13740 20 client.25744 trim_cache size 0 max 16384 2024-03-26T19:05:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.492+0000 7fe4c1c13740 1 Processor -- start 2024-03-26T19:05:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.492+0000 7fe4c1c13740 1 -- start start 2024-03-26T19:05:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.492+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x558171174c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.492+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171176790 0x5581711980d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.492+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 0x55817116fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.492+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81a6b10 con 0x558171176790 2024-03-26T19:05:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.492+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81a8650 con 0x558171175da0 2024-03-26T19:05:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.492+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81a71e0 con 0x55817117b900 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x558171174c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171176790 0x5581711980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x558171174c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54930/0 (socket says 172.21.15.97:54930) 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171176790 0x5581711980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39110/0 (socket says 172.21.15.97:39110) 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7e00640 1 -- 172.21.15.97:0/625950017 learned_addr learned my addr 172.21.15.97:0/625950017 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4bd000640 1 --2- 172.21.15.97:0/625950017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 0x55817116fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7400640 1 -- 172.21.15.97:0/625950017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 msgr2=0x55817116fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7400640 1 --2- 172.21.15.97:0/625950017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 0x55817116fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7400640 1 -- 172.21.15.97:0/625950017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 msgr2=0x558171174c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7400640 1 --2- 172.21.15.97:0/625950017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x558171174c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7400640 1 -- 172.21.15.97:0/625950017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55817117aae0 con 0x558171176790 2024-03-26T19:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.493+0000 7fe4b7e00640 1 --2- 172.21.15.97:0/625950017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x558171174c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.494+0000 7fe4b7400640 1 --2- 172.21.15.97:0/625950017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171176790 0x5581711980d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe4a402c710 tx=0x7fe4a4003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cc94e489549f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.494+0000 7fe4acc00640 1 -- 172.21.15.97:0/625950017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40d5070 con 0x558171176790 2024-03-26T19:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.495+0000 7fe4acc00640 1 -- 172.21.15.97:0/625950017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4004c80 con 0x558171176790 2024-03-26T19:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.495+0000 7fe4acc00640 1 -- 172.21.15.97:0/625950017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40c60f0 con 0x558171176790 2024-03-26T19:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.496+0000 7fe4c1c13740 1 -- 172.21.15.97:0/625950017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171176790 msgr2=0x5581711980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.496+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/625950017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171176790 0x5581711980d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe4a402c710 tx=0x7fe4a4003b20 comp rx=0 tx=0).stop 2024-03-26T19:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.497+0000 7fe4c1c13740 1 -- 172.21.15.97:0/625950017 shutdown_connections 2024-03-26T19:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.497+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/625950017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 0x55817116fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.497+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/625950017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171176790 0x5581711980d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.497+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/625950017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171175da0 0x558171174c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.497+0000 7fe4c1c13740 1 -- 172.21.15.97:0/625950017 >> 172.21.15.97:0/625950017 conn(0x558171171310 msgr2=0x558171174570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.497+0000 7fe4c1c13740 1 -- 172.21.15.97:0/625950017 shutdown_connections 2024-03-26T19:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.497+0000 7fe4c1c13740 1 -- 172.21.15.97:0/625950017 wait complete. 2024-03-26T19:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.498+0000 7fe4c1c13740 1 -- start start 2024-03-26T19:05:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.498+0000 7fe4c1c13740 1 Processor -- start 2024-03-26T19:05:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.499+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 0x558171195490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.499+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171176790 0x5581711959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.499+0000 7fe4c1c13740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 0x5581711969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.499+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81a7f80 con 0x558171175da0 2024-03-26T19:05:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.499+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81a78b0 con 0x558171176790 2024-03-26T19:05:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.499+0000 7fe4c1c13740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8071de0 con 0x55817117b900 2024-03-26T19:05:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.499+0000 7fe4c1c13740 10 client.0 _ll_register_callbacks cb 0x7fe4a4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.500+0000 7fe4bd000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 0x5581711969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.500+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171176790 0x5581711959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.500+0000 7fe4bd000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 0x5581711969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:43442/0 (socket says 172.21.15.97:43442) 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.500+0000 7fe4bd000640 1 -- 172.21.15.97:0/967231668 learned_addr learned my addr 172.21.15.97:0/967231668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.500+0000 7fe4b7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 0x558171195490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.501+0000 7fe4bd000640 1 -- 172.21.15.97:0/967231668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171176790 msgr2=0x5581711959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.501+0000 7fe4bd000640 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171176790 0x5581711959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.501+0000 7fe4bd000640 1 -- 172.21.15.97:0/967231668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 msgr2=0x558171195490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.501+0000 7fe4bd000640 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 0x558171195490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.501+0000 7fe4bd000640 1 -- 172.21.15.97:0/967231668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558171196fd0 con 0x55817117b900 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.501+0000 7fe4bd000640 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 0x5581711969e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000e6e0 tx=0x7fe4b0004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a48dbae41abd80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.501+0000 7fe4a2000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.501+0000 7fe4a2000640 1 -- 172.21.15.97:0/967231668 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0029070 con 0x55817117b900 2024-03-26T19:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4a2000640 1 -- 172.21.15.97:0/967231668 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe4b0023900 con 0x55817117b900 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4c1c13740 1 -- 172.21.15.97:0/967231668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a4004c80 con 0x55817117b900 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4a2000640 1 -- 172.21.15.97:0/967231668 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4b0035050 con 0x55817117b900 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4c1c13740 10 client.25750 fetch_fsmap learned FSMap version 27 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4c1c13740 1 -- 172.21.15.97:0/967231668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558171198350 con 0x55817117b900 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4a2000640 1 -- 172.21.15.97:0/967231668 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4004c80 con 0x55817117b900 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4a2000640 1 -- 172.21.15.97:0/967231668 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4b0022c50 con 0x55817117b900 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4c1c13740 10 client.25750 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4c1c13740 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4c1c13740 20 client.25750 populate_metadata read hostname 'smithi097' 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558171203110 0x558171205530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4c1c13740 4 client.25750 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.502+0000 7fe4c1c13740 1 -- 172.21.15.97:0/967231668 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581710e2710 con 0x558171203110 2024-03-26T19:05:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.503+0000 7fe4b7e00640 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558171203110 0x558171205530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:05:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.503+0000 7fe4a2000640 10 client.25750 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.503+0000 7fe4b7e00640 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558171203110 0x558171205530 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aabfed3891e1382b server_cookie=3cd0e7b57af59ca5 in_seq=0 out_seq=0 2024-03-26T19:05:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.519+0000 7fe4a2000640 1 -- 172.21.15.97:0/967231668 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581710e2710 con 0x558171203110 2024-03-26T19:05:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.519+0000 7fe4a2000640 10 client.25750 handle_command_reply: tid=0 2024-03-26T19:05:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.519+0000 7fe4c1c13740 1 client.25750 shutdown 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4c1c13740 1 -- 172.21.15.97:0/967231668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 msgr2=0x5581711969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 0x5581711969e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000e6e0 tx=0x7fe4b0004b50 comp rx=0 tx=0).stop 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4c1c13740 1 -- 172.21.15.97:0/967231668 shutdown_connections 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558171203110 0x558171205530 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817117b900 0x5581711969e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558171176790 0x5581711959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4c1c13740 1 --2- 172.21.15.97:0/967231668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558171175da0 0x558171195490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4a2000640 0 client.25750 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4c1c13740 1 -- 172.21.15.97:0/967231668 >> 172.21.15.97:0/967231668 conn(0x558171171310 msgr2=0x558171174e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4c1c13740 1 -- 172.21.15.97:0/967231668 shutdown_connections 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.520+0000 7fe4c1c13740 1 -- 172.21.15.97:0/967231668 wait complete. 2024-03-26T19:05:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.521+0000 7fe4c1c13740 20 client.25750 trim_cache size 0 max 16384 2024-03-26T19:05:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:05:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:05:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:05:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 -- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe49c0a30c0 msgr2=0x7fe49c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe49c0a30c0 0x7fe49c0a5580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000a730 tx=0x7fe4b000d040 comp rx=0 tx=0).stop 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 -- 172.21.15.97:0/440208810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8075900 msgr2=0x7fe4b819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8075900 0x7fe4b819e6e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4005ab0 tx=0x7fe4a4005dc0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 -- 172.21.15.97:0/440208810 shutdown_connections 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe49c0a30c0 0x7fe49c0a5580 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b810dec0 0x7fe4b81a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8075900 0x7fe4b819e6e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 --2- 172.21.15.97:0/440208810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b8074320 0x7fe4b819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 -- 172.21.15.97:0/440208810 >> 172.21.15.97:0/440208810 conn(0x7fe4b80fce80 msgr2=0x7fe4b8101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 -- 172.21.15.97:0/440208810 shutdown_connections 2024-03-26T19:05:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:05:49.522+0000 7fe4bf000640 1 -- 172.21.15.97:0/440208810 wait complete. 2024-03-26T19:05:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 385.0 s, 1478.2 tps, lat 67.085 ms stddev 78.784 2024-03-26T19:05:50.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:50 smithi097 ceph-mon[31955]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 664 op/s 2024-03-26T19:05:50.629 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:50 smithi144 ceph-mon[36053]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 664 op/s 2024-03-26T19:05:50.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:50 smithi105 ceph-mon[36270]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 664 op/s 2024-03-26T19:05:52.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:52 smithi097 ceph-mon[31955]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 MiB/s wr, 644 op/s 2024-03-26T19:05:52.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:52 smithi144 ceph-mon[36053]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 MiB/s wr, 644 op/s 2024-03-26T19:05:52.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:52 smithi105 ceph-mon[36270]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 MiB/s wr, 644 op/s 2024-03-26T19:05:54.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:54 smithi097 ceph-mon[31955]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 MiB/s wr, 610 op/s 2024-03-26T19:05:54.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:54 smithi144 ceph-mon[36053]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 MiB/s wr, 610 op/s 2024-03-26T19:05:54.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:54 smithi105 ceph-mon[36270]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 MiB/s wr, 610 op/s 2024-03-26T19:05:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 390.0 s, 2157.8 tps, lat 45.982 ms stddev 58.281 2024-03-26T19:05:55.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:55 smithi105 ceph-mon[36270]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-03-26T19:05:55.857 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:55 smithi097 ceph-mon[31955]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-03-26T19:05:55.880 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:55 smithi144 ceph-mon[36053]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-03-26T19:05:58.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:05:58 smithi105 ceph-mon[36270]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 600 op/s 2024-03-26T19:05:58.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:05:58 smithi097 ceph-mon[31955]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 600 op/s 2024-03-26T19:05:58.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:05:58 smithi144 ceph-mon[36053]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 600 op/s 2024-03-26T19:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 395.0 s, 1462.6 tps, lat 69.900 ms stddev 89.695 2024-03-26T19:06:00.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:00 smithi105 ceph-mon[36270]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 660 op/s 2024-03-26T19:06:00.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:00 smithi097 ceph-mon[31955]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 660 op/s 2024-03-26T19:06:00.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:00 smithi144 ceph-mon[36053]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 660 op/s 2024-03-26T19:06:02.464 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:02 smithi105 ceph-mon[36270]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.1 MiB/s wr, 659 op/s 2024-03-26T19:06:02.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:02 smithi097 ceph-mon[31955]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.1 MiB/s wr, 659 op/s 2024-03-26T19:06:02.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:02 smithi144 ceph-mon[36053]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.1 MiB/s wr, 659 op/s 2024-03-26T19:06:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.736+0000 7f435a800640 1 -- 172.21.15.97:0/4201065630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4354103b20 msgr2=0x7f4354103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.736+0000 7f435a800640 1 --2- 172.21.15.97:0/4201065630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4354103b20 0x7f4354103f40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f433c009900 tx=0x7f433c02bc30 comp rx=0 tx=0).stop 2024-03-26T19:06:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.736+0000 7f435a800640 1 -- 172.21.15.97:0/4201065630 shutdown_connections 2024-03-26T19:06:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.736+0000 7f435a800640 1 --2- 172.21.15.97:0/4201065630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43541056d0 0x7f435410bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.737+0000 7f435a800640 1 --2- 172.21.15.97:0/4201065630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4354104d10 0x7f4354105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.737+0000 7f435a800640 1 --2- 172.21.15.97:0/4201065630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4354103b20 0x7f4354103f40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.737+0000 7f435a800640 1 -- 172.21.15.97:0/4201065630 >> 172.21.15.97:0/4201065630 conn(0x7f43540ff0b0 msgr2=0x7f43541014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.737+0000 7f435a800640 1 -- 172.21.15.97:0/4201065630 shutdown_connections 2024-03-26T19:06:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.737+0000 7f435a800640 1 -- 172.21.15.97:0/4201065630 wait complete. 2024-03-26T19:06:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.739+0000 7f435a800640 1 Processor -- start 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.740+0000 7f435a800640 1 -- start start 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.741+0000 7f435a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4354103b20 0x7f4354070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.742+0000 7f435a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4354104d10 0x7f4354070780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.742+0000 7f435a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541056d0 0x7f43541a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.742+0000 7f435a800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f435410f7c0 con 0x7f4354103b20 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.742+0000 7f435a800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f435410f640 con 0x7f4354104d10 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.742+0000 7f435a800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f435410f940 con 0x7f43541056d0 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.742+0000 7f4353400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4354103b20 0x7f4354070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.742+0000 7f4353e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541056d0 0x7f43541a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.742+0000 7f4353400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4354103b20 0x7f4354070240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43942/0 (socket says 172.21.15.97:43942) 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.742+0000 7f4353400640 1 -- 172.21.15.97:0/1399760519 learned_addr learned my addr 172.21.15.97:0/1399760519 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.743+0000 7f4352a00640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4354104d10 0x7f4354070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.744+0000 7f4352a00640 1 -- 172.21.15.97:0/1399760519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541056d0 msgr2=0x7f43541a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.744+0000 7f4352a00640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541056d0 0x7f43541a9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.744+0000 7f4352a00640 1 -- 172.21.15.97:0/1399760519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4354103b20 msgr2=0x7f4354070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.744+0000 7f4352a00640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4354103b20 0x7f4354070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.744+0000 7f4352a00640 1 -- 172.21.15.97:0/1399760519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43541aa410 con 0x7f4354104d10 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.744+0000 7f4353e00640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541056d0 0x7f43541a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.745+0000 7f4352a00640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4354104d10 0x7f4354070780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f434400d6d0 tx=0x7f434400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.745+0000 7f434be00640 1 -- 172.21.15.97:0/1399760519 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344017070 con 0x7f4354104d10 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.746+0000 7f434be00640 1 -- 172.21.15.97:0/1399760519 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4344009820 con 0x7f4354104d10 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.746+0000 7f435a800640 1 -- 172.21.15.97:0/1399760519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43541aa5e0 con 0x7f4354104d10 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.746+0000 7f435a800640 1 -- 172.21.15.97:0/1399760519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43541aaaa0 con 0x7f4354104d10 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.746+0000 7f434be00640 1 -- 172.21.15.97:0/1399760519 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344004d00 con 0x7f4354104d10 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.755+0000 7f434be00640 1 -- 172.21.15.97:0/1399760519 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f4344005220 con 0x7f4354104d10 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.758+0000 7f435d61e740 1 Processor -- start 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.758+0000 7f435d61e740 1 -- start start 2024-03-26T19:06:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.759+0000 7f435d61e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 0x558463a931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.759+0000 7f435d61e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 0x558463a98290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.759+0000 7f435d61e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463a9ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.759+0000 7f435d61e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5584639f7e80 con 0x558463a92da0 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.759+0000 7f435d61e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5584639a0660 con 0x558463a93790 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.759+0000 7f435d61e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5584639f3e30 con 0x558463a98900 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.760+0000 7f4352a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463a9ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.760+0000 7f4352a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463a9ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:48690/0 (socket says 172.21.15.97:48690) 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.760+0000 7f4352a00640 1 -- 172.21.15.97:0/4248602396 learned_addr learned my addr 172.21.15.97:0/4248602396 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.760+0000 7f4353e00640 1 --2- 172.21.15.97:0/4248602396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 0x558463a931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.761+0000 7f4353400640 1 --2- 172.21.15.97:0/4248602396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 0x558463a98290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.761+0000 7f4352a00640 1 -- 172.21.15.97:0/4248602396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 msgr2=0x558463a98290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.761+0000 7f4352a00640 1 --2- 172.21.15.97:0/4248602396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 0x558463a98290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.761+0000 7f4352a00640 1 -- 172.21.15.97:0/4248602396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 msgr2=0x558463a931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.761+0000 7f4352a00640 1 --2- 172.21.15.97:0/4248602396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 0x558463a931c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.761+0000 7f4352a00640 1 -- 172.21.15.97:0/4248602396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846370c210 con 0x558463a98900 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.762+0000 7f4352a00640 1 --2- 172.21.15.97:0/4248602396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463a9ad90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43440d2db0 tx=0x7f43440d76d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb919511b761217 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.762+0000 7f4348c00640 1 -- 172.21.15.97:0/4248602396 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43440de070 con 0x558463a98900 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.762+0000 7f4348c00640 1 -- 172.21.15.97:0/4248602396 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43440dab60 con 0x558463a98900 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.762+0000 7f4348c00640 1 -- 172.21.15.97:0/4248602396 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43440d9ca0 con 0x558463a98900 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.764+0000 7f435d61e740 1 -- 172.21.15.97:0/4248602396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 msgr2=0x558463a9ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.764+0000 7f435d61e740 1 --2- 172.21.15.97:0/4248602396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463a9ad90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43440d2db0 tx=0x7f43440d76d0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.764+0000 7f434be00640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f43380a3200 0x7f43380a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.764+0000 7f434be00640 1 -- 172.21.15.97:0/1399760519 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f43440ced00 con 0x7f4354104d10 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.765+0000 7f435d61e740 1 -- 172.21.15.97:0/4248602396 shutdown_connections 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.765+0000 7f435d61e740 1 --2- 172.21.15.97:0/4248602396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463a9ad90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.765+0000 7f435d61e740 1 --2- 172.21.15.97:0/4248602396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 0x558463a98290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.765+0000 7f435d61e740 1 --2- 172.21.15.97:0/4248602396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 0x558463a931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.765+0000 7f435d61e740 1 -- 172.21.15.97:0/4248602396 >> 172.21.15.97:0/4248602396 conn(0x558463a8e310 msgr2=0x558463a8e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.766+0000 7f4353e00640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f43380a3200 0x7f43380a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.766+0000 7f435d61e740 1 -- 172.21.15.97:0/4248602396 shutdown_connections 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.766+0000 7f435d61e740 1 -- 172.21.15.97:0/4248602396 wait complete. 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.766+0000 7f435d61e740 1 -- start start 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.767+0000 7f4353e00640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f43380a3200 0x7f43380a56c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x558463a93ff0 tx=0x7f434c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.767+0000 7f435d61e740 1 Processor -- start 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.767+0000 7f435d61e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 0x558463aa5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.767+0000 7f435d61e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 0x558463aa5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.767+0000 7f435d61e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463aaca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.768+0000 7f4352a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 0x558463aa5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.768+0000 7f435d61e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558463a9b8b0 con 0x558463a92da0 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.768+0000 7f435d61e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5584639b3ac0 con 0x558463a93790 2024-03-26T19:06:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.768+0000 7f435d61e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558463a9ba30 con 0x558463a98900 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.768+0000 7f4352a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 0x558463aa5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:42588/0 (socket says 172.21.15.97:42588) 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.768+0000 7f4352a00640 1 -- 172.21.15.97:0/565491666 learned_addr learned my addr 172.21.15.97:0/565491666 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.768+0000 7f435d61e740 10 client.0 _ll_register_callbacks cb 0x5584639b3490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.768+0000 7f4353e00640 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463aaca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.769+0000 7f4353400640 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 0x558463aa5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.770+0000 7f4353400640 1 -- 172.21.15.97:0/565491666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 msgr2=0x558463aaca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.770+0000 7f4353400640 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463aaca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.770+0000 7f4353400640 1 -- 172.21.15.97:0/565491666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 msgr2=0x558463aa5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.770+0000 7f4353400640 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 0x558463aa5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.770+0000 7f4353400640 1 -- 172.21.15.97:0/565491666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584637d8620 con 0x558463a92da0 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.771+0000 7f4353400640 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 0x558463aa5560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f433c000c00 tx=0x7f433c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16b5a11382285af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.774+0000 7f4341600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.774+0000 7f4341600640 1 -- 172.21.15.97:0/565491666 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c005030 con 0x558463a92da0 2024-03-26T19:06:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.774+0000 7f435d61e740 1 -- 172.21.15.97:0/565491666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43440dab60 con 0x558463a92da0 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.775+0000 7f4341600640 1 -- 172.21.15.97:0/565491666 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f433c0051d0 con 0x558463a92da0 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.775+0000 7f4341600640 1 -- 172.21.15.97:0/565491666 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f433c0053d0 con 0x558463a92da0 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.775+0000 7f435d61e740 10 client.15510 fetch_fsmap learned FSMap version 27 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.776+0000 7f435d61e740 1 -- 172.21.15.97:0/565491666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558463aafe80 con 0x558463a92da0 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.776+0000 7f4341600640 1 -- 172.21.15.97:0/565491666 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c005570 con 0x558463a92da0 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.776+0000 7f4341600640 1 -- 172.21.15.97:0/565491666 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f433c003700 con 0x558463a92da0 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.777+0000 7f435d61e740 10 client.15510 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.780+0000 7f435d61e740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.780+0000 7f435d61e740 20 client.15510 populate_metadata read hostname 'smithi097' 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.780+0000 7f435d61e740 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558463ab1520 0x558463ab3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.780+0000 7f435d61e740 4 client.15510 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.780+0000 7f435d61e740 1 -- 172.21.15.97:0/565491666 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f43440dab60 con 0x558463ab1520 2024-03-26T19:06:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.780+0000 7f4352a00640 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558463ab1520 0x558463ab3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.781+0000 7f4352a00640 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558463ab1520 0x558463ab3990 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f553ee04e075ad server_cookie=6c0e3a6c0403de4d in_seq=0 out_seq=0 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.781+0000 7f4341600640 10 client.15510 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.782+0000 7f4341600640 1 -- 172.21.15.97:0/565491666 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f43440dab60 con 0x558463ab1520 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.782+0000 7f4341600640 10 client.15510 handle_command_reply: tid=0 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.783+0000 7f435d61e740 1 client.15510 shutdown 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.784+0000 7f435d61e740 1 -- 172.21.15.97:0/565491666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 msgr2=0x558463aa5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.784+0000 7f435d61e740 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 0x558463aa5560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f433c000c00 tx=0x7f433c004270 comp rx=0 tx=0).stop 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.784+0000 7f435d61e740 1 -- 172.21.15.97:0/565491666 shutdown_connections 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.784+0000 7f435d61e740 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558463ab1520 0x558463ab3990 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.784+0000 7f435d61e740 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463aaca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.784+0000 7f435d61e740 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a93790 0x558463aa5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.784+0000 7f4341600640 0 client.15510 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.784+0000 7f435d61e740 1 --2- 172.21.15.97:0/565491666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a92da0 0x558463aa5560 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.784+0000 7f435d61e740 1 -- 172.21.15.97:0/565491666 >> 172.21.15.97:0/565491666 conn(0x558463a8e310 msgr2=0x558463a8d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.785+0000 7f435d61e740 1 -- 172.21.15.97:0/565491666 shutdown_connections 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.785+0000 7f435d61e740 1 -- 172.21.15.97:0/565491666 wait complete. 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.786+0000 7f435d61e740 20 client.15510 trim_cache size 0 max 16384 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.849+0000 7f435d61e740 1 Processor -- start 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.851+0000 7f435d61e740 1 -- start start 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.852+0000 7f435d61e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463a94850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.852+0000 7f435d61e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a93790 0x558463aac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.852+0000 7f435d61e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a98900 0x558463aa4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.852+0000 7f435d61e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f433000e480 con 0x558463a98900 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.852+0000 7f435d61e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f433000ffc0 con 0x558463a92da0 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.852+0000 7f435d61e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f433000eb50 con 0x558463a93790 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.852+0000 7f4352a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a93790 0x558463aac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.852+0000 7f4353400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463a94850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.852+0000 7f4353400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463a94850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:42594/0 (socket says 172.21.15.97:42594) 2024-03-26T19:06:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.852+0000 7f4353400640 1 -- 172.21.15.97:0/2610661046 learned_addr learned my addr 172.21.15.97:0/2610661046 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.853+0000 7f4353400640 1 -- 172.21.15.97:0/2610661046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a93790 msgr2=0x558463aac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.853+0000 7f4353400640 1 --2- 172.21.15.97:0/2610661046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a93790 0x558463aac990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.853+0000 7f4353400640 1 -- 172.21.15.97:0/2610661046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a98900 msgr2=0x558463aa4b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.853+0000 7f4353e00640 1 --2- 172.21.15.97:0/2610661046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a98900 0x558463aa4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.853+0000 7f4353400640 1 --2- 172.21.15.97:0/2610661046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a98900 0x558463aa4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.854+0000 7f4353400640 1 -- 172.21.15.97:0/2610661046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558463a8ed60 con 0x558463a92da0 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.854+0000 7f4353400640 1 --2- 172.21.15.97:0/2610661046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463a94850 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f433c003ba0 tx=0x7f433c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df8f3f2eecd33d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.855+0000 7f4343e00640 1 -- 172.21.15.97:0/2610661046 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c009520 con 0x558463a92da0 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.855+0000 7f4343e00640 1 -- 172.21.15.97:0/2610661046 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c003d90 con 0x558463a92da0 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.855+0000 7f4343e00640 1 -- 172.21.15.97:0/2610661046 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c03fb30 con 0x558463a92da0 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.857+0000 7f435d61e740 1 -- 172.21.15.97:0/2610661046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 msgr2=0x558463a94850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.857+0000 7f435d61e740 1 --2- 172.21.15.97:0/2610661046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463a94850 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f433c003ba0 tx=0x7f433c02bc30 comp rx=0 tx=0).stop 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.858+0000 7f435d61e740 1 -- 172.21.15.97:0/2610661046 shutdown_connections 2024-03-26T19:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.858+0000 7f435d61e740 1 --2- 172.21.15.97:0/2610661046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a98900 0x558463aa4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.858+0000 7f435d61e740 1 --2- 172.21.15.97:0/2610661046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a93790 0x558463aac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.858+0000 7f435d61e740 1 --2- 172.21.15.97:0/2610661046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463a94850 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.858+0000 7f435d61e740 1 -- 172.21.15.97:0/2610661046 >> 172.21.15.97:0/2610661046 conn(0x558463a8e310 msgr2=0x558463a94140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.859+0000 7f435d61e740 1 -- 172.21.15.97:0/2610661046 shutdown_connections 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.859+0000 7f435d61e740 1 -- 172.21.15.97:0/2610661046 wait complete. 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.859+0000 7f435d61e740 1 -- start start 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.860+0000 7f435d61e740 1 Processor -- start 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.861+0000 7f435d61e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463aa82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.861+0000 7f435d61e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a93790 0x558463aa8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.861+0000 7f435d61e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463aab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.861+0000 7f435d61e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f433000f8f0 con 0x558463a93790 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.861+0000 7f435d61e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f433000f220 con 0x558463a92da0 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.861+0000 7f4353400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463aa82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.861+0000 7f4353400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463aa82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:42608/0 (socket says 172.21.15.97:42608) 2024-03-26T19:06:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.861+0000 7f4353400640 1 -- 172.21.15.97:0/2633626609 learned_addr learned my addr 172.21.15.97:0/2633626609 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.861+0000 7f435d61e740 1 -- 172.21.15.97:0/2633626609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f433000c400 con 0x558463a98900 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.862+0000 7f435d61e740 10 client.0 _ll_register_callbacks cb 0x7f433c02cd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.862+0000 7f4352a00640 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a93790 0x558463aa8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.862+0000 7f4353e00640 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463aab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.862+0000 7f4353400640 1 -- 172.21.15.97:0/2633626609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 msgr2=0x558463aab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.862+0000 7f4353400640 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463aab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.862+0000 7f4353400640 1 -- 172.21.15.97:0/2633626609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a93790 msgr2=0x558463aa8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.862+0000 7f4353400640 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a93790 0x558463aa8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.862+0000 7f4353400640 1 -- 172.21.15.97:0/2633626609 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558463aabe10 con 0x558463a92da0 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.863+0000 7f4353400640 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463aa82c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f433c000c00 tx=0x7f433c003f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daad7372f83aa90a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.863+0000 7f4341600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.863+0000 7f4341600640 1 -- 172.21.15.97:0/2633626609 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c009520 con 0x558463a92da0 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.863+0000 7f4341600640 1 -- 172.21.15.97:0/2633626609 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f433c0037f0 con 0x558463a92da0 2024-03-26T19:06:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.864+0000 7f435d61e740 1 -- 172.21.15.97:0/2633626609 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433c003d90 con 0x558463a92da0 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.864+0000 7f4341600640 1 -- 172.21.15.97:0/2633626609 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f433c03f8b0 con 0x558463a92da0 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.864+0000 7f435d61e740 10 client.25811 fetch_fsmap learned FSMap version 27 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.864+0000 7f435d61e740 1 -- 172.21.15.97:0/2633626609 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558463aa00c0 con 0x558463a92da0 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.865+0000 7f4341600640 1 -- 172.21.15.97:0/2633626609 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c003d90 con 0x558463a92da0 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.866+0000 7f4341600640 1 -- 172.21.15.97:0/2633626609 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f433c04aa90 con 0x558463a92da0 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.866+0000 7f435d61e740 10 client.25811 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.866+0000 7f435d61e740 10 client.25811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.866+0000 7f435d61e740 20 client.25811 populate_metadata read hostname 'smithi097' 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.866+0000 7f435d61e740 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558463aa0a00 0x558463aa2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.866+0000 7f435d61e740 4 client.25811 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.866+0000 7f435d61e740 1 -- 172.21.15.97:0/2633626609 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558463a053c0 con 0x558463aa0a00 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.867+0000 7f4352a00640 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558463aa0a00 0x558463aa2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.869+0000 7f4352a00640 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558463aa0a00 0x558463aa2e20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73097a459a50a6d5 server_cookie=2df4ffbea7b11b01 in_seq=0 out_seq=0 2024-03-26T19:06:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.869+0000 7f4341600640 10 client.25811 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.905+0000 7f4341600640 1 -- 172.21.15.97:0/2633626609 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558463a053c0 con 0x558463aa0a00 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.905+0000 7f4341600640 10 client.25811 handle_command_reply: tid=0 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.905+0000 7f435d61e740 1 client.25811 shutdown 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.905+0000 7f435d61e740 1 -- 172.21.15.97:0/2633626609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 msgr2=0x558463aa82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.905+0000 7f435d61e740 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463aa82c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f433c000c00 tx=0x7f433c003f40 comp rx=0 tx=0).stop 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.905+0000 7f435d61e740 1 -- 172.21.15.97:0/2633626609 shutdown_connections 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.905+0000 7f435d61e740 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558463aa0a00 0x558463aa2e20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.906+0000 7f435d61e740 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558463a98900 0x558463aab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.906+0000 7f435d61e740 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558463a93790 0x558463aa8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.906+0000 7f435d61e740 1 --2- 172.21.15.97:0/2633626609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558463a92da0 0x558463aa82c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.906+0000 7f435d61e740 1 -- 172.21.15.97:0/2633626609 >> 172.21.15.97:0/2633626609 conn(0x558463a8e310 msgr2=0x558463a949d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.906+0000 7f4341600640 0 client.25811 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.906+0000 7f435d61e740 1 -- 172.21.15.97:0/2633626609 shutdown_connections 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.906+0000 7f435d61e740 1 -- 172.21.15.97:0/2633626609 wait complete. 2024-03-26T19:06:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.906+0000 7f435d61e740 20 client.25811 trim_cache size 0 max 16384 2024-03-26T19:06:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:06:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:06:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:06:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 -- 172.21.15.97:0/1399760519 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f43380a3200 msgr2=0x7f43380a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f43380a3200 0x7f43380a56c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x558463a93ff0 tx=0x7f434c009500 comp rx=0 tx=0).stop 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 -- 172.21.15.97:0/1399760519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4354104d10 msgr2=0x7f4354070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4354104d10 0x7f4354070780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f434400d6d0 tx=0x7f434400db90 comp rx=0 tx=0).stop 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 -- 172.21.15.97:0/1399760519 shutdown_connections 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f43380a3200 0x7f43380a56c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541056d0 0x7f43541a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4354104d10 0x7f4354070780 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 --2- 172.21.15.97:0/1399760519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4354103b20 0x7f4354070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 -- 172.21.15.97:0/1399760519 >> 172.21.15.97:0/1399760519 conn(0x7f43540ff0b0 msgr2=0x7f4354107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 -- 172.21.15.97:0/1399760519 shutdown_connections 2024-03-26T19:06:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:02.907+0000 7f435a800640 1 -- 172.21.15.97:0/1399760519 wait complete. 2024-03-26T19:06:04.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:04 smithi097 ceph-mon[31955]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.5 MiB/s wr, 661 op/s 2024-03-26T19:06:04.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:04 smithi144 ceph-mon[36053]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.5 MiB/s wr, 661 op/s 2024-03-26T19:06:04.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:04 smithi105 ceph-mon[36270]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.5 MiB/s wr, 661 op/s 2024-03-26T19:06:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 400.0 s, 1992.0 tps, lat 49.856 ms stddev 56.756 2024-03-26T19:06:06.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:06 smithi097 ceph-mon[31955]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 709 op/s 2024-03-26T19:06:06.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:06 smithi144 ceph-mon[36053]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 709 op/s 2024-03-26T19:06:06.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:06 smithi105 ceph-mon[36270]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 709 op/s 2024-03-26T19:06:08.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:08 smithi097 ceph-mon[31955]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 622 op/s 2024-03-26T19:06:08.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:08 smithi144 ceph-mon[36053]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 622 op/s 2024-03-26T19:06:08.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:08 smithi105 ceph-mon[36270]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 622 op/s 2024-03-26T19:06:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 405.0 s, 1638.6 tps, lat 60.524 ms stddev 71.862 2024-03-26T19:06:10.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:10 smithi097 ceph-mon[31955]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 652 op/s 2024-03-26T19:06:10.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:10 smithi144 ceph-mon[36053]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 652 op/s 2024-03-26T19:06:10.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:10 smithi105 ceph-mon[36270]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 652 op/s 2024-03-26T19:06:12.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:12 smithi097 ceph-mon[31955]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 682 op/s 2024-03-26T19:06:12.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:12 smithi144 ceph-mon[36053]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 682 op/s 2024-03-26T19:06:12.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:12 smithi105 ceph-mon[36270]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 682 op/s 2024-03-26T19:06:14.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:14 smithi097 ceph-mon[31955]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 678 op/s 2024-03-26T19:06:14.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:14 smithi144 ceph-mon[36053]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 678 op/s 2024-03-26T19:06:14.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:14 smithi105 ceph-mon[36270]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 678 op/s 2024-03-26T19:06:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 410.0 s, 2084.4 tps, lat 47.938 ms stddev 55.696 2024-03-26T19:06:15.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:15 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:06:15.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:15 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:06:15.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:15 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:06:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.109+0000 7f44a5400640 1 -- 172.21.15.97:0/1790549261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44a0104c90 msgr2=0x7f44a0107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.109+0000 7f44a5400640 1 --2- 172.21.15.97:0/1790549261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44a0104c90 0x7f44a0107080 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4488009900 tx=0x7f448802bc40 comp rx=0 tx=0).stop 2024-03-26T19:06:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.110+0000 7f44a5400640 1 -- 172.21.15.97:0/1790549261 shutdown_connections 2024-03-26T19:06:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.110+0000 7f44a5400640 1 --2- 172.21.15.97:0/1790549261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a01075c0 0x7f44a0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.110+0000 7f44a5400640 1 --2- 172.21.15.97:0/1790549261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44a0104c90 0x7f44a0107080 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.110+0000 7f44a5400640 1 --2- 172.21.15.97:0/1790549261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a0102360 0x7f44a0104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.110+0000 7f44a5400640 1 -- 172.21.15.97:0/1790549261 >> 172.21.15.97:0/1790549261 conn(0x7f44a00fbd50 msgr2=0x7f44a00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.110+0000 7f44a5400640 1 -- 172.21.15.97:0/1790549261 shutdown_connections 2024-03-26T19:06:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.110+0000 7f44a5400640 1 -- 172.21.15.97:0/1790549261 wait complete. 2024-03-26T19:06:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.112+0000 7f44a5400640 1 Processor -- start 2024-03-26T19:06:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.112+0000 7f44a5400640 1 -- start start 2024-03-26T19:06:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.112+0000 7f44a5400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a0102360 0x7f44a0199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.112+0000 7f44a5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a0104c90 0x7f44a019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.113+0000 7f44a5400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44a01075c0 0x7f44a01a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.113+0000 7f44a5400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d5a0 con 0x7f44a0104c90 2024-03-26T19:06:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.113+0000 7f44a5400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d420 con 0x7f44a01075c0 2024-03-26T19:06:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.113+0000 7f44a5400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d720 con 0x7f44a0102360 2024-03-26T19:06:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.113+0000 7f449f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44a01075c0 0x7f44a01a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.113+0000 7f449e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a0102360 0x7f44a0199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.113+0000 7f449f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44a01075c0 0x7f44a01a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:49860/0 (socket says 172.21.15.97:49860) 2024-03-26T19:06:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.113+0000 7f449f200640 1 -- 172.21.15.97:0/2141401289 learned_addr learned my addr 172.21.15.97:0/2141401289 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.113+0000 7f449f200640 1 -- 172.21.15.97:0/2141401289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a0102360 msgr2=0x7f44a0199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.113+0000 7f449f200640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a0102360 0x7f44a0199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.114+0000 7f4497e00640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a0104c90 0x7f44a019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.114+0000 7f449f200640 1 -- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a0104c90 msgr2=0x7f44a019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.114+0000 7f449f200640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a0104c90 0x7f44a019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.114+0000 7f449f200640 1 -- 172.21.15.97:0/2141401289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a01a1960 con 0x7f44a01075c0 2024-03-26T19:06:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.114+0000 7f449e800640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a0102360 0x7f44a0199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:06:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.114+0000 7f4497e00640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a0104c90 0x7f44a019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:06:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.114+0000 7f449f200640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44a01075c0 0x7f44a01a12c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f449800cce0 tx=0x7f449800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.115+0000 7f4497400640 1 -- 172.21.15.97:0/2141401289 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449800f450 con 0x7f44a01075c0 2024-03-26T19:06:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.115+0000 7f4497400640 1 -- 172.21.15.97:0/2141401289 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449800fad0 con 0x7f44a01075c0 2024-03-26T19:06:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.115+0000 7f44a5400640 1 -- 172.21.15.97:0/2141401289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a01a1c50 con 0x7f44a01075c0 2024-03-26T19:06:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.115+0000 7f44a5400640 1 -- 172.21.15.97:0/2141401289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a01a21a0 con 0x7f44a01075c0 2024-03-26T19:06:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.115+0000 7f4497400640 1 -- 172.21.15.97:0/2141401289 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498018c80 con 0x7f44a01075c0 2024-03-26T19:06:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.124+0000 7f4497400640 1 -- 172.21.15.97:0/2141401289 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f449800f5f0 con 0x7f44a01075c0 2024-03-26T19:06:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.128+0000 7f4497400640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44840ab850 0x7f44840add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.128+0000 7f4497400640 1 -- 172.21.15.97:0/2141401289 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f44980cf740 con 0x7f44a01075c0 2024-03-26T19:06:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.129+0000 7f449e800640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44840ab850 0x7f44840add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.131+0000 7f449e800640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44840ab850 0x7f44840add10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f44900058c0 tx=0x7f449000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.132+0000 7f44a820c740 1 Processor -- start 2024-03-26T19:06:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.135+0000 7f44a820c740 1 -- start start 2024-03-26T19:06:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.136+0000 7f44a820c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 0x55d6c456b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.136+0000 7f44a820c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 0x55d6c4570290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.136+0000 7f44a820c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c4572d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f44a820c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d6c44cfe80 con 0x55d6c456ada0 2024-03-26T19:06:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f44a820c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d6c4478660 con 0x55d6c456b790 2024-03-26T19:06:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f44a820c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d6c44cbe30 con 0x55d6c4570900 2024-03-26T19:06:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f4497e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 0x55d6c4570290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f4497e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 0x55d6c4570290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:49866/0 (socket says 172.21.15.97:49866) 2024-03-26T19:06:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f4497e00640 1 -- 172.21.15.97:0/878457432 learned_addr learned my addr 172.21.15.97:0/878457432 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f449f200640 1 --2- 172.21.15.97:0/878457432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c4572d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f449e800640 1 --2- 172.21.15.97:0/878457432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 0x55d6c456b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f449f200640 1 -- 172.21.15.97:0/878457432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 msgr2=0x55d6c4570290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f449f200640 1 --2- 172.21.15.97:0/878457432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 0x55d6c4570290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f449f200640 1 -- 172.21.15.97:0/878457432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 msgr2=0x55d6c456b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f449f200640 1 --2- 172.21.15.97:0/878457432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 0x55d6c456b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.137+0000 7f449f200640 1 -- 172.21.15.97:0/878457432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c41e4210 con 0x55d6c4570900 2024-03-26T19:06:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.138+0000 7f449f200640 1 --2- 172.21.15.97:0/878457432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c4572d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f44980cd5e0 tx=0x7f4498018e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fc9b9d82a6b4d68 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.138+0000 7f448f400640 1 -- 172.21.15.97:0/878457432 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44980b4020 con 0x55d6c4570900 2024-03-26T19:06:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.139+0000 7f448f400640 1 -- 172.21.15.97:0/878457432 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449809bea0 con 0x55d6c4570900 2024-03-26T19:06:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.139+0000 7f448f400640 1 -- 172.21.15.97:0/878457432 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498098480 con 0x55d6c4570900 2024-03-26T19:06:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.140+0000 7f44a820c740 1 -- 172.21.15.97:0/878457432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 msgr2=0x55d6c4572d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.140+0000 7f44a820c740 1 --2- 172.21.15.97:0/878457432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c4572d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f44980cd5e0 tx=0x7f4498018e20 comp rx=0 tx=0).stop 2024-03-26T19:06:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.141+0000 7f44a820c740 1 -- 172.21.15.97:0/878457432 shutdown_connections 2024-03-26T19:06:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.141+0000 7f44a820c740 1 --2- 172.21.15.97:0/878457432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c4572d90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.141+0000 7f44a820c740 1 --2- 172.21.15.97:0/878457432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 0x55d6c4570290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.141+0000 7f44a820c740 1 --2- 172.21.15.97:0/878457432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 0x55d6c456b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.141+0000 7f44a820c740 1 -- 172.21.15.97:0/878457432 >> 172.21.15.97:0/878457432 conn(0x55d6c4566310 msgr2=0x55d6c4566710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.142+0000 7f44a820c740 1 -- 172.21.15.97:0/878457432 shutdown_connections 2024-03-26T19:06:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.142+0000 7f44a820c740 1 -- 172.21.15.97:0/878457432 wait complete. 2024-03-26T19:06:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.142+0000 7f44a820c740 1 -- start start 2024-03-26T19:06:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.143+0000 7f44a820c740 1 Processor -- start 2024-03-26T19:06:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.143+0000 7f44a820c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4574d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.143+0000 7f44a820c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 0x55d6c45752b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.143+0000 7f44a820c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c4570900 0x55d6c457c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.143+0000 7f44a820c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d6c45738b0 con 0x55d6c4570900 2024-03-26T19:06:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.143+0000 7f44a820c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d6c448bac0 con 0x55d6c456b790 2024-03-26T19:06:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.143+0000 7f44a820c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d6c4573a30 con 0x55d6c456ada0 2024-03-26T19:06:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.143+0000 7f44a820c740 10 client.0 _ll_register_callbacks cb 0x55d6c448b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:06:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.144+0000 7f449e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4574d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.144+0000 7f449e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4574d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:32868/0 (socket says 172.21.15.97:32868) 2024-03-26T19:06:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.144+0000 7f449e800640 1 -- 172.21.15.97:0/2497426591 learned_addr learned my addr 172.21.15.97:0/2497426591 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.144+0000 7f4497e00640 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 0x55d6c45752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.144+0000 7f449f200640 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c4570900 0x55d6c457c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.145+0000 7f449e800640 1 -- 172.21.15.97:0/2497426591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 msgr2=0x55d6c45752b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.145+0000 7f449e800640 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 0x55d6c45752b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.145+0000 7f449e800640 1 -- 172.21.15.97:0/2497426591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c4570900 msgr2=0x55d6c457c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.145+0000 7f449e800640 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c4570900 0x55d6c457c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.145+0000 7f449e800640 1 -- 172.21.15.97:0/2497426591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c42b0620 con 0x55d6c456ada0 2024-03-26T19:06:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.145+0000 7f449e800640 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4574d70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f449000afd0 tx=0x7f4490004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ab8a540b5f16b72 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.146+0000 7f4483e00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:06:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.146+0000 7f449f200640 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c4570900 0x55d6c457c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:06:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.146+0000 7f4483e00640 1 -- 172.21.15.97:0/2497426591 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490016070 con 0x55d6c456ada0 2024-03-26T19:06:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.146+0000 7f4483e00640 1 -- 172.21.15.97:0/2497426591 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f449001cdb0 con 0x55d6c456ada0 2024-03-26T19:06:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.147+0000 7f44a820c740 1 -- 172.21.15.97:0/2497426591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f449809bea0 con 0x55d6c456ada0 2024-03-26T19:06:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.147+0000 7f4483e00640 1 -- 172.21.15.97:0/2497426591 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4490005020 con 0x55d6c456ada0 2024-03-26T19:06:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.147+0000 7f44a820c740 10 client.25777 fetch_fsmap learned FSMap version 27 2024-03-26T19:06:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.147+0000 7f44a820c740 1 -- 172.21.15.97:0/2497426591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d6c457f6d0 con 0x55d6c456ada0 2024-03-26T19:06:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.148+0000 7f4483e00640 1 -- 172.21.15.97:0/2497426591 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449809bea0 con 0x55d6c456ada0 2024-03-26T19:06:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.148+0000 7f4483e00640 1 -- 172.21.15.97:0/2497426591 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4490023030 con 0x55d6c456ada0 2024-03-26T19:06:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.148+0000 7f44a820c740 10 client.25777 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:06:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.148+0000 7f44a820c740 10 client.25777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:06:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.148+0000 7f44a820c740 20 client.25777 populate_metadata read hostname 'smithi097' 2024-03-26T19:06:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.148+0000 7f44a820c740 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d6c4580e10 0x55d6c45832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.148+0000 7f44a820c740 4 client.25777 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:06:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.148+0000 7f44a820c740 1 -- 172.21.15.97:0/2497426591 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6c44dd3c0 con 0x55d6c4580e10 2024-03-26T19:06:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.149+0000 7f4497e00640 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d6c4580e10 0x55d6c45832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.150+0000 7f4483e00640 10 client.25777 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.151+0000 7f4497e00640 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d6c4580e10 0x55d6c45832b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=366e8ec5cf05525e server_cookie=551fa5754cd25261 in_seq=0 out_seq=0 2024-03-26T19:06:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.152+0000 7f4483e00640 1 -- 172.21.15.97:0/2497426591 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d6c44dd3c0 con 0x55d6c4580e10 2024-03-26T19:06:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.152+0000 7f4483e00640 10 client.25777 handle_command_reply: tid=0 2024-03-26T19:06:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.152+0000 7f44a820c740 1 client.25777 shutdown 2024-03-26T19:06:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f44a820c740 1 -- 172.21.15.97:0/2497426591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 msgr2=0x55d6c4574d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f44a820c740 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4574d70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f449000afd0 tx=0x7f4490004730 comp rx=0 tx=0).stop 2024-03-26T19:06:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f44a820c740 1 -- 172.21.15.97:0/2497426591 shutdown_connections 2024-03-26T19:06:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f44a820c740 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d6c4580e10 0x55d6c45832b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f44a820c740 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c4570900 0x55d6c457c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f44a820c740 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c456b790 0x55d6c45752b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f44a820c740 1 --2- 172.21.15.97:0/2497426591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4574d70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f44a820c740 1 -- 172.21.15.97:0/2497426591 >> 172.21.15.97:0/2497426591 conn(0x55d6c4566310 msgr2=0x55d6c4565540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f4483e00640 0 client.25777 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f44a820c740 1 -- 172.21.15.97:0/2497426591 shutdown_connections 2024-03-26T19:06:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.153+0000 7f44a820c740 1 -- 172.21.15.97:0/2497426591 wait complete. 2024-03-26T19:06:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.154+0000 7f44a820c740 20 client.25777 trim_cache size 0 max 16384 2024-03-26T19:06:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.186+0000 7f44a820c740 1 Processor -- start 2024-03-26T19:06:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.186+0000 7f44a820c740 1 -- start start 2024-03-26T19:06:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.186+0000 7f44a820c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4575090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.186+0000 7f44a820c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456b790 0x55d6c457c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.186+0000 7f44a820c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c4570900 0x55d6c4564cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.186+0000 7f44a820c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f447c00e490 con 0x55d6c456b790 2024-03-26T19:06:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.186+0000 7f44a820c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f447c00ffd0 con 0x55d6c4570900 2024-03-26T19:06:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.186+0000 7f44a820c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f447c00eb60 con 0x55d6c456ada0 2024-03-26T19:06:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.187+0000 7f4497e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456b790 0x55d6c457c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.187+0000 7f449e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4575090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.187+0000 7f449f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c4570900 0x55d6c4564cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.187+0000 7f4497e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456b790 0x55d6c457c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45896/0 (socket says 172.21.15.97:45896) 2024-03-26T19:06:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.187+0000 7f4497e00640 1 -- 172.21.15.97:0/3091920821 learned_addr learned my addr 172.21.15.97:0/3091920821 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.187+0000 7f449e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4575090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:32878/0 (socket says 172.21.15.97:32878) 2024-03-26T19:06:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.188+0000 7f4497e00640 1 -- 172.21.15.97:0/3091920821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 msgr2=0x55d6c4575090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.188+0000 7f4497e00640 1 --2- 172.21.15.97:0/3091920821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4575090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.188+0000 7f4497e00640 1 -- 172.21.15.97:0/3091920821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c4570900 msgr2=0x55d6c4564cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.188+0000 7f4497e00640 1 --2- 172.21.15.97:0/3091920821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c4570900 0x55d6c4564cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.188+0000 7f4497e00640 1 -- 172.21.15.97:0/3091920821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c4564ab0 con 0x55d6c456b790 2024-03-26T19:06:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.188+0000 7f449e800640 1 --2- 172.21.15.97:0/3091920821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4575090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:06:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.188+0000 7f449f200640 1 --2- 172.21.15.97:0/3091920821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c4570900 0x55d6c4564cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:06:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.189+0000 7f4497e00640 1 --2- 172.21.15.97:0/3091920821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456b790 0x55d6c457c280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4488005ab0 tx=0x7f448802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d896113b676121 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.190+0000 7f448ea00640 1 -- 172.21.15.97:0/3091920821 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488048cb0 con 0x55d6c456b790 2024-03-26T19:06:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.190+0000 7f448ea00640 1 -- 172.21.15.97:0/3091920821 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4488004040 con 0x55d6c456b790 2024-03-26T19:06:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.190+0000 7f448ea00640 1 -- 172.21.15.97:0/3091920821 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488005600 con 0x55d6c456b790 2024-03-26T19:06:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.191+0000 7f44a820c740 1 -- 172.21.15.97:0/3091920821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456b790 msgr2=0x55d6c457c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.191+0000 7f44a820c740 1 --2- 172.21.15.97:0/3091920821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456b790 0x55d6c457c280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4488005ab0 tx=0x7f448802c6c0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.191+0000 7f44a820c740 1 -- 172.21.15.97:0/3091920821 shutdown_connections 2024-03-26T19:06:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.191+0000 7f44a820c740 1 --2- 172.21.15.97:0/3091920821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c4570900 0x55d6c4564cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.191+0000 7f44a820c740 1 --2- 172.21.15.97:0/3091920821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456b790 0x55d6c457c280 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.191+0000 7f44a820c740 1 --2- 172.21.15.97:0/3091920821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c456ada0 0x55d6c4575090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.191+0000 7f44a820c740 1 -- 172.21.15.97:0/3091920821 >> 172.21.15.97:0/3091920821 conn(0x55d6c4566310 msgr2=0x55d6c4574980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.192+0000 7f44a820c740 1 -- 172.21.15.97:0/3091920821 shutdown_connections 2024-03-26T19:06:16.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.192+0000 7f44a820c740 1 -- 172.21.15.97:0/3091920821 wait complete. 2024-03-26T19:06:16.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.192+0000 7f44a820c740 1 -- start start 2024-03-26T19:06:16.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.193+0000 7f44a820c740 1 Processor -- start 2024-03-26T19:06:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.194+0000 7f44a820c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 0x55d6c457a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.194+0000 7f44a820c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c457a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.194+0000 7f44a820c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c457b7d0 0x55d6c457bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.194+0000 7f44a820c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f447c00f900 con 0x55d6c456ada0 2024-03-26T19:06:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.194+0000 7f44a820c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f447c00f230 con 0x55d6c457b7d0 2024-03-26T19:06:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.194+0000 7f44a820c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f447c00c470 con 0x55d6c4570900 2024-03-26T19:06:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.194+0000 7f44a820c740 10 client.0 _ll_register_callbacks cb 0x7f4490010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:06:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.194+0000 7f449e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 0x55d6c457a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.195+0000 7f4497e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c457a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.195+0000 7f449e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 0x55d6c457a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45906/0 (socket says 172.21.15.97:45906) 2024-03-26T19:06:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.195+0000 7f4497e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c457a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:32892/0 (socket says 172.21.15.97:32892) 2024-03-26T19:06:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.195+0000 7f449e800640 1 -- 172.21.15.97:0/1807275252 learned_addr learned my addr 172.21.15.97:0/1807275252 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.195+0000 7f449f200640 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c457b7d0 0x55d6c457bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.195+0000 7f449e800640 1 -- 172.21.15.97:0/1807275252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 msgr2=0x55d6c457a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.195+0000 7f449e800640 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c457a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.196+0000 7f449e800640 1 -- 172.21.15.97:0/1807275252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c457b7d0 msgr2=0x55d6c457bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.196+0000 7f449e800640 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c457b7d0 0x55d6c457bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.196+0000 7f449e800640 1 -- 172.21.15.97:0/1807275252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c457c190 con 0x55d6c456ada0 2024-03-26T19:06:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.196+0000 7f4497e00640 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c457a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:06:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.196+0000 7f449e800640 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 0x55d6c457a280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f449000fc90 tx=0x7f449001cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e3f898fe1d7270 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.197+0000 7f4483e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T19:06:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.197+0000 7f449f200640 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c457b7d0 0x55d6c457bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:06:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.200+0000 7f4483e00640 1 -- 172.21.15.97:0/1807275252 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449003d060 con 0x55d6c456ada0 2024-03-26T19:06:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.201+0000 7f4483e00640 1 -- 172.21.15.97:0/1807275252 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f4490005c40 con 0x55d6c456ada0 2024-03-26T19:06:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.201+0000 7f44a820c740 1 -- 172.21.15.97:0/1807275252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4488004040 con 0x55d6c456ada0 2024-03-26T19:06:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.201+0000 7f4483e00640 1 -- 172.21.15.97:0/1807275252 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f449001ad50 con 0x55d6c456ada0 2024-03-26T19:06:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.202+0000 7f44a820c740 10 client.15516 fetch_fsmap learned FSMap version 27 2024-03-26T19:06:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.202+0000 7f44a820c740 1 -- 172.21.15.97:0/1807275252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d6c45e7f50 con 0x55d6c456ada0 2024-03-26T19:06:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.205+0000 7f4483e00640 1 -- 172.21.15.97:0/1807275252 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4488004040 con 0x55d6c456ada0 2024-03-26T19:06:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.205+0000 7f4483e00640 1 -- 172.21.15.97:0/1807275252 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4490023040 con 0x55d6c456ada0 2024-03-26T19:06:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.206+0000 7f44a820c740 10 client.15516 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:06:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.206+0000 7f44a820c740 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:06:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.206+0000 7f44a820c740 20 client.15516 populate_metadata read hostname 'smithi097' 2024-03-26T19:06:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.206+0000 7f44a820c740 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d6c45e88e0 0x55d6c45ead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.206+0000 7f44a820c740 4 client.15516 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:06:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.206+0000 7f44a820c740 1 -- 172.21.15.97:0/1807275252 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6c44d7710 con 0x55d6c45e88e0 2024-03-26T19:06:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.206+0000 7f4497e00640 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d6c45e88e0 0x55d6c45ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.211+0000 7f4497e00640 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d6c45e88e0 0x55d6c45ead00 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe74785aa770cab server_cookie=83bdee92c5fe4086 in_seq=0 out_seq=0 2024-03-26T19:06:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.211+0000 7f4483e00640 10 client.15516 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.268+0000 7f4483e00640 1 -- 172.21.15.97:0/1807275252 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d6c44d7710 con 0x55d6c45e88e0 2024-03-26T19:06:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.268+0000 7f4483e00640 10 client.15516 handle_command_reply: tid=0 2024-03-26T19:06:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.269+0000 7f44a820c740 1 client.15516 shutdown 2024-03-26T19:06:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.269+0000 7f44a820c740 1 -- 172.21.15.97:0/1807275252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 msgr2=0x55d6c457a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.269+0000 7f44a820c740 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 0x55d6c457a280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f449000fc90 tx=0x7f449001cde0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.270+0000 7f44a820c740 1 -- 172.21.15.97:0/1807275252 shutdown_connections 2024-03-26T19:06:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.270+0000 7f44a820c740 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d6c45e88e0 0x55d6c45ead00 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.270+0000 7f44a820c740 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6c457b7d0 0x55d6c457bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.270+0000 7f44a820c740 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6c4570900 0x55d6c457a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.270+0000 7f44a820c740 1 --2- 172.21.15.97:0/1807275252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c456ada0 0x55d6c457a280 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.270+0000 7f44a820c740 1 -- 172.21.15.97:0/1807275252 >> 172.21.15.97:0/1807275252 conn(0x55d6c4566310 msgr2=0x55d6c4575180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.270+0000 7f4483e00640 0 client.15516 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.271+0000 7f44a820c740 1 -- 172.21.15.97:0/1807275252 shutdown_connections 2024-03-26T19:06:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.271+0000 7f44a820c740 1 -- 172.21.15.97:0/1807275252 wait complete. 2024-03-26T19:06:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.272+0000 7f44a820c740 20 client.15516 trim_cache size 0 max 16384 2024-03-26T19:06:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:06:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:06:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:06:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:06:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.273+0000 7f44a5400640 1 -- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44840ab850 msgr2=0x7f44840add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.273+0000 7f44a5400640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44840ab850 0x7f44840add10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f44900058c0 tx=0x7f449000a2d0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.274+0000 7f44a5400640 1 -- 172.21.15.97:0/2141401289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44a01075c0 msgr2=0x7f44a01a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.274+0000 7f44a5400640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44a01075c0 0x7f44a01a12c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f449800cce0 tx=0x7f449800a620 comp rx=0 tx=0).stop 2024-03-26T19:06:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.274+0000 7f44a5400640 1 -- 172.21.15.97:0/2141401289 shutdown_connections 2024-03-26T19:06:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.274+0000 7f44a5400640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f44840ab850 0x7f44840add10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.274+0000 7f44a5400640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44a01075c0 0x7f44a01a12c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.274+0000 7f44a5400640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a0104c90 0x7f44a019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.274+0000 7f44a5400640 1 --2- 172.21.15.97:0/2141401289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a0102360 0x7f44a0199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.274+0000 7f44a5400640 1 -- 172.21.15.97:0/2141401289 >> 172.21.15.97:0/2141401289 conn(0x7f44a00fbd50 msgr2=0x7f44a0101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.274+0000 7f44a5400640 1 -- 172.21.15.97:0/2141401289 shutdown_connections 2024-03-26T19:06:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:16.274+0000 7f44a5400640 1 -- 172.21.15.97:0/2141401289 wait complete. 2024-03-26T19:06:16.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:16 smithi097 ceph-mon[31955]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 706 op/s 2024-03-26T19:06:16.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:16 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:16 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:16 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:16 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:16 smithi144 ceph-mon[36053]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 706 op/s 2024-03-26T19:06:16.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:16 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:16 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:16 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:16 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:16 smithi105 ceph-mon[36270]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 706 op/s 2024-03-26T19:06:16.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:16 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:16 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:16 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:16.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:16 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T19:06:17.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:17 smithi097 ceph-mon[31955]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-26T19:06:17.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:17 smithi097 ceph-mon[31955]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-26T19:06:17.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:06:17.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:06:17.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:06:17.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:17 smithi144 ceph-mon[36053]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-26T19:06:17.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:17 smithi144 ceph-mon[36053]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-26T19:06:17.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:06:17.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:06:17.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:06:17.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:17 smithi105 ceph-mon[36270]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-26T19:06:17.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:17 smithi105 ceph-mon[36270]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-26T19:06:17.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:06:17.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:06:17.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:06:18.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:18 smithi097 ceph-mon[31955]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 MiB/s wr, 579 op/s 2024-03-26T19:06:18.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:18 smithi144 ceph-mon[36053]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 MiB/s wr, 579 op/s 2024-03-26T19:06:18.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:18 smithi105 ceph-mon[36270]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 MiB/s wr, 579 op/s 2024-03-26T19:06:20.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 415.0 s, 2167.4 tps, lat 46.004 ms stddev 55.098 2024-03-26T19:06:20.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:20 smithi097 ceph-mon[31955]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 691 op/s 2024-03-26T19:06:20.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:20 smithi144 ceph-mon[36053]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 691 op/s 2024-03-26T19:06:20.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:20 smithi105 ceph-mon[36270]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 691 op/s 2024-03-26T19:06:22.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:22 smithi097 ceph-mon[31955]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 713 op/s 2024-03-26T19:06:22.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:22 smithi144 ceph-mon[36053]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 713 op/s 2024-03-26T19:06:22.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:22 smithi105 ceph-mon[36270]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 713 op/s 2024-03-26T19:06:24.606 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:24 smithi097 ceph-mon[31955]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 657 op/s 2024-03-26T19:06:24.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:24 smithi144 ceph-mon[36053]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 657 op/s 2024-03-26T19:06:24.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:24 smithi105 ceph-mon[36270]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 657 op/s 2024-03-26T19:06:25.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 420.0 s, 1654.0 tps, lat 61.220 ms stddev 70.403 2024-03-26T19:06:26.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:26 smithi097 ceph-mon[31955]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.3 MiB/s wr, 683 op/s 2024-03-26T19:06:26.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:26 smithi144 ceph-mon[36053]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.3 MiB/s wr, 683 op/s 2024-03-26T19:06:26.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:26 smithi105 ceph-mon[36270]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.3 MiB/s wr, 683 op/s 2024-03-26T19:06:28.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:28 smithi097 ceph-mon[31955]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 607 op/s 2024-03-26T19:06:28.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:28 smithi144 ceph-mon[36053]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 607 op/s 2024-03-26T19:06:28.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:28 smithi105 ceph-mon[36270]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 607 op/s 2024-03-26T19:06:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.462+0000 7f2d57200640 1 -- 172.21.15.97:0/1028063100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d50109070 msgr2=0x7f2d501094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.462+0000 7f2d57200640 1 --2- 172.21.15.97:0/1028063100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d50109070 0x7f2d501094f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009900 tx=0x7f2d4002bc40 comp rx=0 tx=0).stop 2024-03-26T19:06:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.462+0000 7f2d57200640 1 -- 172.21.15.97:0/1028063100 shutdown_connections 2024-03-26T19:06:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.462+0000 7f2d57200640 1 --2- 172.21.15.97:0/1028063100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50109a30 0x7f2d50110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.462+0000 7f2d57200640 1 --2- 172.21.15.97:0/1028063100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d50109070 0x7f2d501094f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.462+0000 7f2d57200640 1 --2- 172.21.15.97:0/1028063100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d50107f20 0x7f2d50108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.462+0000 7f2d57200640 1 -- 172.21.15.97:0/1028063100 >> 172.21.15.97:0/1028063100 conn(0x7f2d50076ed0 msgr2=0x7f2d500792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.463+0000 7f2d57200640 1 -- 172.21.15.97:0/1028063100 shutdown_connections 2024-03-26T19:06:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.463+0000 7f2d57200640 1 -- 172.21.15.97:0/1028063100 wait complete. 2024-03-26T19:06:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.464+0000 7f2d57200640 1 Processor -- start 2024-03-26T19:06:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.464+0000 7f2d57200640 1 -- start start 2024-03-26T19:06:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d57200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d50107f20 0x7f2d5019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d57200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50109070 0x7f2d5019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d57200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d50109a30 0x7f2d501a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d57200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d50113c00 con 0x7f2d50109070 2024-03-26T19:06:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d57200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d50113a80 con 0x7f2d50109a30 2024-03-26T19:06:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d57200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d50113d80 con 0x7f2d50107f20 2024-03-26T19:06:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d4fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d50107f20 0x7f2d5019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d4fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d50107f20 0x7f2d5019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56908/0 (socket says 172.21.15.97:56908) 2024-03-26T19:06:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d4fe00640 1 -- 172.21.15.97:0/336552225 learned_addr learned my addr 172.21.15.97:0/336552225 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d4f400640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50109070 0x7f2d5019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d55200640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d50109a30 0x7f2d501a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d4fe00640 1 -- 172.21.15.97:0/336552225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d50109a30 msgr2=0x7f2d501a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d50109a30 0x7f2d501a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d4fe00640 1 -- 172.21.15.97:0/336552225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50109070 msgr2=0x7f2d5019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50109070 0x7f2d5019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d4fe00640 1 -- 172.21.15.97:0/336552225 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d501a5cb0 con 0x7f2d50107f20 2024-03-26T19:06:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.465+0000 7f2d55200640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d50109a30 0x7f2d501a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:06:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.466+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d50107f20 0x7f2d5019e090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00c920 tx=0x7f2d3c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.466+0000 7f2d4cc00640 1 -- 172.21.15.97:0/336552225 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c007c70 con 0x7f2d50107f20 2024-03-26T19:06:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.466+0000 7f2d4cc00640 1 -- 172.21.15.97:0/336552225 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3c007e10 con 0x7f2d50107f20 2024-03-26T19:06:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.466+0000 7f2d4cc00640 1 -- 172.21.15.97:0/336552225 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c018890 con 0x7f2d50107f20 2024-03-26T19:06:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.466+0000 7f2d57200640 1 -- 172.21.15.97:0/336552225 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d501a5fa0 con 0x7f2d50107f20 2024-03-26T19:06:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.466+0000 7f2d57200640 1 -- 172.21.15.97:0/336552225 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d501a63b0 con 0x7f2d50107f20 2024-03-26T19:06:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.472+0000 7f2d59fd0740 1 Processor -- start 2024-03-26T19:06:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.472+0000 7f2d59fd0740 1 -- start start 2024-03-26T19:06:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253dda0 0x5599f253e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253e790 0x5599f2543290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 0x5599f2545d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5599f24a2e80 con 0x5599f253dda0 2024-03-26T19:06:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5599f244b660 con 0x5599f2543900 2024-03-26T19:06:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5599f249ee30 con 0x5599f253e790 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d55200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253dda0 0x5599f253e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d4f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 0x5599f2545d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d55200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253dda0 0x5599f253e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40820/0 (socket says 172.21.15.97:40820) 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d55200640 1 -- 172.21.15.97:0/2422307243 learned_addr learned my addr 172.21.15.97:0/2422307243 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d55200640 1 -- 172.21.15.97:0/2422307243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253e790 msgr2=0x5599f2543290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d55200640 1 --2- 172.21.15.97:0/2422307243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253e790 0x5599f2543290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d55200640 1 -- 172.21.15.97:0/2422307243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 msgr2=0x5599f2545d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d55200640 1 --2- 172.21.15.97:0/2422307243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 0x5599f2545d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.473+0000 7f2d55200640 1 -- 172.21.15.97:0/2422307243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599f21b7210 con 0x5599f253dda0 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.474+0000 7f2d55200640 1 --2- 172.21.15.97:0/2422307243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253dda0 0x5599f253e1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d48008e60 tx=0x7f2d4800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96898235e163ab91 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.474+0000 7f2d4cc00640 1 -- 172.21.15.97:0/336552225 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f2d3c018a70 con 0x7f2d50107f20 2024-03-26T19:06:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.474+0000 7f2d45600640 1 -- 172.21.15.97:0/2422307243 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800f830 con 0x5599f253dda0 2024-03-26T19:06:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.474+0000 7f2d45600640 1 -- 172.21.15.97:0/2422307243 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4800e3b0 con 0x5599f253dda0 2024-03-26T19:06:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.474+0000 7f2d45600640 1 -- 172.21.15.97:0/2422307243 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800b5e0 con 0x5599f253dda0 2024-03-26T19:06:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.475+0000 7f2d59fd0740 1 -- 172.21.15.97:0/2422307243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253dda0 msgr2=0x5599f253e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.475+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/2422307243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253dda0 0x5599f253e1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d48008e60 tx=0x7f2d4800d710 comp rx=0 tx=0).stop 2024-03-26T19:06:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.475+0000 7f2d4cc00640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2d340a3090 0x7f2d340a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.476+0000 7f2d4cc00640 1 -- 172.21.15.97:0/336552225 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2d3c0cf510 con 0x7f2d50107f20 2024-03-26T19:06:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.476+0000 7f2d59fd0740 1 -- 172.21.15.97:0/2422307243 shutdown_connections 2024-03-26T19:06:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.476+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/2422307243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 0x5599f2545d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.476+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/2422307243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253e790 0x5599f2543290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.476+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/2422307243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253dda0 0x5599f253e1c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.476+0000 7f2d59fd0740 1 -- 172.21.15.97:0/2422307243 >> 172.21.15.97:0/2422307243 conn(0x5599f2539310 msgr2=0x5599f2539710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.477+0000 7f2d55200640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2d340a3090 0x7f2d340a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.477+0000 7f2d59fd0740 1 -- 172.21.15.97:0/2422307243 shutdown_connections 2024-03-26T19:06:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.477+0000 7f2d59fd0740 1 -- 172.21.15.97:0/2422307243 wait complete. 2024-03-26T19:06:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.478+0000 7f2d59fd0740 1 -- start start 2024-03-26T19:06:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d55200640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2d340a3090 0x7f2d340a5550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800dc10 tx=0x7f2d4800df60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d59fd0740 1 Processor -- start 2024-03-26T19:06:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 0x5599f2547da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253e790 0x5599f25482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f2543900 0x5599f254f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5599f25468b0 con 0x5599f2543900 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5599f245eac0 con 0x5599f253e790 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5599f2546a30 con 0x5599f253dda0 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d59fd0740 10 client.0 _ll_register_callbacks cb 0x5599f245e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d4f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253e790 0x5599f25482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d4f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253e790 0x5599f25482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:33128/0 (socket says 172.21.15.97:33128) 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.479+0000 7f2d4f400640 1 -- 172.21.15.97:0/625276375 learned_addr learned my addr 172.21.15.97:0/625276375 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.480+0000 7f2d55200640 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f2543900 0x5599f254f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.480+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 0x5599f2547da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.480+0000 7f2d4f400640 1 -- 172.21.15.97:0/625276375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 msgr2=0x5599f2547da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.480+0000 7f2d4f400640 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 0x5599f2547da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.480+0000 7f2d4f400640 1 -- 172.21.15.97:0/625276375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f2543900 msgr2=0x5599f254f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.480+0000 7f2d4f400640 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f2543900 0x5599f254f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.480+0000 7f2d4f400640 1 -- 172.21.15.97:0/625276375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599f2283620 con 0x5599f253e790 2024-03-26T19:06:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.480+0000 7f2d55200640 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f2543900 0x5599f254f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:06:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.480+0000 7f2d4f400640 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253e790 0x5599f25482e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2d4002c250 tx=0x7f2d40004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2328fa5c21f619f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.480+0000 7f2d3a000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:06:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.481+0000 7f2d3a000640 1 -- 172.21.15.97:0/625276375 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d400046c0 con 0x5599f253e790 2024-03-26T19:06:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.481+0000 7f2d3a000640 1 -- 172.21.15.97:0/625276375 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2d40004860 con 0x5599f253e790 2024-03-26T19:06:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.481+0000 7f2d59fd0740 1 -- 172.21.15.97:0/625276375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d4800e3b0 con 0x5599f253e790 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.481+0000 7f2d3a000640 1 -- 172.21.15.97:0/625276375 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d40004a60 con 0x5599f253e790 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.481+0000 7f2d59fd0740 10 client.25835 fetch_fsmap learned FSMap version 27 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.481+0000 7f2d59fd0740 1 -- 172.21.15.97:0/625276375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599f2552700 con 0x5599f253e790 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.482+0000 7f2d3a000640 1 -- 172.21.15.97:0/625276375 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4800e3b0 con 0x5599f253e790 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.483+0000 7f2d3a000640 1 -- 172.21.15.97:0/625276375 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2d400037f0 con 0x5599f253e790 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.483+0000 7f2d59fd0740 10 client.25835 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.483+0000 7f2d59fd0740 10 client.25835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.483+0000 7f2d59fd0740 20 client.25835 populate_metadata read hostname 'smithi097' 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.483+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5599f25542a0 0x5599f2556700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.483+0000 7f2d59fd0740 4 client.25835 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:06:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.483+0000 7f2d59fd0740 1 -- 172.21.15.97:0/625276375 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599f24b03c0 con 0x5599f25542a0 2024-03-26T19:06:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.483+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5599f25542a0 0x5599f2556700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.484+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5599f25542a0 0x5599f2556700 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa075ee80bb5f9aa server_cookie=f9118381755f198d in_seq=0 out_seq=0 2024-03-26T19:06:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.484+0000 7f2d3a000640 10 client.25835 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.485+0000 7f2d3a000640 1 -- 172.21.15.97:0/625276375 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5599f24b03c0 con 0x5599f25542a0 2024-03-26T19:06:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.485+0000 7f2d3a000640 10 client.25835 handle_command_reply: tid=0 2024-03-26T19:06:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.485+0000 7f2d59fd0740 1 client.25835 shutdown 2024-03-26T19:06:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.486+0000 7f2d59fd0740 1 -- 172.21.15.97:0/625276375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253e790 msgr2=0x5599f25482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.486+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253e790 0x5599f25482e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2d4002c250 tx=0x7f2d40004270 comp rx=0 tx=0).stop 2024-03-26T19:06:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.486+0000 7f2d59fd0740 1 -- 172.21.15.97:0/625276375 shutdown_connections 2024-03-26T19:06:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.486+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5599f25542a0 0x5599f2556700 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.486+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f2543900 0x5599f254f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.486+0000 7f2d3a000640 0 client.25835 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.486+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253e790 0x5599f25482e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.486+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/625276375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 0x5599f2547da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.486+0000 7f2d59fd0740 1 -- 172.21.15.97:0/625276375 >> 172.21.15.97:0/625276375 conn(0x5599f2539310 msgr2=0x5599f2538540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.487+0000 7f2d59fd0740 1 -- 172.21.15.97:0/625276375 shutdown_connections 2024-03-26T19:06:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.487+0000 7f2d59fd0740 1 -- 172.21.15.97:0/625276375 wait complete. 2024-03-26T19:06:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.487+0000 7f2d59fd0740 20 client.25835 trim_cache size 0 max 16384 2024-03-26T19:06:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.521+0000 7f2d59fd0740 1 Processor -- start 2024-03-26T19:06:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.521+0000 7f2d59fd0740 1 -- start start 2024-03-26T19:06:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.522+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253dda0 0x5599f253f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.522+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 0x5599f254f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.522+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f2543900 0x5599f253f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.522+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501af1a0 con 0x5599f253e790 2024-03-26T19:06:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.522+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501b0ce0 con 0x5599f253dda0 2024-03-26T19:06:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.522+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501af870 con 0x5599f2543900 2024-03-26T19:06:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.522+0000 7f2d4fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253dda0 0x5599f253f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.523+0000 7f2d4fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253dda0 0x5599f253f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:33138/0 (socket says 172.21.15.97:33138) 2024-03-26T19:06:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.523+0000 7f2d4fe00640 1 -- 172.21.15.97:0/3277063172 learned_addr learned my addr 172.21.15.97:0/3277063172 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.523+0000 7f2d55200640 1 --2- 172.21.15.97:0/3277063172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f2543900 0x5599f253f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.523+0000 7f2d4fe00640 1 -- 172.21.15.97:0/3277063172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f2543900 msgr2=0x5599f253f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.523+0000 7f2d4f400640 1 --2- 172.21.15.97:0/3277063172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 0x5599f254f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.523+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/3277063172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f2543900 0x5599f253f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.523+0000 7f2d4fe00640 1 -- 172.21.15.97:0/3277063172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 msgr2=0x5599f254f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.523+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/3277063172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 0x5599f254f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.523+0000 7f2d4fe00640 1 -- 172.21.15.97:0/3277063172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599f2546e60 con 0x5599f253dda0 2024-03-26T19:06:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.523+0000 7f2d4f400640 1 --2- 172.21.15.97:0/3277063172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 0x5599f254f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:06:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.524+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/3277063172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253dda0 0x5599f253f230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c09bc00 tx=0x7f2d3c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e869036ceb6e40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.524+0000 7f2d44c00640 1 -- 172.21.15.97:0/3277063172 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c0a4680 con 0x5599f253dda0 2024-03-26T19:06:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.524+0000 7f2d44c00640 1 -- 172.21.15.97:0/3277063172 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3c0cd150 con 0x5599f253dda0 2024-03-26T19:06:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.524+0000 7f2d44c00640 1 -- 172.21.15.97:0/3277063172 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c00fc50 con 0x5599f253dda0 2024-03-26T19:06:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.526+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3277063172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253dda0 msgr2=0x5599f253f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.526+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/3277063172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253dda0 0x5599f253f230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c09bc00 tx=0x7f2d3c000bc0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.528+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3277063172 shutdown_connections 2024-03-26T19:06:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.528+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/3277063172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f2543900 0x5599f253f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.528+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/3277063172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 0x5599f254f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.528+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/3277063172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f253dda0 0x5599f253f230 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.528+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3277063172 >> 172.21.15.97:0/3277063172 conn(0x5599f2539310 msgr2=0x5599f253c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.528+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3277063172 shutdown_connections 2024-03-26T19:06:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.528+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3277063172 wait complete. 2024-03-26T19:06:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.528+0000 7f2d59fd0740 1 -- start start 2024-03-26T19:06:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.529+0000 7f2d59fd0740 1 Processor -- start 2024-03-26T19:06:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.529+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 0x5599f254c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.529+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 0x5599f254cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.529+0000 7f2d59fd0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 0x5599f254dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.529+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501b0610 con 0x5599f253e790 2024-03-26T19:06:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.529+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501aff40 con 0x5599f2543900 2024-03-26T19:06:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.529+0000 7f2d59fd0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501ad180 con 0x5599f253dda0 2024-03-26T19:06:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.529+0000 7f2d59fd0740 10 client.0 _ll_register_callbacks cb 0x7f2d4002cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:06:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d4fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 0x5599f254c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d4fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 0x5599f254c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:56954/0 (socket says 172.21.15.97:56954) 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d4fe00640 1 -- 172.21.15.97:0/3528960599 learned_addr learned my addr 172.21.15.97:0/3528960599 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d55200640 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 0x5599f254dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d4f400640 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 0x5599f254cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d55200640 1 -- 172.21.15.97:0/3528960599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 msgr2=0x5599f254c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d55200640 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 0x5599f254c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d55200640 1 -- 172.21.15.97:0/3528960599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 msgr2=0x5599f254cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d55200640 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 0x5599f254cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d55200640 1 -- 172.21.15.97:0/3528960599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599f254e310 con 0x5599f2543900 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.530+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 0x5599f254c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.531+0000 7f2d55200640 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 0x5599f254dd20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d48008e30 tx=0x7f2d48008730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca1d8273865215a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.531+0000 7f2d3a000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:06:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.531+0000 7f2d3a000640 1 -- 172.21.15.97:0/3528960599 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48008970 con 0x5599f2543900 2024-03-26T19:06:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.531+0000 7f2d3a000640 1 -- 172.21.15.97:0/3528960599 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2d4800b800 con 0x5599f2543900 2024-03-26T19:06:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.531+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3528960599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d3c0cd150 con 0x5599f2543900 2024-03-26T19:06:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.532+0000 7f2d3a000640 1 -- 172.21.15.97:0/3528960599 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d4800ba00 con 0x5599f2543900 2024-03-26T19:06:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.533+0000 7f2d59fd0740 10 client.25841 fetch_fsmap learned FSMap version 27 2024-03-26T19:06:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.533+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3528960599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599f254f590 con 0x5599f2543900 2024-03-26T19:06:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.533+0000 7f2d3a000640 1 -- 172.21.15.97:0/3528960599 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3c0cd150 con 0x5599f2543900 2024-03-26T19:06:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.534+0000 7f2d3a000640 1 -- 172.21.15.97:0/3528960599 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2d48005050 con 0x5599f2543900 2024-03-26T19:06:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.534+0000 7f2d59fd0740 10 client.25841 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:06:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.534+0000 7f2d59fd0740 10 client.25841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:06:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.534+0000 7f2d59fd0740 20 client.25841 populate_metadata read hostname 'smithi097' 2024-03-26T19:06:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.534+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5599f25baca0 0x5599f25bd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.534+0000 7f2d59fd0740 4 client.25841 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:06:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.534+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3528960599 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599f24aa710 con 0x5599f25baca0 2024-03-26T19:06:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.534+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5599f25baca0 0x5599f25bd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.536+0000 7f2d4fe00640 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5599f25baca0 0x5599f25bd0c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769aa8c8ed156701 server_cookie=65167eb23b576563 in_seq=0 out_seq=0 2024-03-26T19:06:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.536+0000 7f2d3a000640 10 client.25841 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d3a000640 1 -- 172.21.15.97:0/3528960599 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599f24aa710 con 0x5599f25baca0 2024-03-26T19:06:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d3a000640 10 client.25841 handle_command_reply: tid=0 2024-03-26T19:06:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d59fd0740 1 client.25841 shutdown 2024-03-26T19:06:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3528960599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 msgr2=0x5599f254dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 0x5599f254dd20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d48008e30 tx=0x7f2d48008730 comp rx=0 tx=0).stop 2024-03-26T19:06:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3528960599 shutdown_connections 2024-03-26T19:06:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5599f25baca0 0x5599f25bd0c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599f2543900 0x5599f254dd20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599f253e790 0x5599f254cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d59fd0740 1 --2- 172.21.15.97:0/3528960599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599f253dda0 0x5599f254c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d3a000640 0 client.25841 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3528960599 >> 172.21.15.97:0/3528960599 conn(0x5599f2539310 msgr2=0x5599f2537cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.557+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3528960599 shutdown_connections 2024-03-26T19:06:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.558+0000 7f2d59fd0740 1 -- 172.21.15.97:0/3528960599 wait complete. 2024-03-26T19:06:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.558+0000 7f2d59fd0740 20 client.25841 trim_cache size 0 max 16384 2024-03-26T19:06:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:06:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:06:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:06:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:06:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.560+0000 7f2d57200640 1 -- 172.21.15.97:0/336552225 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2d340a3090 msgr2=0x7f2d340a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.560+0000 7f2d57200640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2d340a3090 0x7f2d340a5550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800dc10 tx=0x7f2d4800df60 comp rx=0 tx=0).stop 2024-03-26T19:06:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.560+0000 7f2d57200640 1 -- 172.21.15.97:0/336552225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d50107f20 msgr2=0x7f2d5019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.560+0000 7f2d57200640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d50107f20 0x7f2d5019e090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00c920 tx=0x7f2d3c00cde0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.560+0000 7f2d57200640 1 -- 172.21.15.97:0/336552225 shutdown_connections 2024-03-26T19:06:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.560+0000 7f2d57200640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2d340a3090 0x7f2d340a5550 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.560+0000 7f2d57200640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d50109a30 0x7f2d501a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.560+0000 7f2d57200640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50109070 0x7f2d5019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.560+0000 7f2d57200640 1 --2- 172.21.15.97:0/336552225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d50107f20 0x7f2d5019e090 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.560+0000 7f2d57200640 1 -- 172.21.15.97:0/336552225 >> 172.21.15.97:0/336552225 conn(0x7f2d50076ed0 msgr2=0x7f2d5010c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.561+0000 7f2d57200640 1 -- 172.21.15.97:0/336552225 shutdown_connections 2024-03-26T19:06:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:29.561+0000 7f2d57200640 1 -- 172.21.15.97:0/336552225 wait complete. 2024-03-26T19:06:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 425.0 s, 1617.0 tps, lat 61.656 ms stddev 73.994 2024-03-26T19:06:30.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:30 smithi097 ceph-mon[31955]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 672 op/s 2024-03-26T19:06:30.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:30 smithi144 ceph-mon[36053]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 672 op/s 2024-03-26T19:06:30.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:30 smithi105 ceph-mon[36270]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 672 op/s 2024-03-26T19:06:32.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:32 smithi097 ceph-mon[31955]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.3 MiB/s wr, 673 op/s 2024-03-26T19:06:32.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:32 smithi144 ceph-mon[36053]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.3 MiB/s wr, 673 op/s 2024-03-26T19:06:32.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:32 smithi105 ceph-mon[36270]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.3 MiB/s wr, 673 op/s 2024-03-26T19:06:34.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:34 smithi097 ceph-mon[31955]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 MiB/s wr, 636 op/s 2024-03-26T19:06:34.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:34 smithi144 ceph-mon[36053]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 MiB/s wr, 636 op/s 2024-03-26T19:06:34.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:34 smithi105 ceph-mon[36270]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 MiB/s wr, 636 op/s 2024-03-26T19:06:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 430.0 s, 1537.6 tps, lat 64.540 ms stddev 78.213 2024-03-26T19:06:36.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:36 smithi097 ceph-mon[31955]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 669 op/s 2024-03-26T19:06:36.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:36 smithi144 ceph-mon[36053]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 669 op/s 2024-03-26T19:06:36.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:36 smithi105 ceph-mon[36270]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 669 op/s 2024-03-26T19:06:38.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:38 smithi097 ceph-mon[31955]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 MiB/s wr, 633 op/s 2024-03-26T19:06:38.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:38 smithi144 ceph-mon[36053]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 MiB/s wr, 633 op/s 2024-03-26T19:06:38.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:38 smithi105 ceph-mon[36270]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 MiB/s wr, 633 op/s 2024-03-26T19:06:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 435.0 s, 1633.8 tps, lat 61.589 ms stddev 70.487 2024-03-26T19:06:40.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:40 smithi097 ceph-mon[31955]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 669 op/s 2024-03-26T19:06:40.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:40 smithi144 ceph-mon[36053]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 669 op/s 2024-03-26T19:06:40.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:40 smithi105 ceph-mon[36270]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 669 op/s 2024-03-26T19:06:42.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:42 smithi097 ceph-mon[31955]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 675 op/s 2024-03-26T19:06:42.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:42 smithi144 ceph-mon[36053]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 675 op/s 2024-03-26T19:06:42.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:42 smithi105 ceph-mon[36270]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 675 op/s 2024-03-26T19:06:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.731+0000 7f26bb600640 1 -- 172.21.15.97:0/760532821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b40758a0 msgr2=0x7f26b4073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.731+0000 7f26bb600640 1 --2- 172.21.15.97:0/760532821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b40758a0 0x7f26b4073cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f269c009900 tx=0x7f269c02bcd0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.731+0000 7f26bb600640 1 -- 172.21.15.97:0/760532821 shutdown_connections 2024-03-26T19:06:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.731+0000 7f26bb600640 1 --2- 172.21.15.97:0/760532821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b4109780 0x7f26b410bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.731+0000 7f26bb600640 1 --2- 172.21.15.97:0/760532821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40742c0 0x7f26b4074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.731+0000 7f26bb600640 1 --2- 172.21.15.97:0/760532821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b40758a0 0x7f26b4073cf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.731+0000 7f26bb600640 1 -- 172.21.15.97:0/760532821 >> 172.21.15.97:0/760532821 conn(0x7f26b40fce80 msgr2=0x7f26b40ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.732+0000 7f26bb600640 1 -- 172.21.15.97:0/760532821 shutdown_connections 2024-03-26T19:06:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.732+0000 7f26bb600640 1 -- 172.21.15.97:0/760532821 wait complete. 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26bb600640 1 Processor -- start 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26bb600640 1 -- start start 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26bb600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40742c0 0x7f26b4199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26bb600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b40758a0 0x7f26b419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26bb600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b4109780 0x7f26b41a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26bb600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26b410f790 con 0x7f26b40742c0 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26bb600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26b410f610 con 0x7f26b40758a0 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26bb600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26b410f910 con 0x7f26b4109780 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26b9600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b4109780 0x7f26b41a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26b3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b40758a0 0x7f26b419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26b9600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b4109780 0x7f26b41a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34912/0 (socket says 172.21.15.97:34912) 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.733+0000 7f26b9600640 1 -- 172.21.15.97:0/3593015077 learned_addr learned my addr 172.21.15.97:0/3593015077 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b8c00640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40742c0 0x7f26b4199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b9600640 1 -- 172.21.15.97:0/3593015077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b40758a0 msgr2=0x7f26b419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b9600640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b40758a0 0x7f26b419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b9600640 1 -- 172.21.15.97:0/3593015077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40742c0 msgr2=0x7f26b4199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b9600640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40742c0 0x7f26b4199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b9600640 1 -- 172.21.15.97:0/3593015077 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b41a1940 con 0x7f26b4109780 2024-03-26T19:06:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b3e00640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b40758a0 0x7f26b419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:06:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b9600640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b4109780 0x7f26b41a12a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00e970 tx=0x7f26ac00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b1600640 1 -- 172.21.15.97:0/3593015077 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00ccc0 con 0x7f26b4109780 2024-03-26T19:06:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26bb600640 1 -- 172.21.15.97:0/3593015077 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b41a1c30 con 0x7f26b4109780 2024-03-26T19:06:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b1600640 1 -- 172.21.15.97:0/3593015077 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac00ce60 con 0x7f26b4109780 2024-03-26T19:06:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26b1600640 1 -- 172.21.15.97:0/3593015077 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac010490 con 0x7f26b4109780 2024-03-26T19:06:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.734+0000 7f26bb600640 1 -- 172.21.15.97:0/3593015077 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b41a2150 con 0x7f26b4109780 2024-03-26T19:06:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.739+0000 7f26be376740 1 Processor -- start 2024-03-26T19:06:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.739+0000 7f26b1600640 1 -- 172.21.15.97:0/3593015077 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f26ac005020 con 0x7f26b4109780 2024-03-26T19:06:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.739+0000 7f26be376740 1 -- start start 2024-03-26T19:06:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.740+0000 7f26be376740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983841da0 0x55f9838421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.740+0000 7f26be376740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983842790 0x55f983847290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.740+0000 7f26be376740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983847900 0x55f983849d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.740+0000 7f26be376740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9837a6e80 con 0x55f983841da0 2024-03-26T19:06:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.740+0000 7f26be376740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f98374f660 con 0x55f983842790 2024-03-26T19:06:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.740+0000 7f26be376740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9837a2e30 con 0x55f983847900 2024-03-26T19:06:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.740+0000 7f26b3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983841da0 0x55f9838421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.740+0000 7f26b8c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983842790 0x55f983847290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.740+0000 7f26b8c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983842790 0x55f983847290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:48756/0 (socket says 172.21.15.97:48756) 2024-03-26T19:06:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.740+0000 7f26b8c00640 1 -- 172.21.15.97:0/543286807 learned_addr learned my addr 172.21.15.97:0/543286807 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.741+0000 7f26b3e00640 1 --2- 172.21.15.97:0/543286807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983847900 0x55f983849d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.741+0000 7f26b3e00640 1 -- 172.21.15.97:0/543286807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983847900 msgr2=0x55f983849d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.741+0000 7f26b3e00640 1 --2- 172.21.15.97:0/543286807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983847900 0x55f983849d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.741+0000 7f26b3e00640 1 -- 172.21.15.97:0/543286807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983842790 msgr2=0x55f983847290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.741+0000 7f26b3e00640 1 --2- 172.21.15.97:0/543286807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983842790 0x55f983847290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.741+0000 7f26b3e00640 1 -- 172.21.15.97:0/543286807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9834bb210 con 0x55f983841da0 2024-03-26T19:06:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.741+0000 7f26b3e00640 1 --2- 172.21.15.97:0/543286807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983841da0 0x55f9838421c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005fd0 tx=0x7f26a4011460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da265ce7b0a50352 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.742+0000 7f26a9c00640 1 -- 172.21.15.97:0/543286807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a401a070 con 0x55f983841da0 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.742+0000 7f26a9c00640 1 -- 172.21.15.97:0/543286807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a400fe90 con 0x55f983841da0 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.742+0000 7f26a9c00640 1 -- 172.21.15.97:0/543286807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4004830 con 0x55f983841da0 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.742+0000 7f26be376740 1 -- 172.21.15.97:0/543286807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983841da0 msgr2=0x55f9838421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.742+0000 7f26be376740 1 --2- 172.21.15.97:0/543286807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983841da0 0x55f9838421c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005fd0 tx=0x7f26a4011460 comp rx=0 tx=0).stop 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.742+0000 7f26b1600640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f26980a3090 0x7f26980a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.743+0000 7f26be376740 1 -- 172.21.15.97:0/543286807 shutdown_connections 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.743+0000 7f26be376740 1 --2- 172.21.15.97:0/543286807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983847900 0x55f983849d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.743+0000 7f26be376740 1 --2- 172.21.15.97:0/543286807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983842790 0x55f983847290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.743+0000 7f26be376740 1 --2- 172.21.15.97:0/543286807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983841da0 0x55f9838421c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.743+0000 7f26be376740 1 -- 172.21.15.97:0/543286807 >> 172.21.15.97:0/543286807 conn(0x55f98383d310 msgr2=0x55f98383d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.743+0000 7f26be376740 1 -- 172.21.15.97:0/543286807 shutdown_connections 2024-03-26T19:06:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.743+0000 7f26b1600640 1 -- 172.21.15.97:0/3593015077 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f26ac0cf990 con 0x7f26b4109780 2024-03-26T19:06:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.743+0000 7f26b3e00640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f26980a3090 0x7f26980a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.743+0000 7f26be376740 1 -- 172.21.15.97:0/543286807 wait complete. 2024-03-26T19:06:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.744+0000 7f26be376740 1 -- start start 2024-03-26T19:06:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.744+0000 7f26be376740 1 Processor -- start 2024-03-26T19:06:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.744+0000 7f26b3e00640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f26980a3090 0x7f26980a5550 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f26a4004310 tx=0x7f26a4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.744+0000 7f26be376740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98384bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26be376740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 0x55f98384c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26be376740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 0x55f983853330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26be376740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f98384a8b0 con 0x55f983842790 2024-03-26T19:06:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26be376740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f983762ac0 con 0x55f983847900 2024-03-26T19:06:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26be376740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f98384aa30 con 0x55f983841da0 2024-03-26T19:06:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26be376740 10 client.0 _ll_register_callbacks cb 0x55f983762490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:06:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26b8c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98384bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26b8c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98384bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34934/0 (socket says 172.21.15.97:34934) 2024-03-26T19:06:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26b8c00640 1 -- 172.21.15.97:0/197440749 learned_addr learned my addr 172.21.15.97:0/197440749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26b9600640 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 0x55f983853330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.745+0000 7f26b3e00640 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 0x55f98384c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.746+0000 7f26b8c00640 1 -- 172.21.15.97:0/197440749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 msgr2=0x55f983853330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.746+0000 7f26b8c00640 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 0x55f983853330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.746+0000 7f26b8c00640 1 -- 172.21.15.97:0/197440749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 msgr2=0x55f98384c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.746+0000 7f26b8c00640 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 0x55f98384c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.746+0000 7f26b8c00640 1 -- 172.21.15.97:0/197440749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f983587620 con 0x55f983841da0 2024-03-26T19:06:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.746+0000 7f26b9600640 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 0x55f983853330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:06:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.746+0000 7f26b8c00640 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98384bdb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f269c036040 tx=0x7f269c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0709953df20e35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.747+0000 7f26a2000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:06:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.747+0000 7f26a2000640 1 -- 172.21.15.97:0/197440749 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c042070 con 0x55f983841da0 2024-03-26T19:06:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.747+0000 7f26a2000640 1 -- 172.21.15.97:0/197440749 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f269c005030 con 0x55f983841da0 2024-03-26T19:06:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.747+0000 7f26be376740 1 -- 172.21.15.97:0/197440749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a400fe90 con 0x55f983841da0 2024-03-26T19:06:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.748+0000 7f26a2000640 1 -- 172.21.15.97:0/197440749 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f269c04c050 con 0x55f983841da0 2024-03-26T19:06:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.748+0000 7f26be376740 10 client.25807 fetch_fsmap learned FSMap version 27 2024-03-26T19:06:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.748+0000 7f26be376740 1 -- 172.21.15.97:0/197440749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f98385ef70 con 0x55f983841da0 2024-03-26T19:06:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.748+0000 7f26a2000640 1 -- 172.21.15.97:0/197440749 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a400fe90 con 0x55f983841da0 2024-03-26T19:06:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.749+0000 7f26a2000640 1 -- 172.21.15.97:0/197440749 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f269c004b00 con 0x55f983841da0 2024-03-26T19:06:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.749+0000 7f26be376740 10 client.25807 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:06:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.749+0000 7f26be376740 10 client.25807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:06:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.749+0000 7f26be376740 20 client.25807 populate_metadata read hostname 'smithi097' 2024-03-26T19:06:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.749+0000 7f26be376740 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f983860640 0x55f983862a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.749+0000 7f26be376740 4 client.25807 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:06:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.749+0000 7f26be376740 1 -- 172.21.15.97:0/197440749 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9837b43c0 con 0x55f983860640 2024-03-26T19:06:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.750+0000 7f26b3e00640 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f983860640 0x55f983862a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.750+0000 7f26b3e00640 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f983860640 0x55f983862a90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e334cfff33b757e7 server_cookie=1c9c1ab226c96e19 in_seq=0 out_seq=0 2024-03-26T19:06:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.751+0000 7f26a2000640 10 client.25807 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.751+0000 7f26a2000640 1 -- 172.21.15.97:0/197440749 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f9837b43c0 con 0x55f983860640 2024-03-26T19:06:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.751+0000 7f26a2000640 10 client.25807 handle_command_reply: tid=0 2024-03-26T19:06:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.751+0000 7f26be376740 1 client.25807 shutdown 2024-03-26T19:06:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.753+0000 7f26be376740 1 -- 172.21.15.97:0/197440749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 msgr2=0x55f98384bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.753+0000 7f26be376740 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98384bdb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f269c036040 tx=0x7f269c004270 comp rx=0 tx=0).stop 2024-03-26T19:06:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.754+0000 7f26be376740 1 -- 172.21.15.97:0/197440749 shutdown_connections 2024-03-26T19:06:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.754+0000 7f26be376740 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f983860640 0x55f983862a90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.754+0000 7f26be376740 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 0x55f983853330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.754+0000 7f26a2000640 0 client.25807 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.754+0000 7f26be376740 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 0x55f98384c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.754+0000 7f26be376740 1 --2- 172.21.15.97:0/197440749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98384bdb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.754+0000 7f26be376740 1 -- 172.21.15.97:0/197440749 >> 172.21.15.97:0/197440749 conn(0x55f98383d310 msgr2=0x55f98383c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.754+0000 7f26be376740 1 -- 172.21.15.97:0/197440749 shutdown_connections 2024-03-26T19:06:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.754+0000 7f26be376740 1 -- 172.21.15.97:0/197440749 wait complete. 2024-03-26T19:06:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.754+0000 7f26be376740 20 client.25807 trim_cache size 0 max 16384 2024-03-26T19:06:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.783+0000 7f26be376740 1 Processor -- start 2024-03-26T19:06:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.783+0000 7f26be376740 1 -- start start 2024-03-26T19:06:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26be376740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98383c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26be376740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 0x55f98384b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26be376740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 0x55f98384b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26be376740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41a6a70 con 0x55f983842790 2024-03-26T19:06:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26be376740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41a85b0 con 0x55f983847900 2024-03-26T19:06:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26be376740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41a7140 con 0x55f983841da0 2024-03-26T19:06:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26b8c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98383c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26b8c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98383c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34942/0 (socket says 172.21.15.97:34942) 2024-03-26T19:06:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26b8c00640 1 -- 172.21.15.97:0/2114298751 learned_addr learned my addr 172.21.15.97:0/2114298751 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26b9600640 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 0x55f98384b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26b3e00640 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 0x55f98384b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.784+0000 7f26b8c00640 1 -- 172.21.15.97:0/2114298751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 msgr2=0x55f98384b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.785+0000 7f26b8c00640 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 0x55f98384b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.785+0000 7f26b8c00640 1 -- 172.21.15.97:0/2114298751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 msgr2=0x55f98384b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.785+0000 7f26b8c00640 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 0x55f98384b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.785+0000 7f26b8c00640 1 -- 172.21.15.97:0/2114298751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f98384ab30 con 0x55f983841da0 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.785+0000 7f26b9600640 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 0x55f98384b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.785+0000 7f26b3e00640 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 0x55f98384b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.785+0000 7f26b8c00640 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98383c3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f269c004ea0 tx=0x7f269c001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c62a4f95eef81f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.785+0000 7f26a9200640 1 -- 172.21.15.97:0/2114298751 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c042070 con 0x55f983841da0 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.786+0000 7f26a9200640 1 -- 172.21.15.97:0/2114298751 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f269c009520 con 0x55f983841da0 2024-03-26T19:06:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.786+0000 7f26a9200640 1 -- 172.21.15.97:0/2114298751 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c0046c0 con 0x55f983841da0 2024-03-26T19:06:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 -- 172.21.15.97:0/2114298751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 msgr2=0x55f98383c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98383c3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f269c004ea0 tx=0x7f269c001e70 comp rx=0 tx=0).stop 2024-03-26T19:06:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 -- 172.21.15.97:0/2114298751 shutdown_connections 2024-03-26T19:06:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983847900 0x55f98384b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983842790 0x55f98384b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 --2- 172.21.15.97:0/2114298751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983841da0 0x55f98383c3c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 -- 172.21.15.97:0/2114298751 >> 172.21.15.97:0/2114298751 conn(0x55f98383d310 msgr2=0x55f98383bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 -- 172.21.15.97:0/2114298751 shutdown_connections 2024-03-26T19:06:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 -- 172.21.15.97:0/2114298751 wait complete. 2024-03-26T19:06:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 -- start start 2024-03-26T19:06:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.787+0000 7f26be376740 1 Processor -- start 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.788+0000 7f26be376740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983841da0 0x55f98384e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.788+0000 7f26be376740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983842790 0x55f98384ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.788+0000 7f26be376740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983847900 0x55f983851be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.788+0000 7f26be376740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41a7ee0 con 0x55f983847900 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.788+0000 7f26be376740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41a7810 con 0x55f983841da0 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.788+0000 7f26be376740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41a49f0 con 0x55f983842790 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.788+0000 7f26be376740 10 client.0 _ll_register_callbacks cb 0x7f269c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.789+0000 7f26b3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983842790 0x55f98384ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.789+0000 7f26b3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983842790 0x55f98384ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:34952/0 (socket says 172.21.15.97:34952) 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.789+0000 7f26b3e00640 1 -- 172.21.15.97:0/3552480822 learned_addr learned my addr 172.21.15.97:0/3552480822 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.789+0000 7f26b9600640 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983847900 0x55f983851be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.789+0000 7f26b8c00640 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983841da0 0x55f98384e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.790+0000 7f26b3e00640 1 -- 172.21.15.97:0/3552480822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983841da0 msgr2=0x55f98384e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.790+0000 7f26b3e00640 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983841da0 0x55f98384e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.790+0000 7f26b3e00640 1 -- 172.21.15.97:0/3552480822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983847900 msgr2=0x55f983851be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.790+0000 7f26b3e00640 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983847900 0x55f983851be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.790+0000 7f26b3e00640 1 -- 172.21.15.97:0/3552480822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9838521d0 con 0x55f983842790 2024-03-26T19:06:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.790+0000 7f26b9600640 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983847900 0x55f983851be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:06:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.791+0000 7f26b3e00640 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983842790 0x55f98384ebc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005fd0 tx=0x7f26a4011b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a4110d3c26d70fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.791+0000 7f26a2000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:06:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.794+0000 7f26a2000640 1 -- 172.21.15.97:0/3552480822 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a403e060 con 0x55f983842790 2024-03-26T19:06:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.794+0000 7f26a2000640 1 -- 172.21.15.97:0/3552480822 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f26a4038440 con 0x55f983842790 2024-03-26T19:06:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.794+0000 7f26be376740 1 -- 172.21.15.97:0/3552480822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f269c009520 con 0x55f983842790 2024-03-26T19:06:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.797+0000 7f26a2000640 1 -- 172.21.15.97:0/3552480822 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26a4027070 con 0x55f983842790 2024-03-26T19:06:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.797+0000 7f26be376740 10 client.25813 fetch_fsmap learned FSMap version 27 2024-03-26T19:06:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.798+0000 7f26be376740 1 -- 172.21.15.97:0/3552480822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f9838d79e0 con 0x55f983842790 2024-03-26T19:06:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.798+0000 7f26a2000640 1 -- 172.21.15.97:0/3552480822 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f269c009520 con 0x55f983842790 2024-03-26T19:06:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.798+0000 7f26a2000640 1 -- 172.21.15.97:0/3552480822 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f26a402a050 con 0x55f983842790 2024-03-26T19:06:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.800+0000 7f26be376740 10 client.25813 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:06:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.800+0000 7f26be376740 10 client.25813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:06:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.800+0000 7f26be376740 20 client.25813 populate_metadata read hostname 'smithi097' 2024-03-26T19:06:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.800+0000 7f26be376740 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f9838d80e0 0x55f9838da500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.800+0000 7f26be376740 4 client.25813 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:06:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.800+0000 7f26be376740 1 -- 172.21.15.97:0/3552480822 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9837ae710 con 0x55f9838d80e0 2024-03-26T19:06:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.800+0000 7f26b8c00640 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f9838d80e0 0x55f9838da500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.802+0000 7f26b8c00640 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f9838d80e0 0x55f9838da500 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8f5736c71069d5d server_cookie=6dd6078ec5bdd399 in_seq=0 out_seq=0 2024-03-26T19:06:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.802+0000 7f26a2000640 10 client.25813 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.892+0000 7f26a2000640 1 -- 172.21.15.97:0/3552480822 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9837ae710 con 0x55f9838d80e0 2024-03-26T19:06:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.892+0000 7f26a2000640 10 client.25813 handle_command_reply: tid=0 2024-03-26T19:06:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.892+0000 7f26be376740 1 client.25813 shutdown 2024-03-26T19:06:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.892+0000 7f26be376740 1 -- 172.21.15.97:0/3552480822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983842790 msgr2=0x55f98384ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.893+0000 7f26be376740 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983842790 0x55f98384ebc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005fd0 tx=0x7f26a4011b40 comp rx=0 tx=0).stop 2024-03-26T19:06:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.893+0000 7f26be376740 1 -- 172.21.15.97:0/3552480822 shutdown_connections 2024-03-26T19:06:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.893+0000 7f26be376740 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55f9838d80e0 0x55f9838da500 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.893+0000 7f26be376740 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f983847900 0x55f983851be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.893+0000 7f26be376740 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f983842790 0x55f98384ebc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.893+0000 7f26a2000640 0 client.25813 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.893+0000 7f26be376740 1 --2- 172.21.15.97:0/3552480822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f983841da0 0x55f98384e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.893+0000 7f26be376740 1 -- 172.21.15.97:0/3552480822 >> 172.21.15.97:0/3552480822 conn(0x55f98383d310 msgr2=0x55f98383c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.894+0000 7f26be376740 1 -- 172.21.15.97:0/3552480822 shutdown_connections 2024-03-26T19:06:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.894+0000 7f26be376740 1 -- 172.21.15.97:0/3552480822 wait complete. 2024-03-26T19:06:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.894+0000 7f26be376740 20 client.25813 trim_cache size 0 max 16384 2024-03-26T19:06:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:06:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:06:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:06:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:06:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.896+0000 7f26bb600640 1 -- 172.21.15.97:0/3593015077 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f26980a3090 msgr2=0x7f26980a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.896+0000 7f26bb600640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f26980a3090 0x7f26980a5550 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f26a4004310 tx=0x7f26a4009500 comp rx=0 tx=0).stop 2024-03-26T19:06:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.897+0000 7f26bb600640 1 -- 172.21.15.97:0/3593015077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b4109780 msgr2=0x7f26b41a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.897+0000 7f26bb600640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b4109780 0x7f26b41a12a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00e970 tx=0x7f26ac00ed70 comp rx=0 tx=0).stop 2024-03-26T19:06:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.897+0000 7f26bb600640 1 -- 172.21.15.97:0/3593015077 shutdown_connections 2024-03-26T19:06:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.897+0000 7f26bb600640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f26980a3090 0x7f26980a5550 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.897+0000 7f26bb600640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b4109780 0x7f26b41a12a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.897+0000 7f26bb600640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b40758a0 0x7f26b419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.897+0000 7f26bb600640 1 --2- 172.21.15.97:0/3593015077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40742c0 0x7f26b4199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.897+0000 7f26bb600640 1 -- 172.21.15.97:0/3593015077 >> 172.21.15.97:0/3593015077 conn(0x7f26b40fce80 msgr2=0x7f26b4105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.898+0000 7f26bb600640 1 -- 172.21.15.97:0/3593015077 shutdown_connections 2024-03-26T19:06:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:42.898+0000 7f26bb600640 1 -- 172.21.15.97:0/3593015077 wait complete. 2024-03-26T19:06:44.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:44 smithi097 ceph-mon[31955]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.4 MiB/s wr, 638 op/s 2024-03-26T19:06:44.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:44 smithi144 ceph-mon[36053]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.4 MiB/s wr, 638 op/s 2024-03-26T19:06:44.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:44 smithi105 ceph-mon[36270]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.4 MiB/s wr, 638 op/s 2024-03-26T19:06:45.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 440.0 s, 1807.4 tps, lat 55.030 ms stddev 63.367 2024-03-26T19:06:46.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:46 smithi144 ceph-mon[36053]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 659 op/s 2024-03-26T19:06:46.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:46 smithi105 ceph-mon[36270]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 659 op/s 2024-03-26T19:06:46.857 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:46 smithi097 ceph-mon[31955]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 659 op/s 2024-03-26T19:06:47.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:47 smithi105 ceph-mon[36270]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.5 MiB/s wr, 574 op/s 2024-03-26T19:06:47.857 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:47 smithi097 ceph-mon[31955]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.5 MiB/s wr, 574 op/s 2024-03-26T19:06:47.880 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:47 smithi144 ceph-mon[36053]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.5 MiB/s wr, 574 op/s 2024-03-26T19:06:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 445.0 s, 1740.6 tps, lat 57.900 ms stddev 70.771 2024-03-26T19:06:50.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:50 smithi097 ceph-mon[31955]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.8 MiB/s wr, 632 op/s 2024-03-26T19:06:50.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:50 smithi144 ceph-mon[36053]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.8 MiB/s wr, 632 op/s 2024-03-26T19:06:50.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:50 smithi105 ceph-mon[36270]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.8 MiB/s wr, 632 op/s 2024-03-26T19:06:52.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:52 smithi097 ceph-mon[31955]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.8 MiB/s wr, 647 op/s 2024-03-26T19:06:52.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:52 smithi144 ceph-mon[36053]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.8 MiB/s wr, 647 op/s 2024-03-26T19:06:52.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:52 smithi105 ceph-mon[36270]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.8 MiB/s wr, 647 op/s 2024-03-26T19:06:54.515 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:54 smithi105 ceph-mon[36270]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-03-26T19:06:54.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:54 smithi097 ceph-mon[31955]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-03-26T19:06:54.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:54 smithi144 ceph-mon[36053]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-03-26T19:06:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 450.0 s, 1490.6 tps, lat 67.233 ms stddev 79.559 2024-03-26T19:06:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.114+0000 7f284b800640 1 -- 172.21.15.97:0/1439620074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28441056d0 msgr2=0x7f284410bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.114+0000 7f284b800640 1 --2- 172.21.15.97:0/1439620074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28441056d0 0x7f284410bf60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2834009860 tx=0x7f283402bb70 comp rx=0 tx=0).stop 2024-03-26T19:06:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.115+0000 7f284b800640 1 -- 172.21.15.97:0/1439620074 shutdown_connections 2024-03-26T19:06:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.115+0000 7f284b800640 1 --2- 172.21.15.97:0/1439620074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28441056d0 0x7f284410bf60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.115+0000 7f284b800640 1 --2- 172.21.15.97:0/1439620074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2844104d10 0x7f2844105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.115+0000 7f284b800640 1 --2- 172.21.15.97:0/1439620074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2844103b20 0x7f2844103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.115+0000 7f284b800640 1 -- 172.21.15.97:0/1439620074 >> 172.21.15.97:0/1439620074 conn(0x7f28440ff0b0 msgr2=0x7f28441014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.115+0000 7f284b800640 1 -- 172.21.15.97:0/1439620074 shutdown_connections 2024-03-26T19:06:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.115+0000 7f284b800640 1 -- 172.21.15.97:0/1439620074 wait complete. 2024-03-26T19:06:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.117+0000 7f284b800640 1 Processor -- start 2024-03-26T19:06:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.117+0000 7f284b800640 1 -- start start 2024-03-26T19:06:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.117+0000 7f284b800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2844103b20 0x7f2844195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.117+0000 7f284b800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844104d10 0x7f2844195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.117+0000 7f284b800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28441056d0 0x7f284419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.117+0000 7f284b800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f284410f780 con 0x7f2844104d10 2024-03-26T19:06:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.117+0000 7f284b800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f284410f600 con 0x7f2844103b20 2024-03-26T19:06:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.117+0000 7f284b800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f284410f900 con 0x7f28441056d0 2024-03-26T19:06:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.118+0000 7f2849800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28441056d0 0x7f284419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.118+0000 7f2849800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28441056d0 0x7f284419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37468/0 (socket says 172.21.15.97:37468) 2024-03-26T19:06:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.118+0000 7f2849800640 1 -- 172.21.15.97:0/1259266405 learned_addr learned my addr 172.21.15.97:0/1259266405 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.118+0000 7f2848e00640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2844103b20 0x7f2844195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.118+0000 7f2843e00640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844104d10 0x7f2844195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.118+0000 7f2849800640 1 -- 172.21.15.97:0/1259266405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2844103b20 msgr2=0x7f2844195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.118+0000 7f2849800640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2844103b20 0x7f2844195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.118+0000 7f2849800640 1 -- 172.21.15.97:0/1259266405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844104d10 msgr2=0x7f2844195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.118+0000 7f2849800640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844104d10 0x7f2844195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.118+0000 7f2849800640 1 -- 172.21.15.97:0/1259266405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28441ae680 con 0x7f28441056d0 2024-03-26T19:06:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.119+0000 7f2849800640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28441056d0 0x7f284419cef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2834009780 tx=0x7f2834004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.119+0000 7f2841600640 1 -- 172.21.15.97:0/1259266405 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834041020 con 0x7f28441056d0 2024-03-26T19:06:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.120+0000 7f2841600640 1 -- 172.21.15.97:0/1259266405 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283402c9f0 con 0x7f28441056d0 2024-03-26T19:06:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.120+0000 7f2841600640 1 -- 172.21.15.97:0/1259266405 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834041020 con 0x7f28441056d0 2024-03-26T19:06:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.120+0000 7f284b800640 1 -- 172.21.15.97:0/1259266405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28441ae850 con 0x7f28441056d0 2024-03-26T19:06:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.120+0000 7f284b800640 1 -- 172.21.15.97:0/1259266405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28441aec00 con 0x7f28441056d0 2024-03-26T19:06:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.126+0000 7f2841600640 1 -- 172.21.15.97:0/1259266405 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f2834044020 con 0x7f28441056d0 2024-03-26T19:06:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.128+0000 7f284e578740 1 Processor -- start 2024-03-26T19:06:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.129+0000 7f284e578740 1 -- start start 2024-03-26T19:06:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.130+0000 7f284e578740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d20a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.131+0000 7f2841600640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28280a32a0 0x7f28280a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.131+0000 7f2848e00640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28280a32a0 0x7f28280a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.132+0000 7f2841600640 1 -- 172.21.15.97:0/1259266405 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f28340f0a20 con 0x7f28441056d0 2024-03-26T19:06:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.132+0000 7f284e578740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 0x55ba9d20f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.132+0000 7f284e578740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 0x55ba9d211d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.132+0000 7f284e578740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ba9d16ee80 con 0x55ba9d209da0 2024-03-26T19:06:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.132+0000 7f284e578740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ba9d117660 con 0x55ba9d20a790 2024-03-26T19:06:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.132+0000 7f284e578740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ba9d16ae30 con 0x55ba9d20f900 2024-03-26T19:06:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.133+0000 7f2843e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 0x55ba9d20f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.133+0000 7f2843e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 0x55ba9d20f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:33148/0 (socket says 172.21.15.97:33148) 2024-03-26T19:06:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.133+0000 7f2843e00640 1 -- 172.21.15.97:0/33815569 learned_addr learned my addr 172.21.15.97:0/33815569 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.133+0000 7f2843e00640 1 --2- 172.21.15.97:0/33815569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d20a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.134+0000 7f2849800640 1 --2- 172.21.15.97:0/33815569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 0x55ba9d211d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.134+0000 7f2843e00640 1 -- 172.21.15.97:0/33815569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 msgr2=0x55ba9d211d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.134+0000 7f2843e00640 1 --2- 172.21.15.97:0/33815569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 0x55ba9d211d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.134+0000 7f2843e00640 1 -- 172.21.15.97:0/33815569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 msgr2=0x55ba9d20a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.134+0000 7f2843e00640 1 --2- 172.21.15.97:0/33815569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d20a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.134+0000 7f2843e00640 1 -- 172.21.15.97:0/33815569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba9ce83210 con 0x55ba9d20a790 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.134+0000 7f2848e00640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28280a32a0 0x7f28280a5760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2844104b70 tx=0x7f283c009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.134+0000 7f2843e00640 1 --2- 172.21.15.97:0/33815569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 0x55ba9d20f290 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f283000bf20 tx=0x7f283000b4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b713d71412de9a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.135+0000 7f283a000640 1 -- 172.21.15.97:0/33815569 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830004020 con 0x55ba9d20a790 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.135+0000 7f283a000640 1 -- 172.21.15.97:0/33815569 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283000ecb0 con 0x55ba9d20a790 2024-03-26T19:06:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.135+0000 7f283a000640 1 -- 172.21.15.97:0/33815569 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300038c0 con 0x55ba9d20a790 2024-03-26T19:06:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.136+0000 7f284e578740 1 -- 172.21.15.97:0/33815569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 msgr2=0x55ba9d20f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.136+0000 7f284e578740 1 --2- 172.21.15.97:0/33815569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 0x55ba9d20f290 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f283000bf20 tx=0x7f283000b4e0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.136+0000 7f284e578740 1 -- 172.21.15.97:0/33815569 shutdown_connections 2024-03-26T19:06:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.136+0000 7f284e578740 1 --2- 172.21.15.97:0/33815569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 0x55ba9d211d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.136+0000 7f284e578740 1 --2- 172.21.15.97:0/33815569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 0x55ba9d20f290 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.136+0000 7f284e578740 1 --2- 172.21.15.97:0/33815569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d20a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.136+0000 7f284e578740 1 -- 172.21.15.97:0/33815569 >> 172.21.15.97:0/33815569 conn(0x55ba9d205310 msgr2=0x55ba9d205710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.138+0000 7f284e578740 1 -- 172.21.15.97:0/33815569 shutdown_connections 2024-03-26T19:06:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.138+0000 7f284e578740 1 -- 172.21.15.97:0/33815569 wait complete. 2024-03-26T19:06:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.138+0000 7f284e578740 1 -- start start 2024-03-26T19:06:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.139+0000 7f284e578740 1 Processor -- start 2024-03-26T19:06:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f284e578740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d213e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f284e578740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 0x55ba9d214370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f284e578740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20f900 0x55ba9d21b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f284e578740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ba9d2128b0 con 0x55ba9d209da0 2024-03-26T19:06:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f284e578740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ba9d12aac0 con 0x55ba9d20f900 2024-03-26T19:06:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f284e578740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ba9d212a30 con 0x55ba9d20a790 2024-03-26T19:06:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f284e578740 10 client.0 _ll_register_callbacks cb 0x55ba9d12a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:06:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f2848e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d213e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f2843e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 0x55ba9d214370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f2848e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d213e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48332/0 (socket says 172.21.15.97:48332) 2024-03-26T19:06:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.140+0000 7f2848e00640 1 -- 172.21.15.97:0/710771713 learned_addr learned my addr 172.21.15.97:0/710771713 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.141+0000 7f2849800640 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20f900 0x55ba9d21b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.141+0000 7f2849800640 1 -- 172.21.15.97:0/710771713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 msgr2=0x55ba9d214370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.141+0000 7f2849800640 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 0x55ba9d214370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.141+0000 7f2849800640 1 -- 172.21.15.97:0/710771713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 msgr2=0x55ba9d213e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.141+0000 7f2849800640 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d213e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.141+0000 7f2849800640 1 -- 172.21.15.97:0/710771713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9cf4f620 con 0x55ba9d20f900 2024-03-26T19:06:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.142+0000 7f2848e00640 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d213e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:06:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.142+0000 7f2849800640 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20f900 0x55ba9d21b3b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f283402cbc0 tx=0x7f283402bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2efbcecbef9cde76 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.142+0000 7f282ea00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:06:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.142+0000 7f282ea00640 1 -- 172.21.15.97:0/710771713 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834039040 con 0x55ba9d20f900 2024-03-26T19:06:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.143+0000 7f282ea00640 1 -- 172.21.15.97:0/710771713 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f28340d6160 con 0x55ba9d20f900 2024-03-26T19:06:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.143+0000 7f284e578740 1 -- 172.21.15.97:0/710771713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283000ecb0 con 0x55ba9d20f900 2024-03-26T19:06:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.143+0000 7f282ea00640 1 -- 172.21.15.97:0/710771713 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f28340bd180 con 0x55ba9d20f900 2024-03-26T19:06:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.143+0000 7f284e578740 10 client.25868 fetch_fsmap learned FSMap version 27 2024-03-26T19:06:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.143+0000 7f284e578740 1 -- 172.21.15.97:0/710771713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba9d22f730 con 0x55ba9d20f900 2024-03-26T19:06:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.144+0000 7f282ea00640 1 -- 172.21.15.97:0/710771713 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283000ecb0 con 0x55ba9d20f900 2024-03-26T19:06:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.144+0000 7f282ea00640 1 -- 172.21.15.97:0/710771713 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f283403c030 con 0x55ba9d20f900 2024-03-26T19:06:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.144+0000 7f284e578740 10 client.25868 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:06:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.144+0000 7f284e578740 10 client.25868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:06:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.144+0000 7f284e578740 20 client.25868 populate_metadata read hostname 'smithi097' 2024-03-26T19:06:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.144+0000 7f284e578740 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ba9d230e10 0x55ba9d233220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.144+0000 7f284e578740 4 client.25868 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:06:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.144+0000 7f284e578740 1 -- 172.21.15.97:0/710771713 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba9d17c3c0 con 0x55ba9d230e10 2024-03-26T19:06:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.145+0000 7f2848e00640 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ba9d230e10 0x55ba9d233220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.146+0000 7f2848e00640 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ba9d230e10 0x55ba9d233220 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ef7c6c83e93adb server_cookie=4326f7e4da1ee107 in_seq=0 out_seq=0 2024-03-26T19:06:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.146+0000 7f282ea00640 10 client.25868 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.148+0000 7f282ea00640 1 -- 172.21.15.97:0/710771713 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ba9d17c3c0 con 0x55ba9d230e10 2024-03-26T19:06:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.148+0000 7f282ea00640 10 client.25868 handle_command_reply: tid=0 2024-03-26T19:06:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.148+0000 7f284e578740 1 client.25868 shutdown 2024-03-26T19:06:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.149+0000 7f284e578740 1 -- 172.21.15.97:0/710771713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20f900 msgr2=0x55ba9d21b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.149+0000 7f284e578740 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20f900 0x55ba9d21b3b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f283402cbc0 tx=0x7f283402bb70 comp rx=0 tx=0).stop 2024-03-26T19:06:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.149+0000 7f284e578740 1 -- 172.21.15.97:0/710771713 shutdown_connections 2024-03-26T19:06:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.149+0000 7f284e578740 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ba9d230e10 0x55ba9d233220 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.149+0000 7f284e578740 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20f900 0x55ba9d21b3b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.149+0000 7f284e578740 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 0x55ba9d214370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.149+0000 7f284e578740 1 --2- 172.21.15.97:0/710771713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d213e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.149+0000 7f284e578740 1 -- 172.21.15.97:0/710771713 >> 172.21.15.97:0/710771713 conn(0x55ba9d205310 msgr2=0x55ba9d204540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.150+0000 7f282ea00640 0 client.25868 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.150+0000 7f284e578740 1 -- 172.21.15.97:0/710771713 shutdown_connections 2024-03-26T19:06:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.150+0000 7f284e578740 1 -- 172.21.15.97:0/710771713 wait complete. 2024-03-26T19:06:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.151+0000 7f284e578740 20 client.25868 trim_cache size 0 max 16384 2024-03-26T19:06:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.184+0000 7f284e578740 1 Processor -- start 2024-03-26T19:06:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.184+0000 7f284e578740 1 -- start start 2024-03-26T19:06:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f284e578740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d208c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f284e578740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 0x55ba9d21b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f284e578740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 0x55ba9d203cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f284e578740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28441af330 con 0x55ba9d209da0 2024-03-26T19:06:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f284e578740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28441b0e70 con 0x55ba9d20a790 2024-03-26T19:06:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f284e578740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28441afa00 con 0x55ba9d20f900 2024-03-26T19:06:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f2843e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 0x55ba9d21b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f2849800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 0x55ba9d203cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f2849800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 0x55ba9d203cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37500/0 (socket says 172.21.15.97:37500) 2024-03-26T19:06:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f2849800640 1 -- 172.21.15.97:0/876243751 learned_addr learned my addr 172.21.15.97:0/876243751 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.185+0000 7f2848e00640 1 --2- 172.21.15.97:0/876243751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d208c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.186+0000 7f2849800640 1 -- 172.21.15.97:0/876243751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 msgr2=0x55ba9d21b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.186+0000 7f2849800640 1 --2- 172.21.15.97:0/876243751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 0x55ba9d21b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.186+0000 7f2849800640 1 -- 172.21.15.97:0/876243751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 msgr2=0x55ba9d208c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.186+0000 7f2849800640 1 --2- 172.21.15.97:0/876243751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d208c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.186+0000 7f2849800640 1 -- 172.21.15.97:0/876243751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba9d200730 con 0x55ba9d20f900 2024-03-26T19:06:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.187+0000 7f2849800640 1 --2- 172.21.15.97:0/876243751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 0x55ba9d203cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f28340b95e0 tx=0x7f28340bb690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f362a57b5535d3fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.187+0000 7f2839600640 1 -- 172.21.15.97:0/876243751 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834039070 con 0x55ba9d20f900 2024-03-26T19:06:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.188+0000 7f2839600640 1 -- 172.21.15.97:0/876243751 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28340c5970 con 0x55ba9d20f900 2024-03-26T19:06:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.188+0000 7f2839600640 1 -- 172.21.15.97:0/876243751 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28340dd0b0 con 0x55ba9d20f900 2024-03-26T19:06:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 -- 172.21.15.97:0/876243751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 msgr2=0x55ba9d203cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 --2- 172.21.15.97:0/876243751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 0x55ba9d203cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f28340b95e0 tx=0x7f28340bb690 comp rx=0 tx=0).stop 2024-03-26T19:06:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 -- 172.21.15.97:0/876243751 shutdown_connections 2024-03-26T19:06:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 --2- 172.21.15.97:0/876243751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20f900 0x55ba9d203cb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 --2- 172.21.15.97:0/876243751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d20a790 0x55ba9d21b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 --2- 172.21.15.97:0/876243751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d209da0 0x55ba9d208c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 -- 172.21.15.97:0/876243751 >> 172.21.15.97:0/876243751 conn(0x55ba9d205310 msgr2=0x55ba9d208570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 -- 172.21.15.97:0/876243751 shutdown_connections 2024-03-26T19:06:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 -- 172.21.15.97:0/876243751 wait complete. 2024-03-26T19:06:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 -- start start 2024-03-26T19:06:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.189+0000 7f284e578740 1 Processor -- start 2024-03-26T19:06:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.190+0000 7f284e578740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d209da0 0x55ba9d2186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.190+0000 7f284e578740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 0x55ba9d218c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.190+0000 7f284e578740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d20f900 0x55ba9d219c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.190+0000 7f284e578740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28441b07a0 con 0x55ba9d20f900 2024-03-26T19:06:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.190+0000 7f284e578740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28441b00d0 con 0x55ba9d209da0 2024-03-26T19:06:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.190+0000 7f284e578740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2844071ed0 con 0x55ba9d20a790 2024-03-26T19:06:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.191+0000 7f2848e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d209da0 0x55ba9d2186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.191+0000 7f284e578740 10 client.0 _ll_register_callbacks cb 0x7f2834009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:06:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.191+0000 7f2843e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 0x55ba9d218c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.191+0000 7f2843e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 0x55ba9d218c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:37510/0 (socket says 172.21.15.97:37510) 2024-03-26T19:06:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.191+0000 7f2843e00640 1 -- 172.21.15.97:0/2089223027 learned_addr learned my addr 172.21.15.97:0/2089223027 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:06:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.191+0000 7f2848e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d209da0 0x55ba9d2186e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:33178/0 (socket says 172.21.15.97:33178) 2024-03-26T19:06:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.191+0000 7f2849800640 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d20f900 0x55ba9d219c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.192+0000 7f2848e00640 1 -- 172.21.15.97:0/2089223027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 msgr2=0x55ba9d218c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.192+0000 7f2848e00640 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 0x55ba9d218c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.192+0000 7f2848e00640 1 -- 172.21.15.97:0/2089223027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d20f900 msgr2=0x55ba9d219c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:06:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.192+0000 7f2848e00640 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d20f900 0x55ba9d219c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.192+0000 7f2848e00640 1 -- 172.21.15.97:0/2089223027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9d21a220 con 0x55ba9d209da0 2024-03-26T19:06:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.192+0000 7f2848e00640 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d209da0 0x55ba9d2186e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55ba9d233a40 tx=0x7f283c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ed7e875469b70a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:06:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.193+0000 7f282ea00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:06:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.193+0000 7f282ea00640 1 -- 172.21.15.97:0/2089223027 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c01cb80 con 0x55ba9d209da0 2024-03-26T19:06:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.193+0000 7f282ea00640 1 -- 172.21.15.97:0/2089223027 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f283c01cd20 con 0x55ba9d209da0 2024-03-26T19:06:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.194+0000 7f284e578740 1 -- 172.21.15.97:0/2089223027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ba9d176710 con 0x55ba9d209da0 2024-03-26T19:06:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.194+0000 7f282ea00640 1 -- 172.21.15.97:0/2089223027 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f283c02a0d0 con 0x55ba9d209da0 2024-03-26T19:06:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.194+0000 7f284e578740 10 client.25874 fetch_fsmap learned FSMap version 27 2024-03-26T19:06:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.194+0000 7f284e578740 1 -- 172.21.15.97:0/2089223027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba9d21b5a0 con 0x55ba9d209da0 2024-03-26T19:06:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.195+0000 7f282ea00640 1 -- 172.21.15.97:0/2089223027 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ba9d176710 con 0x55ba9d209da0 2024-03-26T19:06:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.195+0000 7f282ea00640 1 -- 172.21.15.97:0/2089223027 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f283c037a80 con 0x55ba9d209da0 2024-03-26T19:06:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.196+0000 7f284e578740 10 client.25874 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:06:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.196+0000 7f284e578740 10 client.25874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:06:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.196+0000 7f284e578740 20 client.25874 populate_metadata read hostname 'smithi097' 2024-03-26T19:06:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.197+0000 7f284e578740 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ba9d297830 0x55ba9d299c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:06:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.197+0000 7f284e578740 4 client.25874 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:06:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.197+0000 7f284e578740 1 -- 172.21.15.97:0/2089223027 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba9d0f4630 con 0x55ba9d297830 2024-03-26T19:06:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.197+0000 7f2843e00640 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ba9d297830 0x55ba9d299c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:06:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.197+0000 7f2843e00640 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ba9d297830 0x55ba9d299c20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f896f3191a23376 server_cookie=a71f7623688f0961 in_seq=0 out_seq=0 2024-03-26T19:06:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.197+0000 7f282ea00640 10 client.25874 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f282ea00640 1 -- 172.21.15.97:0/2089223027 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba9d0f4630 con 0x55ba9d297830 2024-03-26T19:06:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f282ea00640 10 client.25874 handle_command_reply: tid=0 2024-03-26T19:06:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f284e578740 1 client.25874 shutdown 2024-03-26T19:06:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f284e578740 1 -- 172.21.15.97:0/2089223027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d209da0 msgr2=0x55ba9d2186e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f284e578740 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d209da0 0x55ba9d2186e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55ba9d233a40 tx=0x7f283c004ab0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f284e578740 1 -- 172.21.15.97:0/2089223027 shutdown_connections 2024-03-26T19:06:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f284e578740 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ba9d297830 0x55ba9d299c20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f284e578740 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba9d20f900 0x55ba9d219c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f284e578740 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ba9d20a790 0x55ba9d218c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f284e578740 1 --2- 172.21.15.97:0/2089223027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ba9d209da0 0x55ba9d2186e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f284e578740 1 -- 172.21.15.97:0/2089223027 >> 172.21.15.97:0/2089223027 conn(0x55ba9d205310 msgr2=0x55ba9d208e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.217+0000 7f282ea00640 0 client.25874 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:06:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.218+0000 7f284e578740 1 -- 172.21.15.97:0/2089223027 shutdown_connections 2024-03-26T19:06:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.218+0000 7f284e578740 1 -- 172.21.15.97:0/2089223027 wait complete. 2024-03-26T19:06:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.218+0000 7f284e578740 20 client.25874 trim_cache size 0 max 16384 2024-03-26T19:06:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:06:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:06:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:06:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:06:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 -- 172.21.15.97:0/1259266405 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28280a32a0 msgr2=0x7f28280a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28280a32a0 0x7f28280a5760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2844104b70 tx=0x7f283c009760 comp rx=0 tx=0).stop 2024-03-26T19:06:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 -- 172.21.15.97:0/1259266405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28441056d0 msgr2=0x7f284419cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:06:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28441056d0 0x7f284419cef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2834009780 tx=0x7f2834004270 comp rx=0 tx=0).stop 2024-03-26T19:06:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 -- 172.21.15.97:0/1259266405 shutdown_connections 2024-03-26T19:06:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f28280a32a0 0x7f28280a5760 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28441056d0 0x7f284419cef0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844104d10 0x7f2844195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 --2- 172.21.15.97:0/1259266405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2844103b20 0x7f2844195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:06:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 -- 172.21.15.97:0/1259266405 >> 172.21.15.97:0/1259266405 conn(0x7f28440ff0b0 msgr2=0x7f2844077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:06:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 -- 172.21.15.97:0/1259266405 shutdown_connections 2024-03-26T19:06:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:06:56.219+0000 7f284b800640 1 -- 172.21.15.97:0/1259266405 wait complete. 2024-03-26T19:06:56.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:56 smithi097 ceph-mon[31955]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.9 MiB/s wr, 654 op/s 2024-03-26T19:06:56.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:56 smithi144 ceph-mon[36053]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.9 MiB/s wr, 654 op/s 2024-03-26T19:06:56.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:56 smithi105 ceph-mon[36270]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.9 MiB/s wr, 654 op/s 2024-03-26T19:06:58.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:06:58 smithi097 ceph-mon[31955]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.2 MiB/s wr, 571 op/s 2024-03-26T19:06:58.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:06:58 smithi144 ceph-mon[36053]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.2 MiB/s wr, 571 op/s 2024-03-26T19:06:58.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:06:58 smithi105 ceph-mon[36270]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.2 MiB/s wr, 571 op/s 2024-03-26T19:07:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 455.0 s, 2503.8 tps, lat 39.736 ms stddev 43.403 2024-03-26T19:07:00.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:00 smithi097 ceph-mon[31955]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 694 op/s 2024-03-26T19:07:00.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:00 smithi144 ceph-mon[36053]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 694 op/s 2024-03-26T19:07:00.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:00 smithi105 ceph-mon[36270]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 694 op/s 2024-03-26T19:07:02.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:02 smithi097 ceph-mon[31955]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 650 op/s 2024-03-26T19:07:02.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:02 smithi144 ceph-mon[36053]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 650 op/s 2024-03-26T19:07:02.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:02 smithi105 ceph-mon[36270]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 650 op/s 2024-03-26T19:07:04.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:04 smithi097 ceph-mon[31955]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.6 MiB/s wr, 676 op/s 2024-03-26T19:07:04.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:04 smithi144 ceph-mon[36053]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.6 MiB/s wr, 676 op/s 2024-03-26T19:07:04.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:04 smithi105 ceph-mon[36270]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.6 MiB/s wr, 676 op/s 2024-03-26T19:07:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 460.0 s, 2021.8 tps, lat 49.428 ms stddev 61.676 2024-03-26T19:07:06.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:06 smithi097 ceph-mon[31955]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 743 op/s 2024-03-26T19:07:06.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:06 smithi144 ceph-mon[36053]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 743 op/s 2024-03-26T19:07:06.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:06 smithi105 ceph-mon[36270]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 743 op/s 2024-03-26T19:07:08.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:08 smithi097 ceph-mon[31955]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 MiB/s wr, 615 op/s 2024-03-26T19:07:08.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:08 smithi144 ceph-mon[36053]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 MiB/s wr, 615 op/s 2024-03-26T19:07:08.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:08 smithi105 ceph-mon[36270]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 MiB/s wr, 615 op/s 2024-03-26T19:07:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.411+0000 7fd777a00640 1 -- 172.21.15.97:0/1452043399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd770104d10 msgr2=0x7fd770105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.411+0000 7fd777a00640 1 --2- 172.21.15.97:0/1452043399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd770104d10 0x7fd770105190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009900 tx=0x7fd75c02bc30 comp rx=0 tx=0).stop 2024-03-26T19:07:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.411+0000 7fd777a00640 1 -- 172.21.15.97:0/1452043399 shutdown_connections 2024-03-26T19:07:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.411+0000 7fd777a00640 1 --2- 172.21.15.97:0/1452043399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7701056d0 0x7fd77010bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.411+0000 7fd777a00640 1 --2- 172.21.15.97:0/1452043399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd770104d10 0x7fd770105190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.411+0000 7fd777a00640 1 --2- 172.21.15.97:0/1452043399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd770103b20 0x7fd770103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.411+0000 7fd777a00640 1 -- 172.21.15.97:0/1452043399 >> 172.21.15.97:0/1452043399 conn(0x7fd7700ff0b0 msgr2=0x7fd7701014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.412+0000 7fd777a00640 1 -- 172.21.15.97:0/1452043399 shutdown_connections 2024-03-26T19:07:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.412+0000 7fd777a00640 1 -- 172.21.15.97:0/1452043399 wait complete. 2024-03-26T19:07:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.414+0000 7fd777a00640 1 Processor -- start 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.415+0000 7fd777a00640 1 -- start start 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.415+0000 7fd777a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd770103b20 0x7fd770199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.415+0000 7fd777a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd770104d10 0x7fd77019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.415+0000 7fd777a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7701056d0 0x7fd7701a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.415+0000 7fd777a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd77010f7f0 con 0x7fd7701056d0 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.415+0000 7fd777a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd77010f670 con 0x7fd770103b20 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.415+0000 7fd777a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd77010f970 con 0x7fd770104d10 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.416+0000 7fd775a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7701056d0 0x7fd7701a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.416+0000 7fd775000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd770103b20 0x7fd770199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.416+0000 7fd775a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7701056d0 0x7fd7701a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45334/0 (socket says 172.21.15.97:45334) 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.416+0000 7fd775a00640 1 -- 172.21.15.97:0/162672364 learned_addr learned my addr 172.21.15.97:0/162672364 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.416+0000 7fd76fe00640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd770104d10 0x7fd77019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.416+0000 7fd775000640 1 -- 172.21.15.97:0/162672364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd770104d10 msgr2=0x7fd77019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.416+0000 7fd775000640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd770104d10 0x7fd77019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.416+0000 7fd775000640 1 -- 172.21.15.97:0/162672364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7701056d0 msgr2=0x7fd7701a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.416+0000 7fd775000640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7701056d0 0x7fd7701a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.416+0000 7fd775000640 1 -- 172.21.15.97:0/162672364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77006b410 con 0x7fd770103b20 2024-03-26T19:07:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.417+0000 7fd775000640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd770103b20 0x7fd770199e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd76400e9c0 tx=0x7fd76400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.417+0000 7fd76d600640 1 -- 172.21.15.97:0/162672364 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76400ccc0 con 0x7fd770103b20 2024-03-26T19:07:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.417+0000 7fd76d600640 1 -- 172.21.15.97:0/162672364 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7640035f0 con 0x7fd770103b20 2024-03-26T19:07:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.417+0000 7fd76d600640 1 -- 172.21.15.97:0/162672364 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76400ce60 con 0x7fd770103b20 2024-03-26T19:07:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.417+0000 7fd777a00640 1 -- 172.21.15.97:0/162672364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77006b700 con 0x7fd770103b20 2024-03-26T19:07:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.417+0000 7fd777a00640 1 -- 172.21.15.97:0/162672364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd770075ee0 con 0x7fd770103b20 2024-03-26T19:07:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.423+0000 7fd76d600640 1 -- 172.21.15.97:0/162672364 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fd7640037a0 con 0x7fd770103b20 2024-03-26T19:07:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.425+0000 7fd77a7fe740 1 Processor -- start 2024-03-26T19:07:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.425+0000 7fd76d600640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7580a3040 0x7fd7580a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.425+0000 7fd76d600640 1 -- 172.21.15.97:0/162672364 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd7640cf390 con 0x7fd770103b20 2024-03-26T19:07:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.426+0000 7fd77a7fe740 1 -- start start 2024-03-26T19:07:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.426+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.426+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dc790 0x564e278e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.426+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278e1900 0x564e278e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.426+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564e27840e80 con 0x564e278dc790 2024-03-26T19:07:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.426+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564e277e9660 con 0x564e278e1900 2024-03-26T19:07:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.427+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564e2783ce30 con 0x564e278dbda0 2024-03-26T19:07:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.427+0000 7fd775a00640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7580a3040 0x7fd7580a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.427+0000 7fd775000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.427+0000 7fd775000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278dc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53660/0 (socket says 172.21.15.97:53660) 2024-03-26T19:07:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.427+0000 7fd76fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dc790 0x564e278e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.427+0000 7fd775000640 1 -- 172.21.15.97:0/373117081 learned_addr learned my addr 172.21.15.97:0/373117081 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.428+0000 7fd775000640 1 -- 172.21.15.97:0/373117081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278e1900 msgr2=0x564e278e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.428+0000 7fd775000640 1 --2- 172.21.15.97:0/373117081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278e1900 0x564e278e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.428+0000 7fd775000640 1 -- 172.21.15.97:0/373117081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dc790 msgr2=0x564e278e1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.428+0000 7fd775000640 1 --2- 172.21.15.97:0/373117081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dc790 0x564e278e1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.428+0000 7fd775000640 1 -- 172.21.15.97:0/373117081 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e27555210 con 0x564e278dbda0 2024-03-26T19:07:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.428+0000 7fd76fe00640 1 --2- 172.21.15.97:0/373117081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dc790 0x564e278e1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:07:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.429+0000 7fd775000640 1 --2- 172.21.15.97:0/373117081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278dc1c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd764098460 tx=0x7fd76409ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f52dc9454d3e8c38 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.429+0000 7fd775a00640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7580a3040 0x7fd7580a5500 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd768005c40 tx=0x7fd768009b30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.429+0000 7fd762000640 1 -- 172.21.15.97:0/373117081 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7640a44a0 con 0x564e278dbda0 2024-03-26T19:07:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.429+0000 7fd762000640 1 -- 172.21.15.97:0/373117081 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7640b2030 con 0x564e278dbda0 2024-03-26T19:07:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.429+0000 7fd762000640 1 -- 172.21.15.97:0/373117081 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76409b1d0 con 0x564e278dbda0 2024-03-26T19:07:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.431+0000 7fd77a7fe740 1 -- 172.21.15.97:0/373117081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 msgr2=0x564e278dc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.431+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/373117081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278dc1c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd764098460 tx=0x7fd76409ba90 comp rx=0 tx=0).stop 2024-03-26T19:07:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.433+0000 7fd77a7fe740 1 -- 172.21.15.97:0/373117081 shutdown_connections 2024-03-26T19:07:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.433+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/373117081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278e1900 0x564e278e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.434+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/373117081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dc790 0x564e278e1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.434+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/373117081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278dc1c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.434+0000 7fd77a7fe740 1 -- 172.21.15.97:0/373117081 >> 172.21.15.97:0/373117081 conn(0x564e278d7310 msgr2=0x564e278d7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.434+0000 7fd77a7fe740 1 -- 172.21.15.97:0/373117081 shutdown_connections 2024-03-26T19:07:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.434+0000 7fd77a7fe740 1 -- 172.21.15.97:0/373117081 wait complete. 2024-03-26T19:07:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.435+0000 7fd77a7fe740 1 -- start start 2024-03-26T19:07:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.435+0000 7fd77a7fe740 1 Processor -- start 2024-03-26T19:07:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.436+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.436+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dc790 0x564e278f7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.437+0000 7fd775000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.437+0000 7fd775000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278f6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53662/0 (socket says 172.21.15.97:53662) 2024-03-26T19:07:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.437+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278e1900 0x564e278fe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.437+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564e278f5860 con 0x564e278dc790 2024-03-26T19:07:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.437+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564e277fcac0 con 0x564e278e1900 2024-03-26T19:07:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.437+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564e278f59e0 con 0x564e278dbda0 2024-03-26T19:07:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.437+0000 7fd77a7fe740 10 client.0 _ll_register_callbacks cb 0x564e277fc490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:07:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.438+0000 7fd775a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278e1900 0x564e278fe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.438+0000 7fd775000640 1 -- 172.21.15.97:0/3590256315 learned_addr learned my addr 172.21.15.97:0/3590256315 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.439+0000 7fd775000640 1 -- 172.21.15.97:0/3590256315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278e1900 msgr2=0x564e278fe340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.439+0000 7fd775000640 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278e1900 0x564e278fe340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.439+0000 7fd775000640 1 -- 172.21.15.97:0/3590256315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dc790 msgr2=0x564e278f7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.439+0000 7fd775000640 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dc790 0x564e278f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.440+0000 7fd775000640 1 -- 172.21.15.97:0/3590256315 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e27621620 con 0x564e278dbda0 2024-03-26T19:07:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.440+0000 7fd775000640 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278f6dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd76409c340 tx=0x7fd764098100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa045cfd61160ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.440+0000 7fd756a00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:07:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.441+0000 7fd756a00640 1 -- 172.21.15.97:0/3590256315 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76407e020 con 0x564e278dbda0 2024-03-26T19:07:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.441+0000 7fd756a00640 1 -- 172.21.15.97:0/3590256315 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd764003b10 con 0x564e278dbda0 2024-03-26T19:07:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.441+0000 7fd77a7fe740 1 -- 172.21.15.97:0/3590256315 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7640b2030 con 0x564e278dbda0 2024-03-26T19:07:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.442+0000 7fd756a00640 1 -- 172.21.15.97:0/3590256315 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd7640ba2c0 con 0x564e278dbda0 2024-03-26T19:07:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.442+0000 7fd77a7fe740 10 client.25840 fetch_fsmap learned FSMap version 27 2024-03-26T19:07:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.442+0000 7fd77a7fe740 1 -- 172.21.15.97:0/3590256315 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e278f5860 con 0x564e278dbda0 2024-03-26T19:07:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.444+0000 7fd756a00640 1 -- 172.21.15.97:0/3590256315 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7640b2030 con 0x564e278dbda0 2024-03-26T19:07:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.445+0000 7fd756a00640 1 -- 172.21.15.97:0/3590256315 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd76409a950 con 0x564e278dbda0 2024-03-26T19:07:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.445+0000 7fd77a7fe740 10 client.25840 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:07:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.446+0000 7fd77a7fe740 10 client.25840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:07:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.446+0000 7fd77a7fe740 20 client.25840 populate_metadata read hostname 'smithi097' 2024-03-26T19:07:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.446+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564e279029b0 0x564e27904da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.446+0000 7fd77a7fe740 4 client.25840 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:07:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.446+0000 7fd77a7fe740 1 -- 172.21.15.97:0/3590256315 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e2784e3c0 con 0x564e279029b0 2024-03-26T19:07:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.447+0000 7fd76fe00640 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564e279029b0 0x564e27904da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.448+0000 7fd756a00640 10 client.25840 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.448+0000 7fd76fe00640 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564e279029b0 0x564e27904da0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f879d9482d72e11 server_cookie=11919ac2164855c0 in_seq=0 out_seq=0 2024-03-26T19:07:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.451+0000 7fd756a00640 1 -- 172.21.15.97:0/3590256315 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564e2784e3c0 con 0x564e279029b0 2024-03-26T19:07:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.451+0000 7fd756a00640 10 client.25840 handle_command_reply: tid=0 2024-03-26T19:07:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.451+0000 7fd77a7fe740 1 client.25840 shutdown 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.452+0000 7fd77a7fe740 1 -- 172.21.15.97:0/3590256315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 msgr2=0x564e278f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.452+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278f6dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd76409c340 tx=0x7fd764098100 comp rx=0 tx=0).stop 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.452+0000 7fd77a7fe740 1 -- 172.21.15.97:0/3590256315 shutdown_connections 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.452+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564e279029b0 0x564e27904da0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.452+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278e1900 0x564e278fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.452+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dc790 0x564e278f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.452+0000 7fd756a00640 0 client.25840 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.452+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/3590256315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dbda0 0x564e278f6dc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.452+0000 7fd77a7fe740 1 -- 172.21.15.97:0/3590256315 >> 172.21.15.97:0/3590256315 conn(0x564e278d7310 msgr2=0x564e278d6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.453+0000 7fd77a7fe740 1 -- 172.21.15.97:0/3590256315 shutdown_connections 2024-03-26T19:07:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.453+0000 7fd77a7fe740 1 -- 172.21.15.97:0/3590256315 wait complete. 2024-03-26T19:07:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.453+0000 7fd77a7fe740 20 client.25840 trim_cache size 0 max 16384 2024-03-26T19:07:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.509+0000 7fd77a7fe740 1 Processor -- start 2024-03-26T19:07:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.511+0000 7fd77a7fe740 1 -- start start 2024-03-26T19:07:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.511+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dbda0 0x564e278f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.511+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dc790 0x564e278fe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.511+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278e1900 0x564e278dcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.511+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd770070390 con 0x564e278e1900 2024-03-26T19:07:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.511+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd770071ed0 con 0x564e278dbda0 2024-03-26T19:07:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.511+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd770070a60 con 0x564e278dc790 2024-03-26T19:07:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.512+0000 7fd76fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dc790 0x564e278fe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.512+0000 7fd76fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dc790 0x564e278fe300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53678/0 (socket says 172.21.15.97:53678) 2024-03-26T19:07:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.512+0000 7fd76fe00640 1 -- 172.21.15.97:0/2463415136 learned_addr learned my addr 172.21.15.97:0/2463415136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.511+0000 7fd775000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dbda0 0x564e278f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.513+0000 7fd775000640 1 -- 172.21.15.97:0/2463415136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dc790 msgr2=0x564e278fe300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.513+0000 7fd775000640 1 --2- 172.21.15.97:0/2463415136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dc790 0x564e278fe300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.513+0000 7fd775000640 1 -- 172.21.15.97:0/2463415136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278e1900 msgr2=0x564e278dcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.513+0000 7fd775000640 1 --2- 172.21.15.97:0/2463415136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278e1900 0x564e278dcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.513+0000 7fd775000640 1 -- 172.21.15.97:0/2463415136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e279053d0 con 0x564e278dbda0 2024-03-26T19:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.514+0000 7fd76fe00640 1 --2- 172.21.15.97:0/2463415136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dc790 0x564e278fe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.514+0000 7fd775000640 1 --2- 172.21.15.97:0/2463415136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dbda0 0x564e278f70e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd764098460 tx=0x7fd764099fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=390efd9c2a1d2291 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.515+0000 7fd761600640 1 -- 172.21.15.97:0/2463415136 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76407e070 con 0x564e278dbda0 2024-03-26T19:07:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.515+0000 7fd761600640 1 -- 172.21.15.97:0/2463415136 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7640b2030 con 0x564e278dbda0 2024-03-26T19:07:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.515+0000 7fd761600640 1 -- 172.21.15.97:0/2463415136 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76407e070 con 0x564e278dbda0 2024-03-26T19:07:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.517+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2463415136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dbda0 msgr2=0x564e278f70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.517+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/2463415136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dbda0 0x564e278f70e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd764098460 tx=0x7fd764099fa0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.517+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2463415136 shutdown_connections 2024-03-26T19:07:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.517+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/2463415136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278e1900 0x564e278dcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.518+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/2463415136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278dc790 0x564e278fe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.518+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/2463415136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dbda0 0x564e278f70e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.518+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2463415136 >> 172.21.15.97:0/2463415136 conn(0x564e278d7310 msgr2=0x564e278f69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.518+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2463415136 shutdown_connections 2024-03-26T19:07:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.519+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2463415136 wait complete. 2024-03-26T19:07:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.520+0000 7fd77a7fe740 1 -- start start 2024-03-26T19:07:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.521+0000 7fd77a7fe740 1 Processor -- start 2024-03-26T19:07:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.521+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dbda0 0x564e278fb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.521+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dc790 0x564e278fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.522+0000 7fd77a7fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278e1900 0x564e278fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.522+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd770071800 con 0x564e278dbda0 2024-03-26T19:07:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.522+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd770071130 con 0x564e278dc790 2024-03-26T19:07:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.522+0000 7fd77a7fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd77006e440 con 0x564e278e1900 2024-03-26T19:07:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.522+0000 7fd77a7fe740 10 client.0 _ll_register_callbacks cb 0x7fd76409b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:07:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.522+0000 7fd775a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278e1900 0x564e278fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.522+0000 7fd775a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278e1900 0x564e278fcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:53688/0 (socket says 172.21.15.97:53688) 2024-03-26T19:07:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.522+0000 7fd775000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dbda0 0x564e278fb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.522+0000 7fd775a00640 1 -- 172.21.15.97:0/2155578939 learned_addr learned my addr 172.21.15.97:0/2155578939 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.522+0000 7fd76fe00640 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dc790 0x564e278fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.523+0000 7fd775a00640 1 -- 172.21.15.97:0/2155578939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dc790 msgr2=0x564e278fbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.523+0000 7fd775a00640 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dc790 0x564e278fbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.523+0000 7fd775a00640 1 -- 172.21.15.97:0/2155578939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dbda0 msgr2=0x564e278fb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.523+0000 7fd775a00640 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dbda0 0x564e278fb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.523+0000 7fd775a00640 1 -- 172.21.15.97:0/2155578939 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e278fd0d0 con 0x564e278e1900 2024-03-26T19:07:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.524+0000 7fd775a00640 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278e1900 0x564e278fcae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd7680095d0 tx=0x7fd768004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c644bc287c4d9af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.524+0000 7fd756a00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:07:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.524+0000 7fd756a00640 1 -- 172.21.15.97:0/2155578939 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76800f990 con 0x564e278e1900 2024-03-26T19:07:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.524+0000 7fd756a00640 1 -- 172.21.15.97:0/2155578939 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd76800f790 con 0x564e278e1900 2024-03-26T19:07:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.527+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2155578939 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7640b2030 con 0x564e278e1900 2024-03-26T19:07:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.527+0000 7fd756a00640 1 -- 172.21.15.97:0/2155578939 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd76800f990 con 0x564e278e1900 2024-03-26T19:07:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.529+0000 7fd77a7fe740 10 client.25846 fetch_fsmap learned FSMap version 27 2024-03-26T19:07:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.529+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2155578939 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e278fe4b0 con 0x564e278e1900 2024-03-26T19:07:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.529+0000 7fd756a00640 1 -- 172.21.15.97:0/2155578939 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd768023070 con 0x564e278e1900 2024-03-26T19:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.529+0000 7fd756a00640 1 -- 172.21.15.97:0/2155578939 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd76801c070 con 0x564e278e1900 2024-03-26T19:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.529+0000 7fd77a7fe740 10 client.25846 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.529+0000 7fd77a7fe740 10 client.25846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.529+0000 7fd77a7fe740 20 client.25846 populate_metadata read hostname 'smithi097' 2024-03-26T19:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.529+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564e27969330 0x564e2796b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.530+0000 7fd77a7fe740 4 client.25846 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.530+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2155578939 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd7640b2030 con 0x564e27969330 2024-03-26T19:07:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.530+0000 7fd775000640 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564e27969330 0x564e2796b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.531+0000 7fd775000640 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564e27969330 0x564e2796b750 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9169d76cb1d72b90 server_cookie=56b6c6d298948fe4 in_seq=0 out_seq=0 2024-03-26T19:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.531+0000 7fd756a00640 10 client.25846 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.556+0000 7fd756a00640 1 -- 172.21.15.97:0/2155578939 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fd7640b2030 con 0x564e27969330 2024-03-26T19:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.556+0000 7fd756a00640 10 client.25846 handle_command_reply: tid=0 2024-03-26T19:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.556+0000 7fd77a7fe740 1 client.25846 shutdown 2024-03-26T19:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.556+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2155578939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278e1900 msgr2=0x564e278fcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.556+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278e1900 0x564e278fcae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd7680095d0 tx=0x7fd768004a40 comp rx=0 tx=0).stop 2024-03-26T19:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.556+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2155578939 shutdown_connections 2024-03-26T19:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.556+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564e27969330 0x564e2796b750 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.556+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e278e1900 0x564e278fcae0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.556+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e278dc790 0x564e278fbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.556+0000 7fd77a7fe740 1 --2- 172.21.15.97:0/2155578939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e278dbda0 0x564e278fb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.557+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2155578939 >> 172.21.15.97:0/2155578939 conn(0x564e278d7310 msgr2=0x564e278d5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.557+0000 7fd756a00640 0 client.25846 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.557+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2155578939 shutdown_connections 2024-03-26T19:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.557+0000 7fd77a7fe740 1 -- 172.21.15.97:0/2155578939 wait complete. 2024-03-26T19:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.557+0000 7fd77a7fe740 20 client.25846 trim_cache size 0 max 16384 2024-03-26T19:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.559+0000 7fd777a00640 1 -- 172.21.15.97:0/162672364 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7580a3040 msgr2=0x7fd7580a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.559+0000 7fd777a00640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7580a3040 0x7fd7580a5500 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd768005c40 tx=0x7fd768009b30 comp rx=0 tx=0).stop 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.559+0000 7fd777a00640 1 -- 172.21.15.97:0/162672364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd770103b20 msgr2=0x7fd770199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.559+0000 7fd777a00640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd770103b20 0x7fd770199e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd76400e9c0 tx=0x7fd76400ee80 comp rx=0 tx=0).stop 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.560+0000 7fd777a00640 1 -- 172.21.15.97:0/162672364 shutdown_connections 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.560+0000 7fd777a00640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd7580a3040 0x7fd7580a5500 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.560+0000 7fd777a00640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7701056d0 0x7fd7701a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.560+0000 7fd777a00640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd770104d10 0x7fd77019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.560+0000 7fd777a00640 1 --2- 172.21.15.97:0/162672364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd770103b20 0x7fd770199e30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.560+0000 7fd777a00640 1 -- 172.21.15.97:0/162672364 >> 172.21.15.97:0/162672364 conn(0x7fd7700ff0b0 msgr2=0x7fd770107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.560+0000 7fd777a00640 1 -- 172.21.15.97:0/162672364 shutdown_connections 2024-03-26T19:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:09.560+0000 7fd777a00640 1 -- 172.21.15.97:0/162672364 wait complete. 2024-03-26T19:07:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 465.0 s, 1697.4 tps, lat 59.082 ms stddev 76.656 2024-03-26T19:07:10.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:10 smithi097 ceph-mon[31955]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 748 op/s 2024-03-26T19:07:10.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:10 smithi144 ceph-mon[36053]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 748 op/s 2024-03-26T19:07:10.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:10 smithi105 ceph-mon[36270]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 748 op/s 2024-03-26T19:07:12.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:12 smithi097 ceph-mon[31955]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.9 MiB/s wr, 699 op/s 2024-03-26T19:07:12.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:12 smithi144 ceph-mon[36053]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.9 MiB/s wr, 699 op/s 2024-03-26T19:07:12.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:12 smithi105 ceph-mon[36270]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.9 MiB/s wr, 699 op/s 2024-03-26T19:07:14.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:14 smithi097 ceph-mon[31955]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 MiB/s wr, 671 op/s 2024-03-26T19:07:14.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:14 smithi144 ceph-mon[36053]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 MiB/s wr, 671 op/s 2024-03-26T19:07:14.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:14 smithi105 ceph-mon[36270]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 MiB/s wr, 671 op/s 2024-03-26T19:07:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 470.0 s, 2170.8 tps, lat 46.160 ms stddev 55.708 2024-03-26T19:07:16.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:16 smithi097 ceph-mon[31955]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 701 op/s 2024-03-26T19:07:16.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:16 smithi144 ceph-mon[36053]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 701 op/s 2024-03-26T19:07:16.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:16 smithi105 ceph-mon[36270]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 701 op/s 2024-03-26T19:07:17.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:17 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:07:17.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:17 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:07:17.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:17 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:07:18.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:18 smithi097 ceph-mon[31955]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 601 op/s 2024-03-26T19:07:18.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:18 smithi144 ceph-mon[36053]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 601 op/s 2024-03-26T19:07:18.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:18 smithi105 ceph-mon[36270]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 601 op/s 2024-03-26T19:07:19.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:07:19.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:07:19.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:07:19.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:07:19.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:07:19.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:07:19.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:07:19.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:07:19.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:07:20.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 475.0 s, 1695.2 tps, lat 58.683 ms stddev 71.502 2024-03-26T19:07:20.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:20 smithi097 ceph-mon[31955]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.0 MiB/s wr, 718 op/s 2024-03-26T19:07:20.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:20 smithi144 ceph-mon[36053]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.0 MiB/s wr, 718 op/s 2024-03-26T19:07:20.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:20 smithi105 ceph-mon[36270]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.0 MiB/s wr, 718 op/s 2024-03-26T19:07:22.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:22 smithi097 ceph-mon[31955]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-03-26T19:07:22.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:22 smithi144 ceph-mon[36053]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-03-26T19:07:22.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:22 smithi105 ceph-mon[36270]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-03-26T19:07:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.737+0000 7fa575600640 1 -- 172.21.15.97:0/2184528802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5700ffef0 msgr2=0x7fa5701070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.737+0000 7fa575600640 1 --2- 172.21.15.97:0/2184528802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5700ffef0 0x7fa5701070d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009900 tx=0x7fa55c02bc40 comp rx=0 tx=0).stop 2024-03-26T19:07:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.738+0000 7fa575600640 1 -- 172.21.15.97:0/2184528802 shutdown_connections 2024-03-26T19:07:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.738+0000 7fa575600640 1 --2- 172.21.15.97:0/2184528802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa570107610 0x7fa570109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.738+0000 7fa575600640 1 --2- 172.21.15.97:0/2184528802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5700ffef0 0x7fa5701070d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.738+0000 7fa575600640 1 --2- 172.21.15.97:0/2184528802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5700ff500 0x7fa5700ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.738+0000 7fa575600640 1 -- 172.21.15.97:0/2184528802 >> 172.21.15.97:0/2184528802 conn(0x7fa5700fb0a0 msgr2=0x7fa5700fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.738+0000 7fa575600640 1 -- 172.21.15.97:0/2184528802 shutdown_connections 2024-03-26T19:07:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.738+0000 7fa575600640 1 -- 172.21.15.97:0/2184528802 wait complete. 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.739+0000 7fa575600640 1 Processor -- start 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa575600640 1 -- start start 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa575600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5700ff500 0x7fa570110c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa575600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5700ffef0 0x7fa570111180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa575600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa570107610 0x7fa5701181f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa575600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701020a0 con 0x7fa570107610 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa575600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa570101f20 con 0x7fa5700ffef0 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa575600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa570102220 con 0x7fa5700ff500 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa56ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5700ff500 0x7fa570110c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa56ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5700ff500 0x7fa570110c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52150/0 (socket says 172.21.15.97:52150) 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa56ea00640 1 -- 172.21.15.97:0/4177843791 learned_addr learned my addr 172.21.15.97:0/4177843791 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.740+0000 7fa56e000640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5700ffef0 0x7fa570111180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.741+0000 7fa56ea00640 1 -- 172.21.15.97:0/4177843791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5700ffef0 msgr2=0x7fa570111180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.741+0000 7fa56ea00640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5700ffef0 0x7fa570111180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.741+0000 7fa56ea00640 1 -- 172.21.15.97:0/4177843791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa570107610 msgr2=0x7fa5701181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.741+0000 7fa56ea00640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa570107610 0x7fa5701181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.741+0000 7fa56ea00640 1 -- 172.21.15.97:0/4177843791 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa570118860 con 0x7fa5700ff500 2024-03-26T19:07:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.744+0000 7fa56ea00640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5700ff500 0x7fa570110c40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa56000e9f0 tx=0x7fa56000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.744+0000 7fa567400640 1 -- 172.21.15.97:0/4177843791 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56000cd80 con 0x7fa5700ff500 2024-03-26T19:07:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.745+0000 7fa575600640 1 -- 172.21.15.97:0/4177843791 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa570118b50 con 0x7fa5700ff500 2024-03-26T19:07:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.745+0000 7fa575600640 1 -- 172.21.15.97:0/4177843791 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5701aeeb0 con 0x7fa5700ff500 2024-03-26T19:07:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.745+0000 7fa567400640 1 -- 172.21.15.97:0/4177843791 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa560010910 con 0x7fa5700ff500 2024-03-26T19:07:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.745+0000 7fa567400640 1 -- 172.21.15.97:0/4177843791 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5600188f0 con 0x7fa5700ff500 2024-03-26T19:07:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.754+0000 7fa578229740 1 Processor -- start 2024-03-26T19:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.756+0000 7fa567400640 1 -- 172.21.15.97:0/4177843791 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fa560010ac0 con 0x7fa5700ff500 2024-03-26T19:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.757+0000 7fa578229740 1 -- start start 2024-03-26T19:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.759+0000 7fa578229740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 0x5621306821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa578229740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130687290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.761+0000 7fa56f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 0x5621306821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa56f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 0x5621306821c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52162/0 (socket says 172.21.15.97:52162) 2024-03-26T19:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa567400640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa5540a3010 0x7fa5540a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa578229740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x562130689d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa578229740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5621305e6e80 con 0x562130687900 2024-03-26T19:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa578229740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56213058f660 con 0x562130682790 2024-03-26T19:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa578229740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5621305e2e30 con 0x562130681da0 2024-03-26T19:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa56ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130687290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa56ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130687290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:40010/0 (socket says 172.21.15.97:40010) 2024-03-26T19:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa56ea00640 1 -- 172.21.15.97:0/2383152921 learned_addr learned my addr 172.21.15.97:0/2383152921 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.762+0000 7fa567400640 1 -- 172.21.15.97:0/4177843791 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa5600cf780 con 0x7fa5700ff500 2024-03-26T19:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.763+0000 7fa56ea00640 1 -- 172.21.15.97:0/2383152921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 msgr2=0x5621306821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.763+0000 7fa56ea00640 1 --2- 172.21.15.97:0/2383152921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 0x5621306821c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.763+0000 7fa56ea00640 1 -- 172.21.15.97:0/2383152921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 msgr2=0x562130689d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.764+0000 7fa56ea00640 1 --2- 172.21.15.97:0/2383152921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x562130689d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.764+0000 7fa56ea00640 1 -- 172.21.15.97:0/2383152921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621302fb210 con 0x562130682790 2024-03-26T19:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.764+0000 7fa56e000640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa5540a3010 0x7fa5540a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.765+0000 7fa56ea00640 1 --2- 172.21.15.97:0/2383152921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130687290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa5600057f0 tx=0x7fa56009bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d8e504e1e41c8f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.765+0000 7fa55b400640 1 -- 172.21.15.97:0/2383152921 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56009bee0 con 0x562130682790 2024-03-26T19:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.765+0000 7fa55b400640 1 -- 172.21.15.97:0/2383152921 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5600a0030 con 0x562130682790 2024-03-26T19:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.765+0000 7fa55b400640 1 -- 172.21.15.97:0/2383152921 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5600a46c0 con 0x562130682790 2024-03-26T19:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.766+0000 7fa578229740 1 -- 172.21.15.97:0/2383152921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 msgr2=0x562130687290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.766+0000 7fa578229740 1 --2- 172.21.15.97:0/2383152921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130687290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa5600057f0 tx=0x7fa56009bae0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.768+0000 7fa56e000640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa5540a3010 0x7fa5540a54d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa55c02c250 tx=0x7fa55c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.768+0000 7fa578229740 1 -- 172.21.15.97:0/2383152921 shutdown_connections 2024-03-26T19:07:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.768+0000 7fa578229740 1 --2- 172.21.15.97:0/2383152921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x562130689d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.768+0000 7fa578229740 1 --2- 172.21.15.97:0/2383152921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130687290 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.768+0000 7fa578229740 1 --2- 172.21.15.97:0/2383152921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 0x5621306821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.768+0000 7fa578229740 1 -- 172.21.15.97:0/2383152921 >> 172.21.15.97:0/2383152921 conn(0x56213067d310 msgr2=0x56213067d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.768+0000 7fa578229740 1 -- 172.21.15.97:0/2383152921 shutdown_connections 2024-03-26T19:07:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.768+0000 7fa578229740 1 -- 172.21.15.97:0/2383152921 wait complete. 2024-03-26T19:07:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.769+0000 7fa578229740 1 -- start start 2024-03-26T19:07:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.769+0000 7fa578229740 1 Processor -- start 2024-03-26T19:07:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.770+0000 7fa578229740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130681da0 0x56213068be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.770+0000 7fa578229740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x56213068c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.770+0000 7fa578229740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130687900 0x5621306933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.771+0000 7fa578229740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56213068a8b0 con 0x562130681da0 2024-03-26T19:07:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.771+0000 7fa578229740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5621305a2ac0 con 0x562130682790 2024-03-26T19:07:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.771+0000 7fa578229740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56213068aa30 con 0x562130687900 2024-03-26T19:07:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.771+0000 7fa578229740 10 client.0 _ll_register_callbacks cb 0x5621305a2490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:07:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.772+0000 7fa56f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130687900 0x5621306933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.772+0000 7fa56f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130687900 0x5621306933b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52174/0 (socket says 172.21.15.97:52174) 2024-03-26T19:07:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.772+0000 7fa56f400640 1 -- 172.21.15.97:0/812091851 learned_addr learned my addr 172.21.15.97:0/812091851 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.772+0000 7fa56e000640 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x56213068c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.772+0000 7fa56ea00640 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130681da0 0x56213068be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.775+0000 7fa56f400640 1 -- 172.21.15.97:0/812091851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 msgr2=0x56213068c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.775+0000 7fa56f400640 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x56213068c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.775+0000 7fa56f400640 1 -- 172.21.15.97:0/812091851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130681da0 msgr2=0x56213068be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.775+0000 7fa56f400640 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130681da0 0x56213068be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.775+0000 7fa56f400640 1 -- 172.21.15.97:0/812091851 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621303c7620 con 0x562130687900 2024-03-26T19:07:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.776+0000 7fa56f400640 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130687900 0x5621306933b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa568011890 tx=0x7fa568011d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c03dd55eb6125e7f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.776+0000 7fa54fe00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:07:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.776+0000 7fa54fe00640 1 -- 172.21.15.97:0/812091851 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568004d20 con 0x562130687900 2024-03-26T19:07:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.776+0000 7fa54fe00640 1 -- 172.21.15.97:0/812091851 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa56800fa70 con 0x562130687900 2024-03-26T19:07:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.776+0000 7fa578229740 1 -- 172.21.15.97:0/812091851 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5600a0030 con 0x562130687900 2024-03-26T19:07:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.777+0000 7fa54fe00640 1 -- 172.21.15.97:0/812091851 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa56800fc70 con 0x562130687900 2024-03-26T19:07:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.778+0000 7fa578229740 10 client.25858 fetch_fsmap learned FSMap version 27 2024-03-26T19:07:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.778+0000 7fa578229740 1 -- 172.21.15.97:0/812091851 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5621306967f0 con 0x562130687900 2024-03-26T19:07:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.778+0000 7fa54fe00640 1 -- 172.21.15.97:0/812091851 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5600a0030 con 0x562130687900 2024-03-26T19:07:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.781+0000 7fa54fe00640 1 -- 172.21.15.97:0/812091851 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa568005020 con 0x562130687900 2024-03-26T19:07:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.782+0000 7fa578229740 10 client.25858 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:07:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.782+0000 7fa578229740 10 client.25858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.782+0000 7fa578229740 20 client.25858 populate_metadata read hostname 'smithi097' 2024-03-26T19:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.782+0000 7fa578229740 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562130697f30 0x56213069a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.782+0000 7fa578229740 4 client.25858 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.782+0000 7fa578229740 1 -- 172.21.15.97:0/812091851 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621305f43c0 con 0x562130697f30 2024-03-26T19:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.782+0000 7fa56ea00640 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562130697f30 0x56213069a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.784+0000 7fa54fe00640 10 client.25858 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.784+0000 7fa56ea00640 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562130697f30 0x56213069a3b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ad37b46f95d071 server_cookie=61f253a650dbccb2 in_seq=0 out_seq=0 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa54fe00640 1 -- 172.21.15.97:0/812091851 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5621305f43c0 con 0x562130697f30 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa54fe00640 10 client.25858 handle_command_reply: tid=0 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa578229740 1 client.25858 shutdown 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa578229740 1 -- 172.21.15.97:0/812091851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130687900 msgr2=0x5621306933b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa578229740 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130687900 0x5621306933b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa568011890 tx=0x7fa568011d50 comp rx=0 tx=0).stop 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa578229740 1 -- 172.21.15.97:0/812091851 shutdown_connections 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa578229740 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x562130697f30 0x56213069a3b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa578229740 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130687900 0x5621306933b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa578229740 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x56213068c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa578229740 1 --2- 172.21.15.97:0/812091851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130681da0 0x56213068be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa578229740 1 -- 172.21.15.97:0/812091851 >> 172.21.15.97:0/812091851 conn(0x56213067d310 msgr2=0x56213067c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.786+0000 7fa54fe00640 0 client.25858 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.787+0000 7fa578229740 1 -- 172.21.15.97:0/812091851 shutdown_connections 2024-03-26T19:07:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.787+0000 7fa578229740 1 -- 172.21.15.97:0/812091851 wait complete. 2024-03-26T19:07:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.787+0000 7fa578229740 20 client.25858 trim_cache size 0 max 16384 2024-03-26T19:07:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.823+0000 7fa578229740 1 Processor -- start 2024-03-26T19:07:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.823+0000 7fa578229740 1 -- start start 2024-03-26T19:07:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.824+0000 7fa578229740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130681da0 0x562130680c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.824+0000 7fa578229740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130682790 0x562130693370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.824+0000 7fa578229740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x56213067bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.824+0000 7fa578229740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa55000e380 con 0x562130687900 2024-03-26T19:07:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.824+0000 7fa578229740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa55000fec0 con 0x562130681da0 2024-03-26T19:07:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.824+0000 7fa578229740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa55000ea50 con 0x562130682790 2024-03-26T19:07:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.824+0000 7fa56ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130681da0 0x562130680c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.824+0000 7fa56ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130681da0 0x562130680c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:40026/0 (socket says 172.21.15.97:40026) 2024-03-26T19:07:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.824+0000 7fa56ea00640 1 -- 172.21.15.97:0/414859454 learned_addr learned my addr 172.21.15.97:0/414859454 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.825+0000 7fa56e000640 1 --2- 172.21.15.97:0/414859454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130682790 0x562130693370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.825+0000 7fa56f400640 1 --2- 172.21.15.97:0/414859454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x56213067bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.825+0000 7fa56ea00640 1 -- 172.21.15.97:0/414859454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130682790 msgr2=0x562130693370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.825+0000 7fa56ea00640 1 --2- 172.21.15.97:0/414859454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130682790 0x562130693370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.825+0000 7fa56ea00640 1 -- 172.21.15.97:0/414859454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 msgr2=0x56213067bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.826+0000 7fa56ea00640 1 --2- 172.21.15.97:0/414859454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x56213067bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.826+0000 7fa56ea00640 1 -- 172.21.15.97:0/414859454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562130686a80 con 0x562130681da0 2024-03-26T19:07:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.826+0000 7fa56f400640 1 --2- 172.21.15.97:0/414859454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x56213067bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:07:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.827+0000 7fa56ea00640 1 --2- 172.21.15.97:0/414859454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130681da0 0x562130680c80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa560098340 tx=0x7fa56009bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d3b7b5e63546b0c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.827+0000 7fa55aa00640 1 -- 172.21.15.97:0/414859454 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5600a0030 con 0x562130681da0 2024-03-26T19:07:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.827+0000 7fa55aa00640 1 -- 172.21.15.97:0/414859454 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa560098550 con 0x562130681da0 2024-03-26T19:07:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.827+0000 7fa55aa00640 1 -- 172.21.15.97:0/414859454 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56009b330 con 0x562130681da0 2024-03-26T19:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.829+0000 7fa578229740 1 -- 172.21.15.97:0/414859454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130681da0 msgr2=0x562130680c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.829+0000 7fa578229740 1 --2- 172.21.15.97:0/414859454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130681da0 0x562130680c80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa560098340 tx=0x7fa56009bb50 comp rx=0 tx=0).stop 2024-03-26T19:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.829+0000 7fa578229740 1 -- 172.21.15.97:0/414859454 shutdown_connections 2024-03-26T19:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.829+0000 7fa578229740 1 --2- 172.21.15.97:0/414859454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x56213067bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.829+0000 7fa578229740 1 --2- 172.21.15.97:0/414859454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130682790 0x562130693370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.829+0000 7fa578229740 1 --2- 172.21.15.97:0/414859454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130681da0 0x562130680c80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.829+0000 7fa578229740 1 -- 172.21.15.97:0/414859454 >> 172.21.15.97:0/414859454 conn(0x56213067d310 msgr2=0x562130680570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.829+0000 7fa578229740 1 -- 172.21.15.97:0/414859454 shutdown_connections 2024-03-26T19:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.829+0000 7fa578229740 1 -- 172.21.15.97:0/414859454 wait complete. 2024-03-26T19:07:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.830+0000 7fa578229740 1 -- start start 2024-03-26T19:07:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.830+0000 7fa578229740 1 Processor -- start 2024-03-26T19:07:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.830+0000 7fa578229740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 0x562130691010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.830+0000 7fa578229740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130691550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.830+0000 7fa578229740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x562130692560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa578229740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa55000f7f0 con 0x562130687900 2024-03-26T19:07:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa578229740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa55000f120 con 0x562130682790 2024-03-26T19:07:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa578229740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa55000c360 con 0x562130681da0 2024-03-26T19:07:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa578229740 10 client.0 _ll_register_callbacks cb 0x7fa568011520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:07:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa56e000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130691550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa56e000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130691550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:40038/0 (socket says 172.21.15.97:40038) 2024-03-26T19:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa56e000640 1 -- 172.21.15.97:0/3446448548 learned_addr learned my addr 172.21.15.97:0/3446448548 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa56ea00640 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 0x562130691010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa56f400640 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x562130692560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa56ea00640 1 -- 172.21.15.97:0/3446448548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 msgr2=0x562130691550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa56ea00640 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130691550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa56ea00640 1 -- 172.21.15.97:0/3446448548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 msgr2=0x562130692560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa56ea00640 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x562130692560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.831+0000 7fa56ea00640 1 -- 172.21.15.97:0/3446448548 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562130692b50 con 0x562130681da0 2024-03-26T19:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.832+0000 7fa56f400640 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x562130692560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:07:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.832+0000 7fa56ea00640 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 0x562130691010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa56009be70 tx=0x7fa5600cfb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff818d3326cd0ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.832+0000 7fa54fe00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:07:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.832+0000 7fa54fe00640 1 -- 172.21.15.97:0/3446448548 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5600a0030 con 0x562130681da0 2024-03-26T19:07:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.832+0000 7fa578229740 1 -- 172.21.15.97:0/3446448548 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa560098550 con 0x562130681da0 2024-03-26T19:07:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.833+0000 7fa54fe00640 1 -- 172.21.15.97:0/3446448548 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa5600a44d0 con 0x562130681da0 2024-03-26T19:07:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.833+0000 7fa54fe00640 1 -- 172.21.15.97:0/3446448548 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa56007a020 con 0x562130681da0 2024-03-26T19:07:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.833+0000 7fa578229740 10 client.25864 fetch_fsmap learned FSMap version 27 2024-03-26T19:07:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.833+0000 7fa578229740 1 -- 172.21.15.97:0/3446448548 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56213070f7d0 con 0x562130681da0 2024-03-26T19:07:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.833+0000 7fa54fe00640 1 -- 172.21.15.97:0/3446448548 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa560098550 con 0x562130681da0 2024-03-26T19:07:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.833+0000 7fa54fe00640 1 -- 172.21.15.97:0/3446448548 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa56009b330 con 0x562130681da0 2024-03-26T19:07:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.833+0000 7fa578229740 10 client.25864 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:07:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.833+0000 7fa578229740 10 client.25864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:07:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.833+0000 7fa578229740 20 client.25864 populate_metadata read hostname 'smithi097' 2024-03-26T19:07:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.833+0000 7fa578229740 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56213070fd50 0x562130712170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.834+0000 7fa578229740 4 client.25864 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:07:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.834+0000 7fa578229740 1 -- 172.21.15.97:0/3446448548 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621305ee710 con 0x56213070fd50 2024-03-26T19:07:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.834+0000 7fa56e000640 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56213070fd50 0x562130712170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.836+0000 7fa56e000640 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56213070fd50 0x562130712170 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab2686b03011bb9f server_cookie=9e13eb45e32941ed in_seq=0 out_seq=0 2024-03-26T19:07:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.836+0000 7fa54fe00640 10 client.25864 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa54fe00640 1 -- 172.21.15.97:0/3446448548 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621305ee710 con 0x56213070fd50 2024-03-26T19:07:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa54fe00640 10 client.25864 handle_command_reply: tid=0 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 client.25864 shutdown 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 -- 172.21.15.97:0/3446448548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 msgr2=0x562130691010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 0x562130691010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa56009be70 tx=0x7fa5600cfb70 comp rx=0 tx=0).stop 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 -- 172.21.15.97:0/3446448548 shutdown_connections 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56213070fd50 0x562130712170 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562130687900 0x562130692560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562130682790 0x562130691550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 --2- 172.21.15.97:0/3446448548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562130681da0 0x562130691010 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 -- 172.21.15.97:0/3446448548 >> 172.21.15.97:0/3446448548 conn(0x56213067d310 msgr2=0x562130680e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa54fe00640 0 client.25864 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 -- 172.21.15.97:0/3446448548 shutdown_connections 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 1 -- 172.21.15.97:0/3446448548 wait complete. 2024-03-26T19:07:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.862+0000 7fa578229740 20 client.25864 trim_cache size 0 max 16384 2024-03-26T19:07:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:07:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:07:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:07:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:07:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 -- 172.21.15.97:0/4177843791 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa5540a3010 msgr2=0x7fa5540a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa5540a3010 0x7fa5540a54d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa55c02c250 tx=0x7fa55c000bc0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 -- 172.21.15.97:0/4177843791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5700ff500 msgr2=0x7fa570110c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5700ff500 0x7fa570110c40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa56000e9f0 tx=0x7fa56000eeb0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 -- 172.21.15.97:0/4177843791 shutdown_connections 2024-03-26T19:07:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa5540a3010 0x7fa5540a54d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa570107610 0x7fa5701181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5700ffef0 0x7fa570111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 --2- 172.21.15.97:0/4177843791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5700ff500 0x7fa570110c40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 -- 172.21.15.97:0/4177843791 >> 172.21.15.97:0/4177843791 conn(0x7fa5700fb0a0 msgr2=0x7fa570102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.863+0000 7fa575600640 1 -- 172.21.15.97:0/4177843791 shutdown_connections 2024-03-26T19:07:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:22.864+0000 7fa575600640 1 -- 172.21.15.97:0/4177843791 wait complete. 2024-03-26T19:07:24.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:24 smithi097 ceph-mon[31955]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 649 op/s 2024-03-26T19:07:24.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:24 smithi144 ceph-mon[36053]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 649 op/s 2024-03-26T19:07:24.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:24 smithi105 ceph-mon[36270]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 649 op/s 2024-03-26T19:07:25.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 480.0 s, 1550.6 tps, lat 64.408 ms stddev 78.073 2024-03-26T19:07:26.357 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:26 smithi097 ceph-mon[31955]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.2 MiB/s wr, 683 op/s 2024-03-26T19:07:26.380 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:26 smithi144 ceph-mon[36053]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.2 MiB/s wr, 683 op/s 2024-03-26T19:07:26.465 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:26 smithi105 ceph-mon[36270]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.2 MiB/s wr, 683 op/s 2024-03-26T19:07:28.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:28 smithi097 ceph-mon[31955]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 545 op/s 2024-03-26T19:07:28.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:28 smithi144 ceph-mon[36053]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 545 op/s 2024-03-26T19:07:28.714 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:28 smithi105 ceph-mon[36270]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 545 op/s 2024-03-26T19:07:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 485.0 s, 1767.0 tps, lat 56.258 ms stddev 68.982 2024-03-26T19:07:30.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:30 smithi097 ceph-mon[31955]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.5 MiB/s wr, 686 op/s 2024-03-26T19:07:30.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:30 smithi144 ceph-mon[36053]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.5 MiB/s wr, 686 op/s 2024-03-26T19:07:30.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:30 smithi105 ceph-mon[36270]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.5 MiB/s wr, 686 op/s 2024-03-26T19:07:32.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:32 smithi097 ceph-mon[31955]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-03-26T19:07:32.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:32 smithi144 ceph-mon[36053]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-03-26T19:07:32.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:32 smithi105 ceph-mon[36270]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-03-26T19:07:34.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:34 smithi097 ceph-mon[31955]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.3 MiB/s wr, 667 op/s 2024-03-26T19:07:34.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:34 smithi144 ceph-mon[36053]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.3 MiB/s wr, 667 op/s 2024-03-26T19:07:34.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:34 smithi105 ceph-mon[36270]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.3 MiB/s wr, 667 op/s 2024-03-26T19:07:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 490.0 s, 1347.8 tps, lat 74.261 ms stddev 90.314 2024-03-26T19:07:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.062+0000 7ff2b6600640 1 -- 172.21.15.97:0/1266708954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b0101910 msgr2=0x7ff2b0101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.062+0000 7ff2b6600640 1 --2- 172.21.15.97:0/1266708954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b0101910 0x7ff2b0101d30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009900 tx=0x7ff2a002bc50 comp rx=0 tx=0).stop 2024-03-26T19:07:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.062+0000 7ff2b6600640 1 -- 172.21.15.97:0/1266708954 shutdown_connections 2024-03-26T19:07:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.062+0000 7ff2b6600640 1 --2- 172.21.15.97:0/1266708954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b01034c0 0x7ff2b0109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.062+0000 7ff2b6600640 1 --2- 172.21.15.97:0/1266708954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b0102b00 0x7ff2b0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.062+0000 7ff2b6600640 1 --2- 172.21.15.97:0/1266708954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b0101910 0x7ff2b0101d30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.062+0000 7ff2b6600640 1 -- 172.21.15.97:0/1266708954 >> 172.21.15.97:0/1266708954 conn(0x7ff2b00fce80 msgr2=0x7ff2b00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.063+0000 7ff2b6600640 1 -- 172.21.15.97:0/1266708954 shutdown_connections 2024-03-26T19:07:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.063+0000 7ff2b6600640 1 -- 172.21.15.97:0/1266708954 wait complete. 2024-03-26T19:07:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.064+0000 7ff2b6600640 1 Processor -- start 2024-03-26T19:07:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.064+0000 7ff2b6600640 1 -- start start 2024-03-26T19:07:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.065+0000 7ff2b6600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b0101910 0x7ff2b0199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.065+0000 7ff2b6600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b0102b00 0x7ff2b019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.065+0000 7ff2b6600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b01034c0 0x7ff2b01a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.065+0000 7ff2b6600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b010d630 con 0x7ff2b01034c0 2024-03-26T19:07:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.065+0000 7ff2b6600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b010d4b0 con 0x7ff2b0102b00 2024-03-26T19:07:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.065+0000 7ff2b6600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b010d7b0 con 0x7ff2b0101910 2024-03-26T19:07:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.065+0000 7ff2aea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b0102b00 0x7ff2b019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.065+0000 7ff2aea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b0102b00 0x7ff2b019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:51234/0 (socket says 172.21.15.97:51234) 2024-03-26T19:07:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.065+0000 7ff2aea00640 1 -- 172.21.15.97:0/3776243270 learned_addr learned my addr 172.21.15.97:0/3776243270 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.066+0000 7ff2afe00640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b01034c0 0x7ff2b01a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.066+0000 7ff2aea00640 1 -- 172.21.15.97:0/3776243270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b0101910 msgr2=0x7ff2b0199df0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T19:07:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.066+0000 7ff2aea00640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b0101910 0x7ff2b0199df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.066+0000 7ff2aea00640 1 -- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b01034c0 msgr2=0x7ff2b01a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.066+0000 7ff2aea00640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b01034c0 0x7ff2b01a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.066+0000 7ff2aea00640 1 -- 172.21.15.97:0/3776243270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b01a1a10 con 0x7ff2b0102b00 2024-03-26T19:07:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.066+0000 7ff2afe00640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b01034c0 0x7ff2b01a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:07:36.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.067+0000 7ff2aea00640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b0102b00 0x7ff2b019a330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00e9f0 tx=0x7ff29c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.068+0000 7ff2a7e00640 1 -- 172.21.15.97:0/3776243270 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00cd80 con 0x7ff2b0102b00 2024-03-26T19:07:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.069+0000 7ff2a7e00640 1 -- 172.21.15.97:0/3776243270 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff29c010910 con 0x7ff2b0102b00 2024-03-26T19:07:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.069+0000 7ff2a7e00640 1 -- 172.21.15.97:0/3776243270 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c018990 con 0x7ff2b0102b00 2024-03-26T19:07:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.069+0000 7ff2b6600640 1 -- 172.21.15.97:0/3776243270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b0103d80 con 0x7ff2b0102b00 2024-03-26T19:07:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.069+0000 7ff2b6600640 1 -- 172.21.15.97:0/3776243270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b01042d0 con 0x7ff2b0102b00 2024-03-26T19:07:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.078+0000 7ff2b9392740 1 Processor -- start 2024-03-26T19:07:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.079+0000 7ff2a7e00640 1 -- 172.21.15.97:0/3776243270 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7ff29c010ac0 con 0x7ff2b0102b00 2024-03-26T19:07:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.080+0000 7ff2b9392740 1 -- start start 2024-03-26T19:07:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.081+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 0x5588a7b481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.081+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b4d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.081+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.081+0000 7ff2b9392740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5588a7aace80 con 0x5588a7b47da0 2024-03-26T19:07:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.081+0000 7ff2b9392740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5588a7a55660 con 0x5588a7b4d900 2024-03-26T19:07:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.081+0000 7ff2b9392740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5588a7aa8e30 con 0x5588a7b48790 2024-03-26T19:07:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.082+0000 7ff2afe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 0x5588a7b481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.082+0000 7ff2af400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b4d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.082+0000 7ff2af400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b4d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52038/0 (socket says 172.21.15.97:52038) 2024-03-26T19:07:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.082+0000 7ff2af400640 1 -- 172.21.15.97:0/68709790 learned_addr learned my addr 172.21.15.97:0/68709790 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.082+0000 7ff2aea00640 1 --2- 172.21.15.97:0/68709790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.083+0000 7ff2af400640 1 -- 172.21.15.97:0/68709790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 msgr2=0x5588a7b4fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.083+0000 7ff2af400640 1 --2- 172.21.15.97:0/68709790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b4fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.083+0000 7ff2af400640 1 -- 172.21.15.97:0/68709790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 msgr2=0x5588a7b481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.083+0000 7ff2af400640 1 --2- 172.21.15.97:0/68709790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 0x5588a7b481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.083+0000 7ff2af400640 1 -- 172.21.15.97:0/68709790 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a77c1210 con 0x5588a7b48790 2024-03-26T19:07:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.083+0000 7ff2a7e00640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff2940a3200 0x7ff2940a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.084+0000 7ff2a7e00640 1 -- 172.21.15.97:0/3776243270 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff29c0cf840 con 0x7ff2b0102b00 2024-03-26T19:07:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.084+0000 7ff2af400640 1 --2- 172.21.15.97:0/68709790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b4d290 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff2a002c790 tx=0x7ff2a002c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10b9fa3e80d0a460 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.085+0000 7ff2a4c00640 1 -- 172.21.15.97:0/68709790 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0043020 con 0x5588a7b48790 2024-03-26T19:07:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.085+0000 7ff2a4c00640 1 -- 172.21.15.97:0/68709790 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a0009500 con 0x5588a7b48790 2024-03-26T19:07:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.085+0000 7ff2a4c00640 1 -- 172.21.15.97:0/68709790 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0005590 con 0x5588a7b48790 2024-03-26T19:07:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.085+0000 7ff2afe00640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff2940a3200 0x7ff2940a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.086+0000 7ff2b9392740 1 -- 172.21.15.97:0/68709790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 msgr2=0x5588a7b4d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.086+0000 7ff2b9392740 1 --2- 172.21.15.97:0/68709790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b4d290 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff2a002c790 tx=0x7ff2a002c680 comp rx=0 tx=0).stop 2024-03-26T19:07:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.087+0000 7ff2b9392740 1 -- 172.21.15.97:0/68709790 shutdown_connections 2024-03-26T19:07:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.087+0000 7ff2b9392740 1 --2- 172.21.15.97:0/68709790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.087+0000 7ff2b9392740 1 --2- 172.21.15.97:0/68709790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b4d290 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.087+0000 7ff2b9392740 1 --2- 172.21.15.97:0/68709790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 0x5588a7b481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.087+0000 7ff2b9392740 1 -- 172.21.15.97:0/68709790 >> 172.21.15.97:0/68709790 conn(0x5588a7b43310 msgr2=0x5588a7b43710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.087+0000 7ff2afe00640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff2940a3200 0x7ff2940a56c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5588a7b48ff0 tx=0x7ff2a800e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.088+0000 7ff2b9392740 1 -- 172.21.15.97:0/68709790 shutdown_connections 2024-03-26T19:07:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.088+0000 7ff2b9392740 1 -- 172.21.15.97:0/68709790 wait complete. 2024-03-26T19:07:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.089+0000 7ff2b9392740 1 -- start start 2024-03-26T19:07:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.089+0000 7ff2b9392740 1 Processor -- start 2024-03-26T19:07:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.091+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b47da0 0x5588a7b51db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.091+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 0x5588a7b522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.091+0000 7ff2af400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b47da0 0x5588a7b51db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.091+0000 7ff2af400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b47da0 0x5588a7b51db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52050/0 (socket says 172.21.15.97:52050) 2024-03-26T19:07:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.092+0000 7ff2af400640 1 -- 172.21.15.97:0/3124266273 learned_addr learned my addr 172.21.15.97:0/3124266273 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.092+0000 7ff2b9392740 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.092+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5588a7b50850 con 0x5588a7b48790 2024-03-26T19:07:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.092+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5588a7a68ac0 con 0x5588a7b4d900 2024-03-26T19:07:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.092+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5588a7b509d0 con 0x5588a7b47da0 2024-03-26T19:07:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.092+0000 7ff2b9392740 10 client.0 _ll_register_callbacks cb 0x5588a7a68490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:07:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.092+0000 7ff2afe00640 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.093+0000 7ff2aea00640 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 0x5588a7b522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.094+0000 7ff2afe00640 1 -- 172.21.15.97:0/3124266273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b47da0 msgr2=0x5588a7b51db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.094+0000 7ff2afe00640 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b47da0 0x5588a7b51db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.094+0000 7ff2afe00640 1 -- 172.21.15.97:0/3124266273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 msgr2=0x5588a7b522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.094+0000 7ff2afe00640 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 0x5588a7b522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.094+0000 7ff2afe00640 1 -- 172.21.15.97:0/3124266273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a788d620 con 0x5588a7b4d900 2024-03-26T19:07:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.094+0000 7ff2aea00640 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 0x5588a7b522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:07:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.095+0000 7ff2afe00640 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b592c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800ce70 tx=0x7ff2a8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=958469e07a76e14e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.095+0000 7ff299600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:07:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.096+0000 7ff299600640 1 -- 172.21.15.97:0/3124266273 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a801e040 con 0x5588a7b4d900 2024-03-26T19:07:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.096+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a0009500 con 0x5588a7b4d900 2024-03-26T19:07:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.096+0000 7ff299600640 1 -- 172.21.15.97:0/3124266273 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff2a8022420 con 0x5588a7b4d900 2024-03-26T19:07:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.097+0000 7ff299600640 1 -- 172.21.15.97:0/3124266273 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2a8022620 con 0x5588a7b4d900 2024-03-26T19:07:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.097+0000 7ff2b9392740 10 client.25916 fetch_fsmap learned FSMap version 27 2024-03-26T19:07:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.097+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5588a7b5c6f0 con 0x5588a7b4d900 2024-03-26T19:07:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.099+0000 7ff299600640 1 -- 172.21.15.97:0/3124266273 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a0009500 con 0x5588a7b4d900 2024-03-26T19:07:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.099+0000 7ff299600640 1 -- 172.21.15.97:0/3124266273 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff2a802b860 con 0x5588a7b4d900 2024-03-26T19:07:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.099+0000 7ff2b9392740 10 client.25916 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:07:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.099+0000 7ff2b9392740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:07:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.099+0000 7ff2b9392740 20 client.25916 populate_metadata read hostname 'smithi097' 2024-03-26T19:07:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.099+0000 7ff2b9392740 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5588a7b5de90 0x5588a7b602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.099+0000 7ff2b9392740 4 client.25916 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:07:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.099+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588a7aba3c0 con 0x5588a7b5de90 2024-03-26T19:07:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.100+0000 7ff2af400640 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5588a7b5de90 0x5588a7b602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.100+0000 7ff2af400640 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5588a7b5de90 0x5588a7b602c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b588bb3ef15a249c server_cookie=d9f419afe9b7ecee in_seq=0 out_seq=0 2024-03-26T19:07:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.101+0000 7ff299600640 10 client.25916 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.102+0000 7ff299600640 1 -- 172.21.15.97:0/3124266273 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5588a7aba3c0 con 0x5588a7b5de90 2024-03-26T19:07:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.102+0000 7ff299600640 10 client.25916 handle_command_reply: tid=0 2024-03-26T19:07:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.102+0000 7ff2b9392740 1 client.25916 shutdown 2024-03-26T19:07:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.102+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 msgr2=0x5588a7b592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.102+0000 7ff2b9392740 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b592c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800ce70 tx=0x7ff2a8004a40 comp rx=0 tx=0).stop 2024-03-26T19:07:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.102+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 shutdown_connections 2024-03-26T19:07:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.102+0000 7ff2b9392740 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5588a7b5de90 0x5588a7b602c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.102+0000 7ff2b9392740 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b592c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.102+0000 7ff2b9392740 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 0x5588a7b522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.102+0000 7ff2b9392740 1 --2- 172.21.15.97:0/3124266273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b47da0 0x5588a7b51db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.103+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 >> 172.21.15.97:0/3124266273 conn(0x5588a7b43310 msgr2=0x5588a7b42540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.103+0000 7ff299600640 0 client.25916 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.104+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 shutdown_connections 2024-03-26T19:07:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.104+0000 7ff2b9392740 1 -- 172.21.15.97:0/3124266273 wait complete. 2024-03-26T19:07:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.104+0000 7ff2b9392740 20 client.25916 trim_cache size 0 max 16384 2024-03-26T19:07:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.138+0000 7ff2b9392740 1 Processor -- start 2024-03-26T19:07:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.138+0000 7ff2b9392740 1 -- start start 2024-03-26T19:07:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.139+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 0x5588a7b49910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.139+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b59280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.139+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b64bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.139+0000 7ff2b9392740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01af270 con 0x5588a7b47da0 2024-03-26T19:07:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.139+0000 7ff2b9392740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01b0db0 con 0x5588a7b4d900 2024-03-26T19:07:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.139+0000 7ff2b9392740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01af940 con 0x5588a7b48790 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.139+0000 7ff2aea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b59280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.139+0000 7ff2aea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b59280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52052/0 (socket says 172.21.15.97:52052) 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.139+0000 7ff2aea00640 1 -- 172.21.15.97:0/1120089027 learned_addr learned my addr 172.21.15.97:0/1120089027 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.139+0000 7ff2afe00640 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b64bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.140+0000 7ff2af400640 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 0x5588a7b49910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.140+0000 7ff2afe00640 1 -- 172.21.15.97:0/1120089027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 msgr2=0x5588a7b59280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.140+0000 7ff2afe00640 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b59280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.140+0000 7ff2afe00640 1 -- 172.21.15.97:0/1120089027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 msgr2=0x5588a7b49910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.140+0000 7ff2afe00640 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 0x5588a7b49910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.140+0000 7ff2afe00640 1 -- 172.21.15.97:0/1120089027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a7b41ab0 con 0x5588a7b4d900 2024-03-26T19:07:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.140+0000 7ff2aea00640 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b59280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:07:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.140+0000 7ff2af400640 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 0x5588a7b49910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:07:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.140+0000 7ff2afe00640 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b64bc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff2a802bd20 tx=0x7ff2a80093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=378e95ee05f0dd15 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.141+0000 7ff29be00640 1 -- 172.21.15.97:0/1120089027 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a803d070 con 0x5588a7b4d900 2024-03-26T19:07:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.141+0000 7ff29be00640 1 -- 172.21.15.97:0/1120089027 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a8027070 con 0x5588a7b4d900 2024-03-26T19:07:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.141+0000 7ff29be00640 1 -- 172.21.15.97:0/1120089027 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a801e040 con 0x5588a7b4d900 2024-03-26T19:07:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.142+0000 7ff2b9392740 1 -- 172.21.15.97:0/1120089027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 msgr2=0x5588a7b64bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.142+0000 7ff2b9392740 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b64bc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff2a802bd20 tx=0x7ff2a80093f0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.142+0000 7ff2b9392740 1 -- 172.21.15.97:0/1120089027 shutdown_connections 2024-03-26T19:07:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.142+0000 7ff2b9392740 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b4d900 0x5588a7b64bc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.142+0000 7ff2b9392740 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b48790 0x5588a7b59280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.142+0000 7ff2b9392740 1 --2- 172.21.15.97:0/1120089027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b47da0 0x5588a7b49910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.142+0000 7ff2b9392740 1 -- 172.21.15.97:0/1120089027 >> 172.21.15.97:0/1120089027 conn(0x5588a7b43310 msgr2=0x5588a7b49200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.143+0000 7ff2b9392740 1 -- 172.21.15.97:0/1120089027 shutdown_connections 2024-03-26T19:07:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.143+0000 7ff2b9392740 1 -- 172.21.15.97:0/1120089027 wait complete. 2024-03-26T19:07:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.144+0000 7ff2b9392740 1 -- start start 2024-03-26T19:07:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.145+0000 7ff2b9392740 1 Processor -- start 2024-03-26T19:07:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b47da0 0x5588a7b57070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 0x5588a7b575b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2b9392740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b4d900 0x5588a7b585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2b9392740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01b06e0 con 0x5588a7b48790 2024-03-26T19:07:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2b9392740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01b0010 con 0x5588a7b47da0 2024-03-26T19:07:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2b9392740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b01ad290 con 0x5588a7b4d900 2024-03-26T19:07:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2b9392740 10 client.0 _ll_register_callbacks cb 0x7ff2a8007ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:07:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2afe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b4d900 0x5588a7b585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2af400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b47da0 0x5588a7b57070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2afe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b4d900 0x5588a7b585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52064/0 (socket says 172.21.15.97:52064) 2024-03-26T19:07:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2afe00640 1 -- 172.21.15.97:0/1288112965 learned_addr learned my addr 172.21.15.97:0/1288112965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2af400640 1 -- 172.21.15.97:0/1288112965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b4d900 msgr2=0x5588a7b585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.146+0000 7ff2af400640 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b4d900 0x5588a7b585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.147+0000 7ff2af400640 1 -- 172.21.15.97:0/1288112965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 msgr2=0x5588a7b575b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:07:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.147+0000 7ff2aea00640 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 0x5588a7b575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.147+0000 7ff2af400640 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 0x5588a7b575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.147+0000 7ff2af400640 1 -- 172.21.15.97:0/1288112965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a7b3e730 con 0x5588a7b47da0 2024-03-26T19:07:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.147+0000 7ff2aea00640 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 0x5588a7b575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:07:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.147+0000 7ff2af400640 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b47da0 0x5588a7b57070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5588a7b60b70 tx=0x7ff2a00054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e19afbf905c0fa87 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.147+0000 7ff299600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:07:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.147+0000 7ff299600640 1 -- 172.21.15.97:0/1288112965 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0043040 con 0x5588a7b47da0 2024-03-26T19:07:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.148+0000 7ff299600640 1 -- 172.21.15.97:0/1288112965 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff2a002cb60 con 0x5588a7b47da0 2024-03-26T19:07:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.148+0000 7ff2b9392740 1 -- 172.21.15.97:0/1288112965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a8027070 con 0x5588a7b47da0 2024-03-26T19:07:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.148+0000 7ff299600640 1 -- 172.21.15.97:0/1288112965 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2a0004040 con 0x5588a7b47da0 2024-03-26T19:07:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.148+0000 7ff2b9392740 10 client.25922 fetch_fsmap learned FSMap version 27 2024-03-26T19:07:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.148+0000 7ff2b9392740 1 -- 172.21.15.97:0/1288112965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5588a7b60940 con 0x5588a7b47da0 2024-03-26T19:07:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.148+0000 7ff299600640 1 -- 172.21.15.97:0/1288112965 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a8027070 con 0x5588a7b47da0 2024-03-26T19:07:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.149+0000 7ff299600640 1 -- 172.21.15.97:0/1288112965 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff2a004d0d0 con 0x5588a7b47da0 2024-03-26T19:07:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.149+0000 7ff2b9392740 10 client.25922 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:07:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.149+0000 7ff2b9392740 10 client.25922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:07:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.149+0000 7ff2b9392740 20 client.25922 populate_metadata read hostname 'smithi097' 2024-03-26T19:07:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.149+0000 7ff2b9392740 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5588a7bd5b00 0x5588a7bd7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.149+0000 7ff2b9392740 4 client.25922 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:07:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.149+0000 7ff2b9392740 1 -- 172.21.15.97:0/1288112965 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588a7ab4710 con 0x5588a7bd5b00 2024-03-26T19:07:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.150+0000 7ff2aea00640 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5588a7bd5b00 0x5588a7bd7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.150+0000 7ff2aea00640 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5588a7bd5b00 0x5588a7bd7ef0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9d3594e1e01f78 server_cookie=e6423afcf3584275 in_seq=0 out_seq=0 2024-03-26T19:07:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.150+0000 7ff299600640 10 client.25922 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.175+0000 7ff299600640 1 -- 172.21.15.97:0/1288112965 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588a7ab4710 con 0x5588a7bd5b00 2024-03-26T19:07:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.175+0000 7ff299600640 10 client.25922 handle_command_reply: tid=0 2024-03-26T19:07:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.175+0000 7ff2b9392740 1 client.25922 shutdown 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 1 -- 172.21.15.97:0/1288112965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b47da0 msgr2=0x5588a7b57070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b47da0 0x5588a7b57070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5588a7b60b70 tx=0x7ff2a00054e0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 1 -- 172.21.15.97:0/1288112965 shutdown_connections 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5588a7bd5b00 0x5588a7bd7ef0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588a7b4d900 0x5588a7b585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588a7b48790 0x5588a7b575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 1 --2- 172.21.15.97:0/1288112965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5588a7b47da0 0x5588a7b57070 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 1 -- 172.21.15.97:0/1288112965 >> 172.21.15.97:0/1288112965 conn(0x5588a7b43310 msgr2=0x5588a7b42540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff299600640 0 client.25922 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 1 -- 172.21.15.97:0/1288112965 shutdown_connections 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 1 -- 172.21.15.97:0/1288112965 wait complete. 2024-03-26T19:07:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.176+0000 7ff2b9392740 20 client.25922 trim_cache size 0 max 16384 2024-03-26T19:07:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:07:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:07:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:07:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:07:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 -- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff2940a3200 msgr2=0x7ff2940a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff2940a3200 0x7ff2940a56c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5588a7b48ff0 tx=0x7ff2a800e040 comp rx=0 tx=0).stop 2024-03-26T19:07:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 -- 172.21.15.97:0/3776243270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b0102b00 msgr2=0x7ff2b019a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b0102b00 0x7ff2b019a330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00e9f0 tx=0x7ff29c00eeb0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 -- 172.21.15.97:0/3776243270 shutdown_connections 2024-03-26T19:07:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7ff2940a3200 0x7ff2940a56c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b01034c0 0x7ff2b01a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b0102b00 0x7ff2b019a330 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 --2- 172.21.15.97:0/3776243270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b0101910 0x7ff2b0199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 -- 172.21.15.97:0/3776243270 >> 172.21.15.97:0/3776243270 conn(0x7ff2b00fce80 msgr2=0x7ff2b0073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 -- 172.21.15.97:0/3776243270 shutdown_connections 2024-03-26T19:07:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:36.177+0000 7ff2b6600640 1 -- 172.21.15.97:0/3776243270 wait complete. 2024-03-26T19:07:36.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:36 smithi097 ceph-mon[31955]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 739 op/s 2024-03-26T19:07:36.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:36 smithi144 ceph-mon[36053]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 739 op/s 2024-03-26T19:07:36.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:36 smithi105 ceph-mon[36270]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 739 op/s 2024-03-26T19:07:38.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:38 smithi097 ceph-mon[31955]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 678 op/s 2024-03-26T19:07:38.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:38 smithi144 ceph-mon[36053]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 678 op/s 2024-03-26T19:07:38.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:38 smithi105 ceph-mon[36270]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 678 op/s 2024-03-26T19:07:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 495.0 s, 1589.4 tps, lat 63.077 ms stddev 75.940 2024-03-26T19:07:40.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:40 smithi097 ceph-mon[31955]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 794 op/s 2024-03-26T19:07:40.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:40 smithi144 ceph-mon[36053]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 794 op/s 2024-03-26T19:07:40.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:40 smithi105 ceph-mon[36270]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 794 op/s 2024-03-26T19:07:42.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:42 smithi097 ceph-mon[31955]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 786 op/s 2024-03-26T19:07:42.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:42 smithi144 ceph-mon[36053]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 786 op/s 2024-03-26T19:07:42.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:42 smithi105 ceph-mon[36270]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 786 op/s 2024-03-26T19:07:44.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:44 smithi097 ceph-mon[31955]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 723 op/s 2024-03-26T19:07:44.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:44 smithi144 ceph-mon[36053]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 723 op/s 2024-03-26T19:07:44.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:44 smithi105 ceph-mon[36270]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 723 op/s 2024-03-26T19:07:45.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 500.0 s, 1825.5 tps, lat 55.118 ms stddev 73.804 2024-03-26T19:07:46.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:46 smithi097 ceph-mon[31955]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 774 op/s 2024-03-26T19:07:46.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:46 smithi144 ceph-mon[36053]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 774 op/s 2024-03-26T19:07:46.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:46 smithi105 ceph-mon[36270]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 774 op/s 2024-03-26T19:07:48.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:48 smithi097 ceph-mon[31955]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 642 op/s 2024-03-26T19:07:48.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:48 smithi144 ceph-mon[36053]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 642 op/s 2024-03-26T19:07:48.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:48 smithi105 ceph-mon[36270]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 642 op/s 2024-03-26T19:07:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.370+0000 7f1adfe00640 1 -- 172.21.15.97:0/2771296865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ae01028f0 msgr2=0x7f1ae0104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.370+0000 7f1adfe00640 1 --2- 172.21.15.97:0/2771296865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ae01028f0 0x7f1ae0104d60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009900 tx=0x7f1acc02bc40 comp rx=0 tx=0).stop 2024-03-26T19:07:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.370+0000 7f1adfe00640 1 -- 172.21.15.97:0/2771296865 shutdown_connections 2024-03-26T19:07:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.371+0000 7f1adfe00640 1 --2- 172.21.15.97:0/2771296865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae01052a0 0x7f1ae0105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.371+0000 7f1adfe00640 1 --2- 172.21.15.97:0/2771296865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ae01028f0 0x7f1ae0104d60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.371+0000 7f1adfe00640 1 --2- 172.21.15.97:0/2771296865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ae00fff80 0x7f1ae01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.371+0000 7f1adfe00640 1 -- 172.21.15.97:0/2771296865 >> 172.21.15.97:0/2771296865 conn(0x7f1ae00fbba0 msgr2=0x7f1ae00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.371+0000 7f1adfe00640 1 -- 172.21.15.97:0/2771296865 shutdown_connections 2024-03-26T19:07:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.371+0000 7f1adfe00640 1 -- 172.21.15.97:0/2771296865 wait complete. 2024-03-26T19:07:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.373+0000 7f1adfe00640 1 Processor -- start 2024-03-26T19:07:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.373+0000 7f1adfe00640 1 -- start start 2024-03-26T19:07:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.374+0000 7f1adfe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae00fff80 0x7f1ae019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.374+0000 7f1adfe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ae01028f0 0x7f1ae019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.374+0000 7f1adfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ae01052a0 0x7f1ae01a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.374+0000 7f1adfe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01139b0 con 0x7f1ae01052a0 2024-03-26T19:07:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.374+0000 7f1adfe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0113830 con 0x7f1ae01028f0 2024-03-26T19:07:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.374+0000 7f1adfe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0113b30 con 0x7f1ae00fff80 2024-03-26T19:07:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.375+0000 7f1adea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae00fff80 0x7f1ae019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.375+0000 7f1ade000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ae01028f0 0x7f1ae019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.375+0000 7f1adea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae00fff80 0x7f1ae019dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:54626/0 (socket says 172.21.15.97:54626) 2024-03-26T19:07:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.375+0000 7f1adea00640 1 -- 172.21.15.97:0/2812332850 learned_addr learned my addr 172.21.15.97:0/2812332850 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.375+0000 7f1adf400640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ae01052a0 0x7f1ae01a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.376+0000 7f1ade000640 1 -- 172.21.15.97:0/2812332850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae00fff80 msgr2=0x7f1ae019dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.376+0000 7f1ade000640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae00fff80 0x7f1ae019dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.376+0000 7f1ade000640 1 -- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ae01052a0 msgr2=0x7f1ae01a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.377+0000 7f1ade000640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ae01052a0 0x7f1ae01a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.377+0000 7f1ade000640 1 -- 172.21.15.97:0/2812332850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae01a5bd0 con 0x7f1ae01028f0 2024-03-26T19:07:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.377+0000 7f1adf400640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ae01052a0 0x7f1ae01a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:07:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.377+0000 7f1ade000640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ae01028f0 0x7f1ae019e4f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1acc036040 tx=0x7f1acc005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.378+0000 7f1ae4a00640 1 -- 172.21.15.97:0/2812332850 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc041070 con 0x7f1ae01028f0 2024-03-26T19:07:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.378+0000 7f1ae4a00640 1 -- 172.21.15.97:0/2812332850 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc0044d0 con 0x7f1ae01028f0 2024-03-26T19:07:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.378+0000 7f1ae4a00640 1 -- 172.21.15.97:0/2812332850 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc034440 con 0x7f1ae01028f0 2024-03-26T19:07:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.378+0000 7f1adfe00640 1 -- 172.21.15.97:0/2812332850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae01a5e60 con 0x7f1ae01028f0 2024-03-26T19:07:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.378+0000 7f1adfe00640 1 -- 172.21.15.97:0/2812332850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae01a6270 con 0x7f1ae01028f0 2024-03-26T19:07:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.383+0000 7f1ae8ba8740 1 Processor -- start 2024-03-26T19:07:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.383+0000 7f1ae8ba8740 1 -- start start 2024-03-26T19:07:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.383+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 0x564312be3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.383+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 0x564312be8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.383+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312beaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.383+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564312b47e80 con 0x564312be2f60 2024-03-26T19:07:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.383+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564312af0660 con 0x564312be3950 2024-03-26T19:07:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.383+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564312b43e30 con 0x564312be8ac0 2024-03-26T19:07:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.383+0000 7f1ae4a00640 1 -- 172.21.15.97:0/2812332850 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f1acc044070 con 0x7f1ae01028f0 2024-03-26T19:07:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1adf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 0x564312be3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1adf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 0x564312be3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45088/0 (socket says 172.21.15.97:45088) 2024-03-26T19:07:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1adf400640 1 -- 172.21.15.97:0/2209164443 learned_addr learned my addr 172.21.15.97:0/2209164443 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1adea00640 1 --2- 172.21.15.97:0/2209164443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 0x564312be8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1ade000640 1 --2- 172.21.15.97:0/2209164443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312beaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1adf400640 1 -- 172.21.15.97:0/2209164443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 msgr2=0x564312beaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1adf400640 1 --2- 172.21.15.97:0/2209164443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312beaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1adf400640 1 -- 172.21.15.97:0/2209164443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 msgr2=0x564312be8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1adf400640 1 --2- 172.21.15.97:0/2209164443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 0x564312be8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1adf400640 1 -- 172.21.15.97:0/2209164443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431285c210 con 0x564312be2f60 2024-03-26T19:07:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.384+0000 7f1ade000640 1 --2- 172.21.15.97:0/2209164443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312beaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:07:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.385+0000 7f1adf400640 1 --2- 172.21.15.97:0/2209164443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 0x564312be3380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009fd0 tx=0x7f1ad80108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6e845545ac6d38 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.385+0000 7f1acbe00640 1 -- 172.21.15.97:0/2209164443 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8011760 con 0x564312be2f60 2024-03-26T19:07:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.385+0000 7f1acbe00640 1 -- 172.21.15.97:0/2209164443 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad8011de0 con 0x564312be2f60 2024-03-26T19:07:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.385+0000 7f1acbe00640 1 -- 172.21.15.97:0/2209164443 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800e3c0 con 0x564312be2f60 2024-03-26T19:07:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.385+0000 7f1ae4a00640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1ac40a2f90 0x7f1ac40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.385+0000 7f1ae4a00640 1 -- 172.21.15.97:0/2812332850 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1acc0f1180 con 0x7f1ae01028f0 2024-03-26T19:07:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.386+0000 7f1adea00640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1ac40a2f90 0x7f1ac40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.386+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/2209164443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 msgr2=0x564312be3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.386+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/2209164443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 0x564312be3380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009fd0 tx=0x7f1ad80108e0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.387+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/2209164443 shutdown_connections 2024-03-26T19:07:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.387+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/2209164443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312beaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.387+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/2209164443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 0x564312be8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.387+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/2209164443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 0x564312be3380 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.387+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/2209164443 >> 172.21.15.97:0/2209164443 conn(0x564312bde4d0 msgr2=0x564312bde8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.387+0000 7f1adea00640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1ac40a2f90 0x7f1ac40a5450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x564312be3d50 tx=0x7f1ad000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.388+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/2209164443 shutdown_connections 2024-03-26T19:07:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.388+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/2209164443 wait complete. 2024-03-26T19:07:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.389+0000 7f1ae8ba8740 1 -- start start 2024-03-26T19:07:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.389+0000 7f1ae8ba8740 1 Processor -- start 2024-03-26T19:07:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312becfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be3950 0x564312bed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be8ac0 0x564312bf4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564312beba70 con 0x564312be8ac0 2024-03-26T19:07:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564312b03ac0 con 0x564312be2f60 2024-03-26T19:07:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564312bebbf0 con 0x564312be3950 2024-03-26T19:07:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1ae8ba8740 10 client.0 _ll_register_callbacks cb 0x564312b03490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:07:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1adea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312becfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1ade000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be3950 0x564312bed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1adea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312becfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38198/0 (socket says 172.21.15.97:38198) 2024-03-26T19:07:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1adea00640 1 -- 172.21.15.97:0/860596093 learned_addr learned my addr 172.21.15.97:0/860596093 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1adf400640 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be8ac0 0x564312bf4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1adea00640 1 -- 172.21.15.97:0/860596093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be3950 msgr2=0x564312bed4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1adea00640 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be3950 0x564312bed4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1adea00640 1 -- 172.21.15.97:0/860596093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be8ac0 msgr2=0x564312bf4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1adea00640 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be8ac0 0x564312bf4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.390+0000 7f1adea00640 1 -- 172.21.15.97:0/860596093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564312928620 con 0x564312be2f60 2024-03-26T19:07:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.391+0000 7f1adf400640 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be8ac0 0x564312bf4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:07:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.391+0000 7f1adea00640 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312becfa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0003f40 tx=0x7f1ad00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e351bc2108e8ffd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.391+0000 7f1ac8c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:07:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.391+0000 7f1ac8c00640 1 -- 172.21.15.97:0/860596093 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0005220 con 0x564312be2f60 2024-03-26T19:07:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.391+0000 7f1ac8c00640 1 -- 172.21.15.97:0/860596093 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1ad00053c0 con 0x564312be2f60 2024-03-26T19:07:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.391+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/860596093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ad8011de0 con 0x564312be2f60 2024-03-26T19:07:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.391+0000 7f1ac8c00640 1 -- 172.21.15.97:0/860596093 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ad00055c0 con 0x564312be2f60 2024-03-26T19:07:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.391+0000 7f1ae8ba8740 10 client.25934 fetch_fsmap learned FSMap version 27 2024-03-26T19:07:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.391+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/860596093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564312bf7900 con 0x564312be2f60 2024-03-26T19:07:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.392+0000 7f1ac8c00640 1 -- 172.21.15.97:0/860596093 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad8011de0 con 0x564312be2f60 2024-03-26T19:07:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.392+0000 7f1ac8c00640 1 -- 172.21.15.97:0/860596093 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1ad0024070 con 0x564312be2f60 2024-03-26T19:07:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.392+0000 7f1ae8ba8740 10 client.25934 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:07:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.392+0000 7f1ae8ba8740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:07:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.392+0000 7f1ae8ba8740 20 client.25934 populate_metadata read hostname 'smithi097' 2024-03-26T19:07:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.392+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564312bf9040 0x564312bfb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.392+0000 7f1ae8ba8740 4 client.25934 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:07:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.392+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/860596093 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564312b553c0 con 0x564312bf9040 2024-03-26T19:07:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.392+0000 7f1ade000640 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564312bf9040 0x564312bfb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.393+0000 7f1ade000640 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564312bf9040 0x564312bfb470 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac73bff914c69a8 server_cookie=ba8f48fbbfc6d966 in_seq=0 out_seq=0 2024-03-26T19:07:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.393+0000 7f1ac8c00640 10 client.25934 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.394+0000 7f1ac8c00640 1 -- 172.21.15.97:0/860596093 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564312b553c0 con 0x564312bf9040 2024-03-26T19:07:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.394+0000 7f1ac8c00640 10 client.25934 handle_command_reply: tid=0 2024-03-26T19:07:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.394+0000 7f1ae8ba8740 1 client.25934 shutdown 2024-03-26T19:07:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.395+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/860596093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 msgr2=0x564312becfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.395+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312becfa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0003f40 tx=0x7f1ad00046c0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.396+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/860596093 shutdown_connections 2024-03-26T19:07:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.396+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564312bf9040 0x564312bfb470 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.396+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be8ac0 0x564312bf4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.396+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be3950 0x564312bed4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.396+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/860596093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312becfa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.396+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/860596093 >> 172.21.15.97:0/860596093 conn(0x564312bde4d0 msgr2=0x564312bdd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.396+0000 7f1ac8c00640 0 client.25934 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.397+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/860596093 shutdown_connections 2024-03-26T19:07:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.397+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/860596093 wait complete. 2024-03-26T19:07:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.398+0000 7f1ae8ba8740 20 client.25934 trim_cache size 0 max 16384 2024-03-26T19:07:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.421+0000 7f1ae8ba8740 1 Processor -- start 2024-03-26T19:07:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.421+0000 7f1ae8ba8740 1 -- start start 2024-03-26T19:07:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312bed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be3950 0x564312be1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312be41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01af090 con 0x564312be3950 2024-03-26T19:07:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01b0bd0 con 0x564312be2f60 2024-03-26T19:07:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01af760 con 0x564312be8ac0 2024-03-26T19:07:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1adea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312bed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1adea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312bed440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38210/0 (socket says 172.21.15.97:38210) 2024-03-26T19:07:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1adea00640 1 -- 172.21.15.97:0/3478582358 learned_addr learned my addr 172.21.15.97:0/3478582358 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1ade000640 1 --2- 172.21.15.97:0/3478582358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be3950 0x564312be1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.422+0000 7f1adf400640 1 --2- 172.21.15.97:0/3478582358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312be41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.423+0000 7f1adea00640 1 -- 172.21.15.97:0/3478582358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 msgr2=0x564312be41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.423+0000 7f1adea00640 1 --2- 172.21.15.97:0/3478582358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312be41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.423+0000 7f1adea00640 1 -- 172.21.15.97:0/3478582358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be3950 msgr2=0x564312be1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.423+0000 7f1adea00640 1 --2- 172.21.15.97:0/3478582358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be3950 0x564312be1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.423+0000 7f1adea00640 1 -- 172.21.15.97:0/3478582358 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564312bdef20 con 0x564312be2f60 2024-03-26T19:07:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.423+0000 7f1adea00640 1 --2- 172.21.15.97:0/3478582358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312bed440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0005f30 tx=0x7f1ad0005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42c0d49978894bf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.423+0000 7f1acb400640 1 -- 172.21.15.97:0/3478582358 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad001b870 con 0x564312be2f60 2024-03-26T19:07:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.423+0000 7f1acb400640 1 -- 172.21.15.97:0/3478582358 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad001ba10 con 0x564312be2f60 2024-03-26T19:07:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.423+0000 7f1acb400640 1 -- 172.21.15.97:0/3478582358 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0010750 con 0x564312be2f60 2024-03-26T19:07:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.424+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3478582358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 msgr2=0x564312bed440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.424+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/3478582358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312bed440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0005f30 tx=0x7f1ad0005f60 comp rx=0 tx=0).stop 2024-03-26T19:07:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.424+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3478582358 shutdown_connections 2024-03-26T19:07:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.424+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/3478582358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312be41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.424+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/3478582358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be3950 0x564312be1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.424+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/3478582358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be2f60 0x564312bed440 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.424+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3478582358 >> 172.21.15.97:0/3478582358 conn(0x564312bde4d0 msgr2=0x564312be1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.425+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3478582358 shutdown_connections 2024-03-26T19:07:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.425+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3478582358 wait complete. 2024-03-26T19:07:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.425+0000 7f1ae8ba8740 1 -- start start 2024-03-26T19:07:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.425+0000 7f1ae8ba8740 1 Processor -- start 2024-03-26T19:07:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 0x564312bf18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 0x564312bf1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ae8ba8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312bf2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01b0500 con 0x564312be2f60 2024-03-26T19:07:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01afe30 con 0x564312be3950 2024-03-26T19:07:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ae8ba8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01ad070 con 0x564312be8ac0 2024-03-26T19:07:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ae8ba8740 10 client.0 _ll_register_callbacks cb 0x7f1ad0003bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:07:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ade000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 0x564312bf1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ade000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 0x564312bf1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38214/0 (socket says 172.21.15.97:38214) 2024-03-26T19:07:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ade000640 1 -- 172.21.15.97:0/3956728318 learned_addr learned my addr 172.21.15.97:0/3956728318 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:07:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1adf400640 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312bf2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1adea00640 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 0x564312bf18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ade000640 1 -- 172.21.15.97:0/3956728318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 msgr2=0x564312bf2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ade000640 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312bf2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ade000640 1 -- 172.21.15.97:0/3956728318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 msgr2=0x564312bf18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:07:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ade000640 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 0x564312bf18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.426+0000 7f1ade000640 1 -- 172.21.15.97:0/3956728318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564312bf33d0 con 0x564312be3950 2024-03-26T19:07:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ade000640 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 0x564312bf1e20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1acc006010 tx=0x7f1acc02cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f61690b4aea97d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:07:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ac8c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:07:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ac8c00640 1 -- 172.21.15.97:0/3956728318 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0f0040 con 0x564312be3950 2024-03-26T19:07:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ac8c00640 1 -- 172.21.15.97:0/3956728318 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1acc0b9f50 con 0x564312be3950 2024-03-26T19:07:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3956728318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ad001ba10 con 0x564312be3950 2024-03-26T19:07:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ac8c00640 1 -- 172.21.15.97:0/3956728318 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1acc04b040 con 0x564312be3950 2024-03-26T19:07:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ae8ba8740 10 client.25940 fetch_fsmap learned FSMap version 27 2024-03-26T19:07:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3956728318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564312c703c0 con 0x564312be3950 2024-03-26T19:07:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ac8c00640 1 -- 172.21.15.97:0/3956728318 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad001ba10 con 0x564312be3950 2024-03-26T19:07:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ac8c00640 1 -- 172.21.15.97:0/3956728318 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1acc0036f0 con 0x564312be3950 2024-03-26T19:07:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ae8ba8740 10 client.25940 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:07:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ae8ba8740 10 client.25940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:07:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ae8ba8740 20 client.25940 populate_metadata read hostname 'smithi097' 2024-03-26T19:07:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.427+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564312c70cc0 0x564312c730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:07:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.428+0000 7f1ae8ba8740 4 client.25940 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:07:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.428+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3956728318 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564312b4f710 con 0x564312c70cc0 2024-03-26T19:07:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.428+0000 7f1adea00640 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564312c70cc0 0x564312c730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:07:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.428+0000 7f1adea00640 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564312c70cc0 0x564312c730e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e59b17cabc815c server_cookie=656a718ac8e1ebc in_seq=0 out_seq=0 2024-03-26T19:07:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.428+0000 7f1ac8c00640 10 client.25940 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.477+0000 7f1ac8c00640 1 -- 172.21.15.97:0/3956728318 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564312b4f710 con 0x564312c70cc0 2024-03-26T19:07:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.477+0000 7f1ac8c00640 10 client.25940 handle_command_reply: tid=0 2024-03-26T19:07:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.477+0000 7f1ae8ba8740 1 client.25940 shutdown 2024-03-26T19:07:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.477+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3956728318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 msgr2=0x564312bf1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.477+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 0x564312bf1e20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1acc006010 tx=0x7f1acc02cf90 comp rx=0 tx=0).stop 2024-03-26T19:07:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.477+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3956728318 shutdown_connections 2024-03-26T19:07:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.477+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564312c70cc0 0x564312c730e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.477+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564312be8ac0 0x564312bf2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.478+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564312be3950 0x564312bf1e20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.478+0000 7f1ae8ba8740 1 --2- 172.21.15.97:0/3956728318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564312be2f60 0x564312bf18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.478+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3956728318 >> 172.21.15.97:0/3956728318 conn(0x564312bde4d0 msgr2=0x564312bed2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.478+0000 7f1ac8c00640 0 client.25940 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:07:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.478+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3956728318 shutdown_connections 2024-03-26T19:07:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.479+0000 7f1ae8ba8740 1 -- 172.21.15.97:0/3956728318 wait complete. 2024-03-26T19:07:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.479+0000 7f1ae8ba8740 20 client.25940 trim_cache size 0 max 16384 2024-03-26T19:07:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:07:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:07:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:07:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:07:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 -- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1ac40a2f90 msgr2=0x7f1ac40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1ac40a2f90 0x7f1ac40a5450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x564312be3d50 tx=0x7f1ad000b480 comp rx=0 tx=0).stop 2024-03-26T19:07:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 -- 172.21.15.97:0/2812332850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ae01028f0 msgr2=0x7f1ae019e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:07:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ae01028f0 0x7f1ae019e4f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1acc036040 tx=0x7f1acc005d50 comp rx=0 tx=0).stop 2024-03-26T19:07:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 -- 172.21.15.97:0/2812332850 shutdown_connections 2024-03-26T19:07:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f1ac40a2f90 0x7f1ac40a5450 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ae01052a0 0x7f1ae01a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ae01028f0 0x7f1ae019e4f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 --2- 172.21.15.97:0/2812332850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae00fff80 0x7f1ae019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:07:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 -- 172.21.15.97:0/2812332850 >> 172.21.15.97:0/2812332850 conn(0x7f1ae00fbba0 msgr2=0x7f1ae0100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:07:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 -- 172.21.15.97:0/2812332850 shutdown_connections 2024-03-26T19:07:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:07:49.481+0000 7f1adfe00640 1 -- 172.21.15.97:0/2812332850 wait complete. 2024-03-26T19:07:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 505.0 s, 1806.5 tps, lat 55.296 ms stddev 66.875 2024-03-26T19:07:50.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:50 smithi097 ceph-mon[31955]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 721 op/s 2024-03-26T19:07:50.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:50 smithi144 ceph-mon[36053]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 721 op/s 2024-03-26T19:07:50.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:50 smithi105 ceph-mon[36270]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 721 op/s 2024-03-26T19:07:52.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:52 smithi097 ceph-mon[31955]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 702 op/s 2024-03-26T19:07:52.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:52 smithi144 ceph-mon[36053]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 702 op/s 2024-03-26T19:07:52.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:52 smithi105 ceph-mon[36270]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 702 op/s 2024-03-26T19:07:54.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:54 smithi097 ceph-mon[31955]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 619 op/s 2024-03-26T19:07:54.630 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:54 smithi144 ceph-mon[36053]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 619 op/s 2024-03-26T19:07:54.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:54 smithi105 ceph-mon[36270]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 619 op/s 2024-03-26T19:07:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 510.0 s, 1468.2 tps, lat 67.940 ms stddev 81.796 2024-03-26T19:07:56.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:56 smithi097 ceph-mon[31955]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 675 op/s 2024-03-26T19:07:56.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:56 smithi144 ceph-mon[36053]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 675 op/s 2024-03-26T19:07:56.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:56 smithi105 ceph-mon[36270]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 675 op/s 2024-03-26T19:07:58.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:07:58 smithi097 ceph-mon[31955]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 573 op/s 2024-03-26T19:07:58.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:07:58 smithi144 ceph-mon[36053]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 573 op/s 2024-03-26T19:07:58.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:07:58 smithi105 ceph-mon[36270]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 573 op/s 2024-03-26T19:08:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 515.0 s, 1402.6 tps, lat 70.882 ms stddev 81.289 2024-03-26T19:08:00.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:00 smithi097 ceph-mon[31955]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 681 op/s 2024-03-26T19:08:00.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:00 smithi144 ceph-mon[36053]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 681 op/s 2024-03-26T19:08:00.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:00 smithi105 ceph-mon[36270]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 681 op/s 2024-03-26T19:08:02.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:02 smithi097 ceph-mon[31955]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 678 op/s 2024-03-26T19:08:02.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:02 smithi144 ceph-mon[36053]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 678 op/s 2024-03-26T19:08:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.680+0000 7fe4e2e00640 1 -- 172.21.15.97:0/3992371707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc104cf0 msgr2=0x7fe4dc1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.680+0000 7fe4e2e00640 1 --2- 172.21.15.97:0/3992371707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc104cf0 0x7fe4dc1070e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009900 tx=0x7fe4c802bc30 comp rx=0 tx=0).stop 2024-03-26T19:08:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.680+0000 7fe4e2e00640 1 -- 172.21.15.97:0/3992371707 shutdown_connections 2024-03-26T19:08:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.680+0000 7fe4e2e00640 1 --2- 172.21.15.97:0/3992371707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4dc107620 0x7fe4dc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.680+0000 7fe4e2e00640 1 --2- 172.21.15.97:0/3992371707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc104cf0 0x7fe4dc1070e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.680+0000 7fe4e2e00640 1 --2- 172.21.15.97:0/3992371707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc1023c0 0x7fe4dc1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.680+0000 7fe4e2e00640 1 -- 172.21.15.97:0/3992371707 >> 172.21.15.97:0/3992371707 conn(0x7fe4dc0fbd50 msgr2=0x7fe4dc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.681+0000 7fe4e2e00640 1 -- 172.21.15.97:0/3992371707 shutdown_connections 2024-03-26T19:08:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.681+0000 7fe4e2e00640 1 -- 172.21.15.97:0/3992371707 wait complete. 2024-03-26T19:08:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.682+0000 7fe4e2e00640 1 Processor -- start 2024-03-26T19:08:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.682+0000 7fe4e2e00640 1 -- start start 2024-03-26T19:08:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.683+0000 7fe4e2e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc1023c0 0x7fe4dc199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.683+0000 7fe4e2e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4dc104cf0 0x7fe4dc19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.683+0000 7fe4e2e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc107620 0x7fe4dc1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.683+0000 7fe4e2e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc10f780 con 0x7fe4dc1023c0 2024-03-26T19:08:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.683+0000 7fe4e2e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc10f600 con 0x7fe4dc104cf0 2024-03-26T19:08:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.683+0000 7fe4e2e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc10f900 con 0x7fe4dc107620 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.683+0000 7fe4db400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4dc104cf0 0x7fe4dc19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.683+0000 7fe4db400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4dc104cf0 0x7fe4dc19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:49648/0 (socket says 172.21.15.97:49648) 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.683+0000 7fe4db400640 1 -- 172.21.15.97:0/1456877076 learned_addr learned my addr 172.21.15.97:0/1456877076 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.683+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc107620 0x7fe4dc1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.684+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc1023c0 0x7fe4dc199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.684+0000 7fe4e0e00640 1 -- 172.21.15.97:0/1456877076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4dc104cf0 msgr2=0x7fe4dc19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.684+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4dc104cf0 0x7fe4dc19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.684+0000 7fe4e0e00640 1 -- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc1023c0 msgr2=0x7fe4dc199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.684+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc1023c0 0x7fe4dc199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.684+0000 7fe4e0e00640 1 -- 172.21.15.97:0/1456877076 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc1a1930 con 0x7fe4dc107620 2024-03-26T19:08:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.684+0000 7fe4db400640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4dc104cf0 0x7fe4dc19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:08:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.684+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc1023c0 0x7fe4dc199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:08:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.685+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc107620 0x7fe4dc1a1290 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400e940 tx=0x7fe4d400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.685+0000 7fe4d8c00640 1 -- 172.21.15.97:0/1456877076 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4004d20 con 0x7fe4dc107620 2024-03-26T19:08:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.685+0000 7fe4d8c00640 1 -- 172.21.15.97:0/1456877076 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4005020 con 0x7fe4dc107620 2024-03-26T19:08:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.685+0000 7fe4d8c00640 1 -- 172.21.15.97:0/1456877076 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40116b0 con 0x7fe4dc107620 2024-03-26T19:08:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.685+0000 7fe4e2e00640 1 -- 172.21.15.97:0/1456877076 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc1a1c20 con 0x7fe4dc107620 2024-03-26T19:08:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.685+0000 7fe4e2e00640 1 -- 172.21.15.97:0/1456877076 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc1a2170 con 0x7fe4dc107620 2024-03-26T19:08:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.690+0000 7fe4d8c00640 1 -- 172.21.15.97:0/1456877076 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fe4d40054d0 con 0x7fe4dc107620 2024-03-26T19:08:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.691+0000 7fe4d8c00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe4c00a3110 0x7fe4c00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.691+0000 7fe4d8c00640 1 -- 172.21.15.97:0/1456877076 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe4d40cf4f0 con 0x7fe4dc107620 2024-03-26T19:08:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.692+0000 7fe4e5a6b740 1 Processor -- start 2024-03-26T19:08:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.692+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe4c00a3110 0x7fe4c00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.692+0000 7fe4e5a6b740 1 -- start start 2024-03-26T19:08:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.692+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c82da0 0x5613d9c831c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.692+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 0x5613d9c88290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c88900 0x5613d9c8ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4e5a6b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5613d9be7e80 con 0x5613d9c88900 2024-03-26T19:08:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4e5a6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5613d9b90660 con 0x5613d9c83790 2024-03-26T19:08:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4e5a6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5613d9be3e30 con 0x5613d9c82da0 2024-03-26T19:08:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe4c00a3110 0x7fe4c00a55d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005fd0 tx=0x7fe4cc005f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c82da0 0x5613d9c831c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c82da0 0x5613d9c831c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51082/0 (socket says 172.21.15.97:51082) 2024-03-26T19:08:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4dbe00640 1 -- 172.21.15.97:0/3701281155 learned_addr learned my addr 172.21.15.97:0/3701281155 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4db400640 1 --2- 172.21.15.97:0/3701281155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 0x5613d9c88290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4dbe00640 1 -- 172.21.15.97:0/3701281155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 msgr2=0x5613d9c88290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/3701281155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 0x5613d9c88290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4dbe00640 1 -- 172.21.15.97:0/3701281155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c88900 msgr2=0x5613d9c8ad90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:08:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.693+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/3701281155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c88900 0x5613d9c8ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.694+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/3701281155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c88900 0x5613d9c8ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.694+0000 7fe4dbe00640 1 -- 172.21.15.97:0/3701281155 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d98fc210 con 0x5613d9c82da0 2024-03-26T19:08:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.694+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/3701281155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c88900 0x5613d9c8ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:08:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.694+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/3701281155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c82da0 0x5613d9c831c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00ef00 tx=0x7fe4cc011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d61897a978baff98 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.694+0000 7fe4d1600640 1 -- 172.21.15.97:0/3701281155 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc021070 con 0x5613d9c82da0 2024-03-26T19:08:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.695+0000 7fe4d1600640 1 -- 172.21.15.97:0/3701281155 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc010e40 con 0x5613d9c82da0 2024-03-26T19:08:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.695+0000 7fe4d1600640 1 -- 172.21.15.97:0/3701281155 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc01da60 con 0x5613d9c82da0 2024-03-26T19:08:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.696+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3701281155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c82da0 msgr2=0x5613d9c831c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.696+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3701281155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c82da0 0x5613d9c831c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00ef00 tx=0x7fe4cc011000 comp rx=0 tx=0).stop 2024-03-26T19:08:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.696+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3701281155 shutdown_connections 2024-03-26T19:08:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.696+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3701281155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c88900 0x5613d9c8ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.696+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3701281155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 0x5613d9c88290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.696+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3701281155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c82da0 0x5613d9c831c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.696+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3701281155 >> 172.21.15.97:0/3701281155 conn(0x5613d9c7e310 msgr2=0x5613d9c7e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.697+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3701281155 shutdown_connections 2024-03-26T19:08:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.697+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3701281155 wait complete. 2024-03-26T19:08:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.697+0000 7fe4e5a6b740 1 -- start start 2024-03-26T19:08:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.698+0000 7fe4e5a6b740 1 Processor -- start 2024-03-26T19:08:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c95580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 0x5613d9c95ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c95580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c95580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:49664/0 (socket says 172.21.15.97:49664) 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4dbe00640 1 -- 172.21.15.97:0/3377852211 learned_addr learned my addr 172.21.15.97:0/3377852211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c9cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5613d9c8b8b0 con 0x5613d9c83790 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5613d9ba3ac0 con 0x5613d9c82da0 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5613d9c8ba30 con 0x5613d9c88900 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4e5a6b740 10 client.0 _ll_register_callbacks cb 0x5613d9ba3490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c9cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.699+0000 7fe4db400640 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 0x5613d9c95ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4dbe00640 1 -- 172.21.15.97:0/3377852211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 msgr2=0x5613d9c9cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c9cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4dbe00640 1 -- 172.21.15.97:0/3377852211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 msgr2=0x5613d9c95ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 0x5613d9c95ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4dbe00640 1 -- 172.21.15.97:0/3377852211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d99c8620 con 0x5613d9c82da0 2024-03-26T19:08:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4db400640 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 0x5613d9c95ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:08:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c9cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:08:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c95580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005200 tx=0x7fe4cc005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8758f195a357ac17 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4c6000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:08:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4c6000640 1 -- 172.21.15.97:0/3377852211 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc021040 con 0x5613d9c82da0 2024-03-26T19:08:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.700+0000 7fe4c6000640 1 -- 172.21.15.97:0/3377852211 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe4cc01d760 con 0x5613d9c82da0 2024-03-26T19:08:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.701+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5613d9bf53c0 con 0x5613d9c82da0 2024-03-26T19:08:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.701+0000 7fe4c6000640 1 -- 172.21.15.97:0/3377852211 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4cc00eb80 con 0x5613d9c82da0 2024-03-26T19:08:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.701+0000 7fe4e5a6b740 10 client.25952 fetch_fsmap learned FSMap version 27 2024-03-26T19:08:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.701+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613d9c9fdc0 con 0x5613d9c82da0 2024-03-26T19:08:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.703+0000 7fe4c6000640 1 -- 172.21.15.97:0/3377852211 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5613d9bf53c0 con 0x5613d9c82da0 2024-03-26T19:08:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.703+0000 7fe4c6000640 1 -- 172.21.15.97:0/3377852211 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4cc03b9f0 con 0x5613d9c82da0 2024-03-26T19:08:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.703+0000 7fe4e5a6b740 10 client.25952 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:08:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.703+0000 7fe4e5a6b740 10 client.25952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:08:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.703+0000 7fe4e5a6b740 20 client.25952 populate_metadata read hostname 'smithi097' 2024-03-26T19:08:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.704+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613d9ca11e0 0x5613d9ca36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.704+0000 7fe4e5a6b740 4 client.25952 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:08:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.704+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613d9bef710 con 0x5613d9ca11e0 2024-03-26T19:08:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.705+0000 7fe4db400640 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613d9ca11e0 0x5613d9ca36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.706+0000 7fe4db400640 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613d9ca11e0 0x5613d9ca36a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a8df03aaf3024c server_cookie=6f6185f299bd8a17 in_seq=0 out_seq=0 2024-03-26T19:08:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.706+0000 7fe4c6000640 10 client.25952 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.707+0000 7fe4c6000640 1 -- 172.21.15.97:0/3377852211 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5613d9bef710 con 0x5613d9ca11e0 2024-03-26T19:08:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.707+0000 7fe4c6000640 10 client.25952 handle_command_reply: tid=0 2024-03-26T19:08:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.707+0000 7fe4e5a6b740 1 client.25952 shutdown 2024-03-26T19:08:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.708+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 msgr2=0x5613d9c95580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.708+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c95580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005200 tx=0x7fe4cc005230 comp rx=0 tx=0).stop 2024-03-26T19:08:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.708+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 shutdown_connections 2024-03-26T19:08:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.708+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613d9ca11e0 0x5613d9ca36a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.708+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c9cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.708+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 0x5613d9c95ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.708+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/3377852211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c95580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.708+0000 7fe4c6000640 0 client.25952 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.708+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 >> 172.21.15.97:0/3377852211 conn(0x5613d9c7e310 msgr2=0x5613d9c7d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.709+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 shutdown_connections 2024-03-26T19:08:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.709+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/3377852211 wait complete. 2024-03-26T19:08:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.710+0000 7fe4e5a6b740 20 client.25952 trim_cache size 0 max 16384 2024-03-26T19:08:02.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:02 smithi105 ceph-mon[36270]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 678 op/s 2024-03-26T19:08:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.731+0000 7fe4e5a6b740 1 Processor -- start 2024-03-26T19:08:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.732+0000 7fe4e5a6b740 1 -- start start 2024-03-26T19:08:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.732+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c83ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.732+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 0x5613d9c9cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.732+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c84530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.732+0000 7fe4e5a6b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1a6a20 con 0x5613d9c83790 2024-03-26T19:08:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.732+0000 7fe4e5a6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1a8560 con 0x5613d9c82da0 2024-03-26T19:08:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.732+0000 7fe4e5a6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1a70f0 con 0x5613d9c88900 2024-03-26T19:08:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.733+0000 7fe4e0e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c84530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.733+0000 7fe4e0e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c84530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:51106/0 (socket says 172.21.15.97:51106) 2024-03-26T19:08:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.733+0000 7fe4e0e00640 1 -- 172.21.15.97:0/1950348558 learned_addr learned my addr 172.21.15.97:0/1950348558 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.733+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c83ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.733+0000 7fe4db400640 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 0x5613d9c9cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.734+0000 7fe4e0e00640 1 -- 172.21.15.97:0/1950348558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 msgr2=0x5613d9c83ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.734+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c83ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.734+0000 7fe4e0e00640 1 -- 172.21.15.97:0/1950348558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 msgr2=0x5613d9c9cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.734+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 0x5613d9c9cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.734+0000 7fe4e0e00640 1 -- 172.21.15.97:0/1950348558 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d9c7cab0 con 0x5613d9c88900 2024-03-26T19:08:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.734+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c83ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:08:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.734+0000 7fe4db400640 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 0x5613d9c9cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:08:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.735+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c84530 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4005c40 tx=0x7fe4d409b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82238bdb4ed9cf56 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.735+0000 7fe4d0c00640 1 -- 172.21.15.97:0/1950348558 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d401f040 con 0x5613d9c88900 2024-03-26T19:08:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.736+0000 7fe4d0c00640 1 -- 172.21.15.97:0/1950348558 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d409fd40 con 0x5613d9c88900 2024-03-26T19:08:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.736+0000 7fe4d0c00640 1 -- 172.21.15.97:0/1950348558 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d409b040 con 0x5613d9c88900 2024-03-26T19:08:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.737+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/1950348558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 msgr2=0x5613d9c84530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.737+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c84530 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4005c40 tx=0x7fe4d409b620 comp rx=0 tx=0).stop 2024-03-26T19:08:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.737+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/1950348558 shutdown_connections 2024-03-26T19:08:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.737+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c84530 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.737+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c83790 0x5613d9c9cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.737+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/1950348558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c82da0 0x5613d9c83ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.737+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/1950348558 >> 172.21.15.97:0/1950348558 conn(0x5613d9c7e310 msgr2=0x5613d9c7d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.738+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/1950348558 shutdown_connections 2024-03-26T19:08:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.738+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/1950348558 wait complete. 2024-03-26T19:08:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.738+0000 7fe4e5a6b740 1 -- start start 2024-03-26T19:08:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.739+0000 7fe4e5a6b740 1 Processor -- start 2024-03-26T19:08:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.739+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c82da0 0x5613d9c97d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.739+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 0x5613d9c98270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.739+0000 7fe4e5a6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c9b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.739+0000 7fe4e5a6b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1a7e90 con 0x5613d9c82da0 2024-03-26T19:08:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.739+0000 7fe4e5a6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1a77c0 con 0x5613d9c83790 2024-03-26T19:08:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.739+0000 7fe4e5a6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1a4a20 con 0x5613d9c88900 2024-03-26T19:08:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.739+0000 7fe4e5a6b740 10 client.0 _ll_register_callbacks cb 0x7fe4cc004b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:08:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.740+0000 7fe4db400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 0x5613d9c98270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.740+0000 7fe4db400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 0x5613d9c98270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:49686/0 (socket says 172.21.15.97:49686) 2024-03-26T19:08:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.740+0000 7fe4db400640 1 -- 172.21.15.97:0/807514583 learned_addr learned my addr 172.21.15.97:0/807514583 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.740+0000 7fe4e0e00640 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c9b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.740+0000 7fe4db400640 1 -- 172.21.15.97:0/807514583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 msgr2=0x5613d9c9b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.740+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c82da0 0x5613d9c97d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.740+0000 7fe4db400640 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c9b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.740+0000 7fe4db400640 1 -- 172.21.15.97:0/807514583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c82da0 msgr2=0x5613d9c97d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.740+0000 7fe4db400640 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c82da0 0x5613d9c97d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.740+0000 7fe4db400640 1 -- 172.21.15.97:0/807514583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d9c88650 con 0x5613d9c83790 2024-03-26T19:08:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.741+0000 7fe4db400640 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 0x5613d9c98270 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009850 tx=0x7fe4c8004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=febb82ad9616a7ba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.741+0000 7fe4c6000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:08:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.741+0000 7fe4c6000640 1 -- 172.21.15.97:0/807514583 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c804c070 con 0x5613d9c83790 2024-03-26T19:08:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.741+0000 7fe4c6000640 1 -- 172.21.15.97:0/807514583 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe4c8005310 con 0x5613d9c83790 2024-03-26T19:08:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.741+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/807514583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d409fd40 con 0x5613d9c83790 2024-03-26T19:08:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.742+0000 7fe4c6000640 1 -- 172.21.15.97:0/807514583 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4c8005510 con 0x5613d9c83790 2024-03-26T19:08:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.742+0000 7fe4e5a6b740 10 client.25955 fetch_fsmap learned FSMap version 27 2024-03-26T19:08:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.742+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/807514583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613d9c9c9d0 con 0x5613d9c83790 2024-03-26T19:08:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.742+0000 7fe4c6000640 1 -- 172.21.15.97:0/807514583 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d409fd40 con 0x5613d9c83790 2024-03-26T19:08:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.742+0000 7fe4c6000640 1 -- 172.21.15.97:0/807514583 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4c80037b0 con 0x5613d9c83790 2024-03-26T19:08:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.743+0000 7fe4e5a6b740 10 client.25955 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:08:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.743+0000 7fe4e5a6b740 10 client.25955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:08:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.743+0000 7fe4e5a6b740 20 client.25955 populate_metadata read hostname 'smithi097' 2024-03-26T19:08:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.743+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613d9c90130 0x5613d9c92550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.743+0000 7fe4e5a6b740 4 client.25955 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:08:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.743+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/807514583 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613d9b6d630 con 0x5613d9c90130 2024-03-26T19:08:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.744+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613d9c90130 0x5613d9c92550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.745+0000 7fe4dbe00640 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613d9c90130 0x5613d9c92550 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f75b83cb54e90b30 server_cookie=25f720f96f4fdedd in_seq=0 out_seq=0 2024-03-26T19:08:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.745+0000 7fe4c6000640 10 client.25955 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.773+0000 7fe4c6000640 1 -- 172.21.15.97:0/807514583 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613d9b6d630 con 0x5613d9c90130 2024-03-26T19:08:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.773+0000 7fe4c6000640 10 client.25955 handle_command_reply: tid=0 2024-03-26T19:08:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.773+0000 7fe4e5a6b740 1 client.25955 shutdown 2024-03-26T19:08:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.773+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/807514583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 msgr2=0x5613d9c98270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.773+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 0x5613d9c98270 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009850 tx=0x7fe4c8004480 comp rx=0 tx=0).stop 2024-03-26T19:08:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.774+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/807514583 shutdown_connections 2024-03-26T19:08:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.774+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613d9c90130 0x5613d9c92550 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.774+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d9c88900 0x5613d9c9b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.774+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d9c83790 0x5613d9c98270 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.774+0000 7fe4e5a6b740 1 --2- 172.21.15.97:0/807514583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613d9c82da0 0x5613d9c97d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.774+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/807514583 >> 172.21.15.97:0/807514583 conn(0x5613d9c7e310 msgr2=0x5613d9c7ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.774+0000 7fe4c6000640 0 client.25955 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.775+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/807514583 shutdown_connections 2024-03-26T19:08:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.775+0000 7fe4e5a6b740 1 -- 172.21.15.97:0/807514583 wait complete. 2024-03-26T19:08:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.775+0000 7fe4e5a6b740 20 client.25955 trim_cache size 0 max 16384 2024-03-26T19:08:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:08:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:08:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:08:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:08:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 -- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe4c00a3110 msgr2=0x7fe4c00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe4c00a3110 0x7fe4c00a55d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005fd0 tx=0x7fe4cc005f40 comp rx=0 tx=0).stop 2024-03-26T19:08:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 -- 172.21.15.97:0/1456877076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc107620 msgr2=0x7fe4dc1a1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc107620 0x7fe4dc1a1290 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400e940 tx=0x7fe4d400ee00 comp rx=0 tx=0).stop 2024-03-26T19:08:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 -- 172.21.15.97:0/1456877076 shutdown_connections 2024-03-26T19:08:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fe4c00a3110 0x7fe4c00a55d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc107620 0x7fe4dc1a1290 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4dc104cf0 0x7fe4dc19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 --2- 172.21.15.97:0/1456877076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc1023c0 0x7fe4dc199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 -- 172.21.15.97:0/1456877076 >> 172.21.15.97:0/1456877076 conn(0x7fe4dc0fbd50 msgr2=0x7fe4dc102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.777+0000 7fe4e2e00640 1 -- 172.21.15.97:0/1456877076 shutdown_connections 2024-03-26T19:08:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:02.778+0000 7fe4e2e00640 1 -- 172.21.15.97:0/1456877076 wait complete. 2024-03-26T19:08:04.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:04 smithi144 ceph-mon[36053]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 609 op/s 2024-03-26T19:08:04.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:04 smithi105 ceph-mon[36270]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 609 op/s 2024-03-26T19:08:04.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:04 smithi097 ceph-mon[31955]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 609 op/s 2024-03-26T19:08:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 520.0 s, 1588.7 tps, lat 63.498 ms stddev 82.704 2024-03-26T19:08:06.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:06 smithi144 ceph-mon[36053]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 685 op/s 2024-03-26T19:08:06.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:06 smithi105 ceph-mon[36270]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 685 op/s 2024-03-26T19:08:06.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:06 smithi097 ceph-mon[31955]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 685 op/s 2024-03-26T19:08:08.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:08 smithi105 ceph-mon[36270]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 599 op/s 2024-03-26T19:08:08.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:08 smithi097 ceph-mon[31955]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 599 op/s 2024-03-26T19:08:08.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:08 smithi144 ceph-mon[36053]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 599 op/s 2024-03-26T19:08:09.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:09 smithi105 ceph-mon[36270]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 663 op/s 2024-03-26T19:08:09.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:09 smithi097 ceph-mon[31955]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 663 op/s 2024-03-26T19:08:09.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:09 smithi144 ceph-mon[36053]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 663 op/s 2024-03-26T19:08:10.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 525.0 s, 1587.3 tps, lat 62.534 ms stddev 79.368 2024-03-26T19:08:12.584 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:12 smithi144 ceph-mon[36053]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 707 op/s 2024-03-26T19:08:12.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:12 smithi097 ceph-mon[31955]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 707 op/s 2024-03-26T19:08:12.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:12 smithi105 ceph-mon[36270]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 707 op/s 2024-03-26T19:08:14.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:14 smithi097 ceph-mon[31955]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 817 op/s 2024-03-26T19:08:14.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:14 smithi144 ceph-mon[36053]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 817 op/s 2024-03-26T19:08:14.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:14 smithi105 ceph-mon[36270]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 817 op/s 2024-03-26T19:08:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 530.0 s, 1278.0 tps, lat 78.207 ms stddev 149.891 2024-03-26T19:08:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.958+0000 7fa776000640 1 -- 172.21.15.97:0/4047411945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7701054c0 msgr2=0x7fa770105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.958+0000 7fa776000640 1 --2- 172.21.15.97:0/4047411945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7701054c0 0x7fa770105940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa768009ca0 tx=0x7fa76802c100 comp rx=0 tx=0).stop 2024-03-26T19:08:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.958+0000 7fa776000640 1 -- 172.21.15.97:0/4047411945 shutdown_connections 2024-03-26T19:08:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.959+0000 7fa776000640 1 --2- 172.21.15.97:0/4047411945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7701054c0 0x7fa770105940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.959+0000 7fa776000640 1 --2- 172.21.15.97:0/4047411945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa770102b10 0x7fa770104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.959+0000 7fa776000640 1 --2- 172.21.15.97:0/4047411945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7701001a0 0x7fa7701025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.959+0000 7fa776000640 1 -- 172.21.15.97:0/4047411945 >> 172.21.15.97:0/4047411945 conn(0x7fa7700fbda0 msgr2=0x7fa7700fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.959+0000 7fa776000640 1 -- 172.21.15.97:0/4047411945 shutdown_connections 2024-03-26T19:08:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.959+0000 7fa776000640 1 -- 172.21.15.97:0/4047411945 wait complete. 2024-03-26T19:08:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.960+0000 7fa776000640 1 Processor -- start 2024-03-26T19:08:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.960+0000 7fa776000640 1 -- start start 2024-03-26T19:08:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa776000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7701001a0 0x7fa77019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa776000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa770102b10 0x7fa77019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa776000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7701054c0 0x7fa7701a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa776000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa770113bb0 con 0x7fa7701001a0 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa776000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa770113a30 con 0x7fa770102b10 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa776000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa770113d30 con 0x7fa7701054c0 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa76f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7701001a0 0x7fa77019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa76f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7701001a0 0x7fa77019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49406/0 (socket says 172.21.15.97:49406) 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa76f400640 1 -- 172.21.15.97:0/3057310786 learned_addr learned my addr 172.21.15.97:0/3057310786 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa76fe00640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7701054c0 0x7fa7701a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.961+0000 7fa76ea00640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa770102b10 0x7fa77019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.962+0000 7fa76f400640 1 -- 172.21.15.97:0/3057310786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7701054c0 msgr2=0x7fa7701a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.962+0000 7fa76f400640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7701054c0 0x7fa7701a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.962+0000 7fa76f400640 1 -- 172.21.15.97:0/3057310786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa770102b10 msgr2=0x7fa77019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.962+0000 7fa76f400640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa770102b10 0x7fa77019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.962+0000 7fa76f400640 1 -- 172.21.15.97:0/3057310786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7701a5d00 con 0x7fa7701001a0 2024-03-26T19:08:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.962+0000 7fa76ea00640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa770102b10 0x7fa77019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:08:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.962+0000 7fa76fe00640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7701054c0 0x7fa7701a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:08:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.962+0000 7fa76f400640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7701001a0 0x7fa77019e0e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa76000d770 tx=0x7fa76000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.963+0000 7fa767e00640 1 -- 172.21.15.97:0/3057310786 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760017070 con 0x7fa7701001a0 2024-03-26T19:08:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.963+0000 7fa767e00640 1 -- 172.21.15.97:0/3057310786 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa760005500 con 0x7fa7701001a0 2024-03-26T19:08:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.963+0000 7fa767e00640 1 -- 172.21.15.97:0/3057310786 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760004e00 con 0x7fa7701001a0 2024-03-26T19:08:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.963+0000 7fa776000640 1 -- 172.21.15.97:0/3057310786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7701a5ff0 con 0x7fa7701001a0 2024-03-26T19:08:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.963+0000 7fa776000640 1 -- 172.21.15.97:0/3057310786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7701a6400 con 0x7fa7701001a0 2024-03-26T19:08:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.970+0000 7fa767e00640 1 -- 172.21.15.97:0/3057310786 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fa760009c30 con 0x7fa7701001a0 2024-03-26T19:08:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.970+0000 7fa7789fd740 1 Processor -- start 2024-03-26T19:08:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.970+0000 7fa7789fd740 1 -- start start 2024-03-26T19:08:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.971+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 0x55c1f0e451c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.971+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e45790 0x55c1f0e4a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.971+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e4cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.971+0000 7fa7789fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c1f0da9e80 con 0x55c1f0e44da0 2024-03-26T19:08:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.971+0000 7fa7789fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c1f0d52660 con 0x55c1f0e4a900 2024-03-26T19:08:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.971+0000 7fa7789fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c1f0da5e30 con 0x55c1f0e45790 2024-03-26T19:08:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.971+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 0x55c1f0e451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.971+0000 7fa76f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e45790 0x55c1f0e4a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.971+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 0x55c1f0e451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49412/0 (socket says 172.21.15.97:49412) 2024-03-26T19:08:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.971+0000 7fa76fe00640 1 -- 172.21.15.97:0/4168496536 learned_addr learned my addr 172.21.15.97:0/4168496536 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.972+0000 7fa76f400640 1 -- 172.21.15.97:0/4168496536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 msgr2=0x55c1f0e4cd90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:08:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.972+0000 7fa76f400640 1 --2- 172.21.15.97:0/4168496536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e4cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.972+0000 7fa76f400640 1 -- 172.21.15.97:0/4168496536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 msgr2=0x55c1f0e451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.972+0000 7fa76f400640 1 --2- 172.21.15.97:0/4168496536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 0x55c1f0e451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.972+0000 7fa76f400640 1 -- 172.21.15.97:0/4168496536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1f0abe210 con 0x55c1f0e45790 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.972+0000 7fa76f400640 1 --2- 172.21.15.97:0/4168496536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e45790 0x55c1f0e4a290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa7600cc970 tx=0x7fa7600d7570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3576a56c2af31101 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.972+0000 7fa767e00640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa7540a2fc0 0x7fa7540a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.972+0000 7fa75fe00640 1 -- 172.21.15.97:0/4168496536 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600e2070 con 0x55c1f0e45790 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.972+0000 7fa75fe00640 1 -- 172.21.15.97:0/4168496536 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76009bcd0 con 0x55c1f0e45790 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.972+0000 7fa75fe00640 1 -- 172.21.15.97:0/4168496536 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76009af30 con 0x55c1f0e45790 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.973+0000 7fa76fe00640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa7540a2fc0 0x7fa7540a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.973+0000 7fa767e00640 1 -- 172.21.15.97:0/3057310786 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa7600cee70 con 0x7fa7701001a0 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.973+0000 7fa7789fd740 1 -- 172.21.15.97:0/4168496536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e45790 msgr2=0x55c1f0e4a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.973+0000 7fa7789fd740 1 --2- 172.21.15.97:0/4168496536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e45790 0x55c1f0e4a290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa7600cc970 tx=0x7fa7600d7570 comp rx=0 tx=0).stop 2024-03-26T19:08:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.973+0000 7fa7789fd740 1 -- 172.21.15.97:0/4168496536 shutdown_connections 2024-03-26T19:08:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.973+0000 7fa7789fd740 1 --2- 172.21.15.97:0/4168496536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e4cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.973+0000 7fa7789fd740 1 --2- 172.21.15.97:0/4168496536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e45790 0x55c1f0e4a290 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.973+0000 7fa7789fd740 1 --2- 172.21.15.97:0/4168496536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 0x55c1f0e451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.973+0000 7fa7789fd740 1 -- 172.21.15.97:0/4168496536 >> 172.21.15.97:0/4168496536 conn(0x55c1f0e40310 msgr2=0x55c1f0e40710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.975+0000 7fa7789fd740 1 -- 172.21.15.97:0/4168496536 shutdown_connections 2024-03-26T19:08:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.975+0000 7fa76fe00640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa7540a2fc0 0x7fa7540a5480 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa768006010 tx=0x7fa768032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.975+0000 7fa7789fd740 1 -- 172.21.15.97:0/4168496536 wait complete. 2024-03-26T19:08:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.976+0000 7fa7789fd740 1 -- start start 2024-03-26T19:08:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.976+0000 7fa7789fd740 1 Processor -- start 2024-03-26T19:08:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e4eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 0x55c1f0e4f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e562b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa7789fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c1f0e4d8b0 con 0x55c1f0e45790 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa7789fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c1f0d65ac0 con 0x55c1f0e4a900 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa7789fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c1f0e4da30 con 0x55c1f0e44da0 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa7789fd740 10 client.0 _ll_register_callbacks cb 0x55c1f0d65490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa76f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e4eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa76f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e4eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52702/0 (socket says 172.21.15.97:52702) 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa76f400640 1 -- 172.21.15.97:0/1234127529 learned_addr learned my addr 172.21.15.97:0/1234127529 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa76fe00640 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa76f400640 1 -- 172.21.15.97:0/1234127529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 msgr2=0x55c1f0e562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa76f400640 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa76f400640 1 -- 172.21.15.97:0/1234127529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 msgr2=0x55c1f0e4f2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:08:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.977+0000 7fa76f400640 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 0x55c1f0e4f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa76f400640 1 -- 172.21.15.97:0/1234127529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1f0b8a620 con 0x55c1f0e44da0 2024-03-26T19:08:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa76fe00640 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:08:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa76f400640 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e4eda0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa76009b560 tx=0x7fa76006e750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73d64daa25b23034 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa75cc00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:08:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa75cc00640 1 -- 172.21.15.97:0/1234127529 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76006e370 con 0x55c1f0e44da0 2024-03-26T19:08:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa75cc00640 1 -- 172.21.15.97:0/1234127529 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa76006c6d0 con 0x55c1f0e44da0 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa7789fd740 1 -- 172.21.15.97:0/1234127529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa76009bcd0 con 0x55c1f0e44da0 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa75cc00640 1 -- 172.21.15.97:0/1234127529 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa76006e5b0 con 0x55c1f0e44da0 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa7789fd740 10 client.25927 fetch_fsmap learned FSMap version 27 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa7789fd740 1 -- 172.21.15.97:0/1234127529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c1f0e4d8b0 con 0x55c1f0e44da0 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.978+0000 7fa75cc00640 1 -- 172.21.15.97:0/1234127529 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76009bcd0 con 0x55c1f0e44da0 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.979+0000 7fa75cc00640 1 -- 172.21.15.97:0/1234127529 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa760097c40 con 0x55c1f0e44da0 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.979+0000 7fa7789fd740 10 client.25927 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.979+0000 7fa7789fd740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.979+0000 7fa7789fd740 20 client.25927 populate_metadata read hostname 'smithi097' 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.979+0000 7fa7789fd740 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c1f0e5a7f0 0x55c1f0e5cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.979+0000 7fa7789fd740 4 client.25927 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.979+0000 7fa7789fd740 1 -- 172.21.15.97:0/1234127529 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1f0db73c0 con 0x55c1f0e5a7f0 2024-03-26T19:08:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.979+0000 7fa76ea00640 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c1f0e5a7f0 0x55c1f0e5cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.980+0000 7fa76ea00640 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c1f0e5a7f0 0x55c1f0e5cbe0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9271eb8aa078574b server_cookie=627c81ea2e79f8e8 in_seq=0 out_seq=0 2024-03-26T19:08:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.980+0000 7fa75cc00640 10 client.25927 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.981+0000 7fa75cc00640 1 -- 172.21.15.97:0/1234127529 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c1f0db73c0 con 0x55c1f0e5a7f0 2024-03-26T19:08:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.981+0000 7fa75cc00640 10 client.25927 handle_command_reply: tid=0 2024-03-26T19:08:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.981+0000 7fa7789fd740 1 client.25927 shutdown 2024-03-26T19:08:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.982+0000 7fa7789fd740 1 -- 172.21.15.97:0/1234127529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 msgr2=0x55c1f0e4eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.982+0000 7fa7789fd740 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e4eda0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa76009b560 tx=0x7fa76006e750 comp rx=0 tx=0).stop 2024-03-26T19:08:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.983+0000 7fa7789fd740 1 -- 172.21.15.97:0/1234127529 shutdown_connections 2024-03-26T19:08:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.983+0000 7fa7789fd740 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c1f0e5a7f0 0x55c1f0e5cbe0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.983+0000 7fa7789fd740 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.983+0000 7fa7789fd740 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 0x55c1f0e4f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.983+0000 7fa7789fd740 1 --2- 172.21.15.97:0/1234127529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e4eda0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.983+0000 7fa7789fd740 1 -- 172.21.15.97:0/1234127529 >> 172.21.15.97:0/1234127529 conn(0x55c1f0e40310 msgr2=0x55c1f0e3f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.983+0000 7fa75cc00640 0 client.25927 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.983+0000 7fa7789fd740 1 -- 172.21.15.97:0/1234127529 shutdown_connections 2024-03-26T19:08:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.984+0000 7fa7789fd740 1 -- 172.21.15.97:0/1234127529 wait complete. 2024-03-26T19:08:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:15.984+0000 7fa7789fd740 20 client.25927 trim_cache size 0 max 16384 2024-03-26T19:08:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.012+0000 7fa7789fd740 1 Processor -- start 2024-03-26T19:08:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.012+0000 7fa7789fd740 1 -- start start 2024-03-26T19:08:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.013+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 0x55c1f0e4e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.013+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e45790 0x55c1f0e4e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.013+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e4a900 0x55c1f0e4ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.013+0000 7fa7789fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701af1f0 con 0x55c1f0e44da0 2024-03-26T19:08:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.013+0000 7fa7789fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701b0d30 con 0x55c1f0e45790 2024-03-26T19:08:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.013+0000 7fa7789fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701af8c0 con 0x55c1f0e4a900 2024-03-26T19:08:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.013+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e4a900 0x55c1f0e4ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.013+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e4a900 0x55c1f0e4ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:52708/0 (socket says 172.21.15.97:52708) 2024-03-26T19:08:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.013+0000 7fa76fe00640 1 -- 172.21.15.97:0/1435795908 learned_addr learned my addr 172.21.15.97:0/1435795908 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.013+0000 7fa76ea00640 1 --2- 172.21.15.97:0/1435795908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e45790 0x55c1f0e4e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.014+0000 7fa76f400640 1 --2- 172.21.15.97:0/1435795908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 0x55c1f0e4e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.014+0000 7fa76fe00640 1 -- 172.21.15.97:0/1435795908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e45790 msgr2=0x55c1f0e4e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.014+0000 7fa76fe00640 1 --2- 172.21.15.97:0/1435795908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e45790 0x55c1f0e4e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.014+0000 7fa76fe00640 1 -- 172.21.15.97:0/1435795908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 msgr2=0x55c1f0e4e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.014+0000 7fa76fe00640 1 --2- 172.21.15.97:0/1435795908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 0x55c1f0e4e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.014+0000 7fa76fe00640 1 -- 172.21.15.97:0/1435795908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1f0e41080 con 0x55c1f0e4a900 2024-03-26T19:08:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.015+0000 7fa76fe00640 1 --2- 172.21.15.97:0/1435795908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e4a900 0x55c1f0e4ee40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa7680324c0 tx=0x7fa768041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad054f0fbb0087a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.016+0000 7fa75f400640 1 -- 172.21.15.97:0/1435795908 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768051070 con 0x55c1f0e4a900 2024-03-26T19:08:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.016+0000 7fa75f400640 1 -- 172.21.15.97:0/1435795908 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa768003cf0 con 0x55c1f0e4a900 2024-03-26T19:08:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.016+0000 7fa75f400640 1 -- 172.21.15.97:0/1435795908 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768039040 con 0x55c1f0e4a900 2024-03-26T19:08:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.017+0000 7fa7789fd740 1 -- 172.21.15.97:0/1435795908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e4a900 msgr2=0x55c1f0e4ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.017+0000 7fa7789fd740 1 --2- 172.21.15.97:0/1435795908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e4a900 0x55c1f0e4ee40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa7680324c0 tx=0x7fa768041000 comp rx=0 tx=0).stop 2024-03-26T19:08:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.018+0000 7fa7789fd740 1 -- 172.21.15.97:0/1435795908 shutdown_connections 2024-03-26T19:08:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.018+0000 7fa7789fd740 1 --2- 172.21.15.97:0/1435795908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e4a900 0x55c1f0e4ee40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.018+0000 7fa7789fd740 1 --2- 172.21.15.97:0/1435795908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e45790 0x55c1f0e4e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.018+0000 7fa7789fd740 1 --2- 172.21.15.97:0/1435795908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e44da0 0x55c1f0e4e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.018+0000 7fa7789fd740 1 -- 172.21.15.97:0/1435795908 >> 172.21.15.97:0/1435795908 conn(0x55c1f0e40310 msgr2=0x55c1f0e46a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.019+0000 7fa7789fd740 1 -- 172.21.15.97:0/1435795908 shutdown_connections 2024-03-26T19:08:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.019+0000 7fa7789fd740 1 -- 172.21.15.97:0/1435795908 wait complete. 2024-03-26T19:08:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.020+0000 7fa7789fd740 1 -- start start 2024-03-26T19:08:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.020+0000 7fa7789fd740 1 Processor -- start 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.021+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e53ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.021+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 0x55c1f0e53ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.021+0000 7fa7789fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.021+0000 7fa76ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 0x55c1f0e53ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.022+0000 7fa76ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 0x55c1f0e53ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49434/0 (socket says 172.21.15.97:49434) 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.023+0000 7fa7789fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701b0660 con 0x55c1f0e45790 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.023+0000 7fa7789fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701aff90 con 0x55c1f0e4a900 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.023+0000 7fa7789fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701ad1d0 con 0x55c1f0e44da0 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.023+0000 7fa76ea00640 1 -- 172.21.15.97:0/294244672 learned_addr learned my addr 172.21.15.97:0/294244672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.023+0000 7fa76f400640 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e53ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.023+0000 7fa76fe00640 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e55000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.023+0000 7fa7789fd740 10 client.0 _ll_register_callbacks cb 0x7fa7600d2e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:08:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.023+0000 7fa76fe00640 1 -- 172.21.15.97:0/294244672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 msgr2=0x55c1f0e53ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.024+0000 7fa76fe00640 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e53ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.024+0000 7fa76fe00640 1 -- 172.21.15.97:0/294244672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 msgr2=0x55c1f0e53ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.024+0000 7fa76fe00640 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 0x55c1f0e53ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.024+0000 7fa76fe00640 1 -- 172.21.15.97:0/294244672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1f0e555f0 con 0x55c1f0e4a900 2024-03-26T19:08:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.024+0000 7fa76f400640 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e53ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:08:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.024+0000 7fa76ea00640 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 0x55c1f0e53ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:08:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.024+0000 7fa76fe00640 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e55000 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa76802c640 tx=0x7fa76802c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8bbafced154a14 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.024+0000 7fa75cc00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:08:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.025+0000 7fa75cc00640 1 -- 172.21.15.97:0/294244672 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76805d070 con 0x55c1f0e4a900 2024-03-26T19:08:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.025+0000 7fa75cc00640 1 -- 172.21.15.97:0/294244672 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa7680347f0 con 0x55c1f0e4a900 2024-03-26T19:08:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.025+0000 7fa7789fd740 1 -- 172.21.15.97:0/294244672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa768003cf0 con 0x55c1f0e4a900 2024-03-26T19:08:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.026+0000 7fa75cc00640 1 -- 172.21.15.97:0/294244672 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa768067050 con 0x55c1f0e4a900 2024-03-26T19:08:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.026+0000 7fa7789fd740 10 client.25967 fetch_fsmap learned FSMap version 27 2024-03-26T19:08:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.026+0000 7fa7789fd740 1 -- 172.21.15.97:0/294244672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c1f0ed1fc0 con 0x55c1f0e4a900 2024-03-26T19:08:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.027+0000 7fa75cc00640 1 -- 172.21.15.97:0/294244672 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa768003cf0 con 0x55c1f0e4a900 2024-03-26T19:08:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.028+0000 7fa75cc00640 1 -- 172.21.15.97:0/294244672 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa76804db90 con 0x55c1f0e4a900 2024-03-26T19:08:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.028+0000 7fa7789fd740 10 client.25967 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:08:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.028+0000 7fa7789fd740 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:08:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.028+0000 7fa7789fd740 20 client.25967 populate_metadata read hostname 'smithi097' 2024-03-26T19:08:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.028+0000 7fa7789fd740 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c1f0ed2960 0x55c1f0ed4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.028+0000 7fa7789fd740 4 client.25967 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:08:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.028+0000 7fa7789fd740 1 -- 172.21.15.97:0/294244672 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1f0db1710 con 0x55c1f0ed2960 2024-03-26T19:08:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.029+0000 7fa76f400640 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c1f0ed2960 0x55c1f0ed4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.030+0000 7fa76f400640 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c1f0ed2960 0x55c1f0ed4d80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f9a8c5458af488 server_cookie=bd4f3be4c4904e92 in_seq=0 out_seq=0 2024-03-26T19:08:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.030+0000 7fa75cc00640 10 client.25967 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.172+0000 7fa75cc00640 1 -- 172.21.15.97:0/294244672 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1f0db1710 con 0x55c1f0ed2960 2024-03-26T19:08:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.172+0000 7fa75cc00640 10 client.25967 handle_command_reply: tid=0 2024-03-26T19:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.172+0000 7fa7789fd740 1 client.25967 shutdown 2024-03-26T19:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.173+0000 7fa7789fd740 1 -- 172.21.15.97:0/294244672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 msgr2=0x55c1f0e55000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.173+0000 7fa7789fd740 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e55000 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa76802c640 tx=0x7fa76802c840 comp rx=0 tx=0).stop 2024-03-26T19:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.173+0000 7fa7789fd740 1 -- 172.21.15.97:0/294244672 shutdown_connections 2024-03-26T19:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.173+0000 7fa7789fd740 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c1f0ed2960 0x55c1f0ed4d80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.173+0000 7fa7789fd740 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1f0e4a900 0x55c1f0e55000 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.173+0000 7fa7789fd740 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1f0e45790 0x55c1f0e53ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.173+0000 7fa7789fd740 1 --2- 172.21.15.97:0/294244672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1f0e44da0 0x55c1f0e53ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.173+0000 7fa7789fd740 1 -- 172.21.15.97:0/294244672 >> 172.21.15.97:0/294244672 conn(0x55c1f0e40310 msgr2=0x55c1f0e45fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.174+0000 7fa75cc00640 0 client.25967 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.174+0000 7fa7789fd740 1 -- 172.21.15.97:0/294244672 shutdown_connections 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.174+0000 7fa7789fd740 1 -- 172.21.15.97:0/294244672 wait complete. 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.174+0000 7fa7789fd740 20 client.25967 trim_cache size 0 max 16384 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:08:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:08:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 -- 172.21.15.97:0/3057310786 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa7540a2fc0 msgr2=0x7fa7540a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa7540a2fc0 0x7fa7540a5480 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa768006010 tx=0x7fa768032590 comp rx=0 tx=0).stop 2024-03-26T19:08:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 -- 172.21.15.97:0/3057310786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7701001a0 msgr2=0x7fa77019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7701001a0 0x7fa77019e0e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa76000d770 tx=0x7fa76000dc30 comp rx=0 tx=0).stop 2024-03-26T19:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 -- 172.21.15.97:0/3057310786 shutdown_connections 2024-03-26T19:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa7540a2fc0 0x7fa7540a5480 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7701054c0 0x7fa7701a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa770102b10 0x7fa77019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 --2- 172.21.15.97:0/3057310786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7701001a0 0x7fa77019e0e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 -- 172.21.15.97:0/3057310786 >> 172.21.15.97:0/3057310786 conn(0x7fa7700fbda0 msgr2=0x7fa770100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 -- 172.21.15.97:0/3057310786 shutdown_connections 2024-03-26T19:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:16.175+0000 7fa776000640 1 -- 172.21.15.97:0/3057310786 wait complete. 2024-03-26T19:08:16.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:16 smithi097 ceph-mon[31955]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.50k op/s 2024-03-26T19:08:16.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:16 smithi144 ceph-mon[36053]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.50k op/s 2024-03-26T19:08:16.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:16 smithi105 ceph-mon[36270]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.50k op/s 2024-03-26T19:08:18.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:18 smithi097 ceph-mon[31955]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 1.56k op/s 2024-03-26T19:08:18.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:18 smithi144 ceph-mon[36053]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 1.56k op/s 2024-03-26T19:08:18.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:18 smithi105 ceph-mon[36270]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 1.56k op/s 2024-03-26T19:08:19.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:19 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:08:19.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:19 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:08:19.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:19 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:08:20.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 535.0 s, 1546.8 tps, lat 64.793 ms stddev 76.857 2024-03-26T19:08:20.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:20 smithi097 ceph-mon[31955]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-03-26T19:08:20.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:20 smithi144 ceph-mon[36053]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-03-26T19:08:20.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:20 smithi105 ceph-mon[36270]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-03-26T19:08:21.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:21 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:08:21.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:21 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:08:21.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:21 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:08:21.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:21 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:08:21.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:21 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:08:21.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:21 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:08:21.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:21 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:08:21.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:21 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:08:21.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:21 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:08:22.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:22 smithi097 ceph-mon[31955]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 34 MiB/s wr, 1.80k op/s 2024-03-26T19:08:22.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:22 smithi144 ceph-mon[36053]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 34 MiB/s wr, 1.80k op/s 2024-03-26T19:08:22.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:22 smithi105 ceph-mon[36270]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 34 MiB/s wr, 1.80k op/s 2024-03-26T19:08:24.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:24 smithi097 ceph-mon[31955]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 32 MiB/s wr, 1.67k op/s 2024-03-26T19:08:24.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:24 smithi144 ceph-mon[36053]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 32 MiB/s wr, 1.67k op/s 2024-03-26T19:08:24.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:24 smithi105 ceph-mon[36270]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 32 MiB/s wr, 1.67k op/s 2024-03-26T19:08:25.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 540.0 s, 1417.8 tps, lat 70.789 ms stddev 81.574 2024-03-26T19:08:26.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:26 smithi097 ceph-mon[31955]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 29 MiB/s wr, 1.58k op/s 2024-03-26T19:08:26.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:26 smithi144 ceph-mon[36053]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 29 MiB/s wr, 1.58k op/s 2024-03-26T19:08:26.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:26 smithi105 ceph-mon[36270]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 29 MiB/s wr, 1.58k op/s 2024-03-26T19:08:28.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:28 smithi097 ceph-mon[31955]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 14 MiB/s wr, 863 op/s 2024-03-26T19:08:28.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:28 smithi144 ceph-mon[36053]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 14 MiB/s wr, 863 op/s 2024-03-26T19:08:28.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:28 smithi105 ceph-mon[36270]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 14 MiB/s wr, 863 op/s 2024-03-26T19:08:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.364+0000 7f809b200640 1 -- 172.21.15.97:0/2440548143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8094074320 msgr2=0x7f80940747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.364+0000 7f809b200640 1 --2- 172.21.15.97:0/2440548143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8094074320 0x7f80940747a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8080009900 tx=0x7f808002bc40 comp rx=0 tx=0).stop 2024-03-26T19:08:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.364+0000 7f809b200640 1 -- 172.21.15.97:0/2440548143 shutdown_connections 2024-03-26T19:08:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.364+0000 7f809b200640 1 --2- 172.21.15.97:0/2440548143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809410dec0 0x7f80941102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.364+0000 7f809b200640 1 --2- 172.21.15.97:0/2440548143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8094074320 0x7f80940747a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.364+0000 7f809b200640 1 --2- 172.21.15.97:0/2440548143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8094075900 0x7f8094073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.364+0000 7f809b200640 1 -- 172.21.15.97:0/2440548143 >> 172.21.15.97:0/2440548143 conn(0x7f80940fce80 msgr2=0x7f80940ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.365+0000 7f809b200640 1 -- 172.21.15.97:0/2440548143 shutdown_connections 2024-03-26T19:08:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.365+0000 7f809b200640 1 -- 172.21.15.97:0/2440548143 wait complete. 2024-03-26T19:08:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.366+0000 7f809b200640 1 Processor -- start 2024-03-26T19:08:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.366+0000 7f809b200640 1 -- start start 2024-03-26T19:08:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.367+0000 7f809b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8094074320 0x7f809419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.367+0000 7f809b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8094075900 0x7f809419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.367+0000 7f809b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f809410dec0 0x7f80941a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.367+0000 7f809b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8094113c10 con 0x7f8094075900 2024-03-26T19:08:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.367+0000 7f809b200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8094113a90 con 0x7f8094074320 2024-03-26T19:08:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.367+0000 7f809b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8094113d90 con 0x7f809410dec0 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.367+0000 7f8093e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8094074320 0x7f809419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.367+0000 7f8093e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8094074320 0x7f809419e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60856/0 (socket says 172.21.15.97:60856) 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.367+0000 7f8093e00640 1 -- 172.21.15.97:0/3510637067 learned_addr learned my addr 172.21.15.97:0/3510637067 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.368+0000 7f8099200640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f809410dec0 0x7f80941a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.368+0000 7f8093400640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8094075900 0x7f809419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.368+0000 7f8093e00640 1 -- 172.21.15.97:0/3510637067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f809410dec0 msgr2=0x7f80941a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.368+0000 7f8093e00640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f809410dec0 0x7f80941a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.368+0000 7f8093e00640 1 -- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8094075900 msgr2=0x7f809419e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.368+0000 7f8093e00640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8094075900 0x7f809419e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.368+0000 7f8093e00640 1 -- 172.21.15.97:0/3510637067 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80941a5dc0 con 0x7f8094074320 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.368+0000 7f8093400640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8094075900 0x7f809419e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:08:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.368+0000 7f8093e00640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8094074320 0x7f809419e1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f808400d950 tx=0x7f808400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.368+0000 7f8090c00640 1 -- 172.21.15.97:0/3510637067 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084017030 con 0x7f8094074320 2024-03-26T19:08:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.369+0000 7f809b200640 1 -- 172.21.15.97:0/3510637067 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80941a60b0 con 0x7f8094074320 2024-03-26T19:08:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.369+0000 7f8090c00640 1 -- 172.21.15.97:0/3510637067 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8084004020 con 0x7f8094074320 2024-03-26T19:08:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.369+0000 7f8090c00640 1 -- 172.21.15.97:0/3510637067 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80840047a0 con 0x7f8094074320 2024-03-26T19:08:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.369+0000 7f809b200640 1 -- 172.21.15.97:0/3510637067 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80941a6600 con 0x7f8094074320 2024-03-26T19:08:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.373+0000 7f8090c00640 1 -- 172.21.15.97:0/3510637067 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f80840041d0 con 0x7f8094074320 2024-03-26T19:08:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.378+0000 7f809de0f740 1 Processor -- start 2024-03-26T19:08:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.378+0000 7f8090c00640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80780a3040 0x7f80780a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f809de0f740 1 -- start start 2024-03-26T19:08:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f8090c00640 1 -- 172.21.15.97:0/3510637067 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f80840cef50 con 0x7f8094074320 2024-03-26T19:08:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f8099200640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80780a3040 0x7f80780a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f809de0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf1d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f809de0f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf22290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f809de0f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 0x564ebcf24d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f809de0f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ebce81e80 con 0x564ebcf22900 2024-03-26T19:08:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f809de0f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564ebce2a660 con 0x564ebcf1d790 2024-03-26T19:08:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f809de0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564ebce7de30 con 0x564ebcf1cda0 2024-03-26T19:08:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f8093e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf1d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f8093400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf22290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f8093e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf1d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:45552/0 (socket says 172.21.15.97:45552) 2024-03-26T19:08:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f8093400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf22290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60860/0 (socket says 172.21.15.97:60860) 2024-03-26T19:08:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.379+0000 7f8093e00640 1 -- 172.21.15.97:0/4086523450 learned_addr learned my addr 172.21.15.97:0/4086523450 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.380+0000 7f8099200640 1 --2- 172.21.15.97:0/4086523450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 0x564ebcf24d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.380+0000 7f8093e00640 1 -- 172.21.15.97:0/4086523450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 msgr2=0x564ebcf22290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.380+0000 7f8093e00640 1 --2- 172.21.15.97:0/4086523450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf22290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.380+0000 7f8093e00640 1 -- 172.21.15.97:0/4086523450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 msgr2=0x564ebcf24d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.380+0000 7f8093e00640 1 --2- 172.21.15.97:0/4086523450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 0x564ebcf24d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.380+0000 7f8093e00640 1 -- 172.21.15.97:0/4086523450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ebcb96210 con 0x564ebcf1cda0 2024-03-26T19:08:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.381+0000 7f8093400640 1 --2- 172.21.15.97:0/4086523450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf22290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:08:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.381+0000 7f8099200640 1 --2- 172.21.15.97:0/4086523450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 0x564ebcf24d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:08:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.381+0000 7f8099200640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80780a3040 0x7f80780a5500 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f808c009ef0 tx=0x7f808c00c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.382+0000 7f8093e00640 1 --2- 172.21.15.97:0/4086523450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf1d1c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8084098020 tx=0x7f808409afb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9ab4a3d92ae2ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.382+0000 7f8089600640 1 -- 172.21.15.97:0/4086523450 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808409f7a0 con 0x564ebcf1cda0 2024-03-26T19:08:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.383+0000 7f8089600640 1 -- 172.21.15.97:0/4086523450 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80840a4060 con 0x564ebcf1cda0 2024-03-26T19:08:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.383+0000 7f8089600640 1 -- 172.21.15.97:0/4086523450 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808409bb10 con 0x564ebcf1cda0 2024-03-26T19:08:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.384+0000 7f809de0f740 1 -- 172.21.15.97:0/4086523450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 msgr2=0x564ebcf1d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.384+0000 7f809de0f740 1 --2- 172.21.15.97:0/4086523450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf1d1c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8084098020 tx=0x7f808409afb0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.384+0000 7f809de0f740 1 -- 172.21.15.97:0/4086523450 shutdown_connections 2024-03-26T19:08:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.384+0000 7f809de0f740 1 --2- 172.21.15.97:0/4086523450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 0x564ebcf24d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.384+0000 7f809de0f740 1 --2- 172.21.15.97:0/4086523450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf22290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.384+0000 7f809de0f740 1 --2- 172.21.15.97:0/4086523450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf1d1c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.384+0000 7f809de0f740 1 -- 172.21.15.97:0/4086523450 >> 172.21.15.97:0/4086523450 conn(0x564ebcf18310 msgr2=0x564ebcf18710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.384+0000 7f809de0f740 1 -- 172.21.15.97:0/4086523450 shutdown_connections 2024-03-26T19:08:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.384+0000 7f809de0f740 1 -- 172.21.15.97:0/4086523450 wait complete. 2024-03-26T19:08:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.384+0000 7f809de0f740 1 -- start start 2024-03-26T19:08:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f809de0f740 1 Processor -- start 2024-03-26T19:08:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f809de0f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf1cda0 0x564ebcf26d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f809de0f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f809de0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf22900 0x564ebcf2e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f809de0f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ebcf258a0 con 0x564ebcf1cda0 2024-03-26T19:08:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f809de0f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564ebce3dac0 con 0x564ebcf1d790 2024-03-26T19:08:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f809de0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564ebcf25a20 con 0x564ebcf22900 2024-03-26T19:08:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f809de0f740 10 client.0 _ll_register_callbacks cb 0x564ebce3d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:08:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f8093400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f8093e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf1cda0 0x564ebcf26d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f8093400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf272a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60874/0 (socket says 172.21.15.97:60874) 2024-03-26T19:08:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.385+0000 7f8093400640 1 -- 172.21.15.97:0/1633652937 learned_addr learned my addr 172.21.15.97:0/1633652937 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.386+0000 7f8099200640 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf22900 0x564ebcf2e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.386+0000 7f8093e00640 1 -- 172.21.15.97:0/1633652937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf22900 msgr2=0x564ebcf2e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.386+0000 7f8093e00640 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf22900 0x564ebcf2e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.386+0000 7f8093e00640 1 -- 172.21.15.97:0/1633652937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 msgr2=0x564ebcf272a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.386+0000 7f8093e00640 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf272a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.386+0000 7f8093e00640 1 -- 172.21.15.97:0/1633652937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ebcc62620 con 0x564ebcf1cda0 2024-03-26T19:08:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.386+0000 7f8099200640 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf22900 0x564ebcf2e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:08:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.386+0000 7f8093400640 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf272a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:08:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.386+0000 7f8093e00640 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf1cda0 0x564ebcf26d60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8084000c00 tx=0x7f808409b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a756bd11c37505 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.386+0000 7f807e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T19:08:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.388+0000 7f807e000640 1 -- 172.21.15.97:0/1633652937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808409f7a0 con 0x564ebcf1cda0 2024-03-26T19:08:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.389+0000 7f809de0f740 1 -- 172.21.15.97:0/1633652937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80840a4060 con 0x564ebcf1cda0 2024-03-26T19:08:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.389+0000 7f807e000640 1 -- 172.21.15.97:0/1633652937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f808409bac0 con 0x564ebcf1cda0 2024-03-26T19:08:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.390+0000 7f807e000640 1 -- 172.21.15.97:0/1633652937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80840a3bd0 con 0x564ebcf1cda0 2024-03-26T19:08:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.390+0000 7f809de0f740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-03-26T19:08:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.390+0000 7f809de0f740 1 -- 172.21.15.97:0/1633652937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ebcf31650 con 0x564ebcf1cda0 2024-03-26T19:08:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.392+0000 7f807e000640 1 -- 172.21.15.97:0/1633652937 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80840a4060 con 0x564ebcf1cda0 2024-03-26T19:08:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.392+0000 7f807e000640 1 -- 172.21.15.97:0/1633652937 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f80840a0930 con 0x564ebcf1cda0 2024-03-26T19:08:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.392+0000 7f809de0f740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:08:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.392+0000 7f809de0f740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:08:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.392+0000 7f809de0f740 20 client.15633 populate_metadata read hostname 'smithi097' 2024-03-26T19:08:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.392+0000 7f809de0f740 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564ebcf329a0 0x564ebcf34d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.392+0000 7f809de0f740 4 client.15633 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:08:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.392+0000 7f809de0f740 1 -- 172.21.15.97:0/1633652937 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ebce8f3c0 con 0x564ebcf329a0 2024-03-26T19:08:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.392+0000 7f8093400640 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564ebcf329a0 0x564ebcf34d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.393+0000 7f8093400640 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564ebcf329a0 0x564ebcf34d90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eabfdfc50afc40e0 server_cookie=f704685428ad3571 in_seq=0 out_seq=0 2024-03-26T19:08:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.393+0000 7f807e000640 10 client.15633 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.394+0000 7f807e000640 1 -- 172.21.15.97:0/1633652937 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564ebce8f3c0 con 0x564ebcf329a0 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.394+0000 7f807e000640 10 client.15633 handle_command_reply: tid=0 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.394+0000 7f809de0f740 1 client.15633 shutdown 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 1 -- 172.21.15.97:0/1633652937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf1cda0 msgr2=0x564ebcf26d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf1cda0 0x564ebcf26d60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8084000c00 tx=0x7f808409b710 comp rx=0 tx=0).stop 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 1 -- 172.21.15.97:0/1633652937 shutdown_connections 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564ebcf329a0 0x564ebcf34d90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf22900 0x564ebcf2e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1d790 0x564ebcf272a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 1 --2- 172.21.15.97:0/1633652937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf1cda0 0x564ebcf26d60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f807e000640 0 client.15633 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 1 -- 172.21.15.97:0/1633652937 >> 172.21.15.97:0/1633652937 conn(0x564ebcf18310 msgr2=0x564ebcf17540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 1 -- 172.21.15.97:0/1633652937 shutdown_connections 2024-03-26T19:08:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 1 -- 172.21.15.97:0/1633652937 wait complete. 2024-03-26T19:08:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.395+0000 7f809de0f740 20 client.15633 trim_cache size 0 max 16384 2024-03-26T19:08:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.417+0000 7f809de0f740 1 Processor -- start 2024-03-26T19:08:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.417+0000 7f809de0f740 1 -- start start 2024-03-26T19:08:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.417+0000 7f809de0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf27200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f809de0f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf1d790 0x564ebcf2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f809de0f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf22900 0x564ebcf1bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f809de0f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f80941af390 con 0x564ebcf1d790 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f809de0f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80941b0ed0 con 0x564ebcf22900 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f809de0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80941afa60 con 0x564ebcf1cda0 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f8093e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf27200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f8099200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf22900 0x564ebcf1bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f8093e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf27200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:45572/0 (socket says 172.21.15.97:45572) 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f8093e00640 1 -- 172.21.15.97:0/2659170615 learned_addr learned my addr 172.21.15.97:0/2659170615 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f8093e00640 1 -- 172.21.15.97:0/2659170615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf22900 msgr2=0x564ebcf1bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f8093e00640 1 --2- 172.21.15.97:0/2659170615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf22900 0x564ebcf1bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f8093e00640 1 -- 172.21.15.97:0/2659170615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf1d790 msgr2=0x564ebcf2e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f8093e00640 1 --2- 172.21.15.97:0/2659170615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf1d790 0x564ebcf2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.418+0000 7f8093e00640 1 -- 172.21.15.97:0/2659170615 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ebcf1c9e0 con 0x564ebcf1cda0 2024-03-26T19:08:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.419+0000 7f8093e00640 1 --2- 172.21.15.97:0/2659170615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf27200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8084097ab0 tx=0x7f808409afb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9241a09d5c2b6798 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.419+0000 7f8088c00640 1 -- 172.21.15.97:0/2659170615 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80840978f0 con 0x564ebcf1cda0 2024-03-26T19:08:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.419+0000 7f8088c00640 1 -- 172.21.15.97:0/2659170615 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8084065070 con 0x564ebcf1cda0 2024-03-26T19:08:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.419+0000 7f8088c00640 1 -- 172.21.15.97:0/2659170615 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80840a0930 con 0x564ebcf1cda0 2024-03-26T19:08:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.420+0000 7f809de0f740 1 -- 172.21.15.97:0/2659170615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 msgr2=0x564ebcf27200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.420+0000 7f809de0f740 1 --2- 172.21.15.97:0/2659170615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf27200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8084097ab0 tx=0x7f808409afb0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.421+0000 7f809de0f740 1 -- 172.21.15.97:0/2659170615 shutdown_connections 2024-03-26T19:08:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.421+0000 7f809de0f740 1 --2- 172.21.15.97:0/2659170615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf22900 0x564ebcf1bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.421+0000 7f809de0f740 1 --2- 172.21.15.97:0/2659170615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf1d790 0x564ebcf2e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.421+0000 7f809de0f740 1 --2- 172.21.15.97:0/2659170615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1cda0 0x564ebcf27200 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.421+0000 7f809de0f740 1 -- 172.21.15.97:0/2659170615 >> 172.21.15.97:0/2659170615 conn(0x564ebcf18310 msgr2=0x564ebcf1b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.421+0000 7f809de0f740 1 -- 172.21.15.97:0/2659170615 shutdown_connections 2024-03-26T19:08:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.422+0000 7f809de0f740 1 -- 172.21.15.97:0/2659170615 wait complete. 2024-03-26T19:08:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.422+0000 7f809de0f740 1 -- start start 2024-03-26T19:08:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.422+0000 7f809de0f740 1 Processor -- start 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.424+0000 7f809de0f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1cda0 0x564ebcf2baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.424+0000 7f809de0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1d790 0x564ebcf2c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.424+0000 7f809de0f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 0x564ebcf2cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.424+0000 7f809de0f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f80941b0800 con 0x564ebcf22900 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.424+0000 7f809de0f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80941b0130 con 0x564ebcf1cda0 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.424+0000 7f809de0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80941ad370 con 0x564ebcf1d790 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.424+0000 7f809de0f740 10 client.0 _ll_register_callbacks cb 0x7f808409b4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.424+0000 7f8093e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1cda0 0x564ebcf2baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.424+0000 7f8093e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1cda0 0x564ebcf2baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60892/0 (socket says 172.21.15.97:60892) 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.424+0000 7f8093e00640 1 -- 172.21.15.97:0/1637641909 learned_addr learned my addr 172.21.15.97:0/1637641909 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f8099200640 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 0x564ebcf2cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f8093400640 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1d790 0x564ebcf2c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f8093e00640 1 -- 172.21.15.97:0/1637641909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1d790 msgr2=0x564ebcf2c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f8093e00640 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1d790 0x564ebcf2c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f8093e00640 1 -- 172.21.15.97:0/1637641909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 msgr2=0x564ebcf2cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f8093e00640 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 0x564ebcf2cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f8093e00640 1 -- 172.21.15.97:0/1637641909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ebcf13730 con 0x564ebcf1cda0 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f8093e00640 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1cda0 0x564ebcf2baf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8084005fd0 tx=0x7f80840a0b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1840c31e26ce66 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f807e000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f807e000640 1 -- 172.21.15.97:0/1637641909 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80840978f0 con 0x564ebcf1cda0 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f807e000640 1 -- 172.21.15.97:0/1637641909 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f808409a720 con 0x564ebcf1cda0 2024-03-26T19:08:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.425+0000 7f809de0f740 1 -- 172.21.15.97:0/1637641909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8084065070 con 0x564ebcf1cda0 2024-03-26T19:08:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f807e000640 1 -- 172.21.15.97:0/1637641909 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80840af050 con 0x564ebcf1cda0 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f809de0f740 10 client.25985 fetch_fsmap learned FSMap version 27 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f809de0f740 1 -- 172.21.15.97:0/1637641909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ebcf13730 con 0x564ebcf1cda0 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f807e000640 1 -- 172.21.15.97:0/1637641909 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8084065070 con 0x564ebcf1cda0 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f807e000640 1 -- 172.21.15.97:0/1637641909 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f808409ac40 con 0x564ebcf1cda0 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f809de0f740 10 client.25985 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f809de0f740 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f809de0f740 20 client.25985 populate_metadata read hostname 'smithi097' 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f809de0f740 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564ebcf98f20 0x564ebcf9b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f809de0f740 4 client.25985 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f809de0f740 1 -- 172.21.15.97:0/1637641909 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ebce89710 con 0x564ebcf98f20 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.426+0000 7f8093400640 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564ebcf98f20 0x564ebcf9b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.427+0000 7f807e000640 10 client.25985 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.428+0000 7f8093400640 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564ebcf98f20 0x564ebcf9b310 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=422303cf09135146 server_cookie=2bd4b06b81c2b1d5 in_seq=0 out_seq=0 2024-03-26T19:08:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.497+0000 7f807e000640 1 -- 172.21.15.97:0/1637641909 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ebce89710 con 0x564ebcf98f20 2024-03-26T19:08:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.497+0000 7f807e000640 10 client.25985 handle_command_reply: tid=0 2024-03-26T19:08:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 client.25985 shutdown 2024-03-26T19:08:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 -- 172.21.15.97:0/1637641909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1cda0 msgr2=0x564ebcf2baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1cda0 0x564ebcf2baf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8084005fd0 tx=0x7f80840a0b60 comp rx=0 tx=0).stop 2024-03-26T19:08:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 -- 172.21.15.97:0/1637641909 shutdown_connections 2024-03-26T19:08:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x564ebcf98f20 0x564ebcf9b310 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ebcf22900 0x564ebcf2cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ebcf1d790 0x564ebcf2c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 --2- 172.21.15.97:0/1637641909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ebcf1cda0 0x564ebcf2baf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 -- 172.21.15.97:0/1637641909 >> 172.21.15.97:0/1637641909 conn(0x564ebcf18310 msgr2=0x564ebcf1be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f807e000640 0 client.25985 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 -- 172.21.15.97:0/1637641909 shutdown_connections 2024-03-26T19:08:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.498+0000 7f809de0f740 1 -- 172.21.15.97:0/1637641909 wait complete. 2024-03-26T19:08:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.499+0000 7f809de0f740 20 client.25985 trim_cache size 0 max 16384 2024-03-26T19:08:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:08:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:08:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:08:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:08:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.500+0000 7f809b200640 1 -- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80780a3040 msgr2=0x7f80780a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.500+0000 7f809b200640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80780a3040 0x7f80780a5500 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f808c009ef0 tx=0x7f808c00c7f0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.500+0000 7f809b200640 1 -- 172.21.15.97:0/3510637067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8094074320 msgr2=0x7f809419e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.500+0000 7f809b200640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8094074320 0x7f809419e1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f808400d950 tx=0x7f808400de10 comp rx=0 tx=0).stop 2024-03-26T19:08:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.501+0000 7f809b200640 1 -- 172.21.15.97:0/3510637067 shutdown_connections 2024-03-26T19:08:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.501+0000 7f809b200640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f80780a3040 0x7f80780a5500 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.501+0000 7f809b200640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f809410dec0 0x7f80941a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.501+0000 7f809b200640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8094075900 0x7f809419e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.501+0000 7f809b200640 1 --2- 172.21.15.97:0/3510637067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8094074320 0x7f809419e1a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.501+0000 7f809b200640 1 -- 172.21.15.97:0/3510637067 >> 172.21.15.97:0/3510637067 conn(0x7f80940fce80 msgr2=0x7f8094101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.501+0000 7f809b200640 1 -- 172.21.15.97:0/3510637067 shutdown_connections 2024-03-26T19:08:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:29.501+0000 7f809b200640 1 -- 172.21.15.97:0/3510637067 wait complete. 2024-03-26T19:08:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 545.0 s, 1368.6 tps, lat 73.198 ms stddev 87.319 2024-03-26T19:08:30.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:30 smithi097 ceph-mon[31955]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 11 MiB/s wr, 767 op/s 2024-03-26T19:08:30.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:30 smithi144 ceph-mon[36053]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 11 MiB/s wr, 767 op/s 2024-03-26T19:08:30.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:30 smithi105 ceph-mon[36270]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 11 MiB/s wr, 767 op/s 2024-03-26T19:08:32.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:32 smithi097 ceph-mon[31955]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-03-26T19:08:32.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:32 smithi144 ceph-mon[36053]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-03-26T19:08:32.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:32 smithi105 ceph-mon[36270]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-03-26T19:08:34.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:34 smithi097 ceph-mon[31955]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 MiB/s wr, 664 op/s 2024-03-26T19:08:34.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:34 smithi144 ceph-mon[36053]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 MiB/s wr, 664 op/s 2024-03-26T19:08:34.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:34 smithi105 ceph-mon[36270]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 MiB/s wr, 664 op/s 2024-03-26T19:08:35.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 550.0 s, 1509.5 tps, lat 65.628 ms stddev 79.299 2024-03-26T19:08:36.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:36 smithi144 ceph-mon[36053]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 730 op/s 2024-03-26T19:08:36.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:36 smithi105 ceph-mon[36270]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 730 op/s 2024-03-26T19:08:36.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:36 smithi097 ceph-mon[31955]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 730 op/s 2024-03-26T19:08:38.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:38 smithi144 ceph-mon[36053]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.1 MiB/s wr, 612 op/s 2024-03-26T19:08:38.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:38 smithi105 ceph-mon[36270]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.1 MiB/s wr, 612 op/s 2024-03-26T19:08:38.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:38 smithi097 ceph-mon[31955]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.1 MiB/s wr, 612 op/s 2024-03-26T19:08:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 555.0 s, 1805.4 tps, lat 55.539 ms stddev 71.274 2024-03-26T19:08:40.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:40 smithi144 ceph-mon[36053]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 679 op/s 2024-03-26T19:08:40.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:40 smithi105 ceph-mon[36270]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 679 op/s 2024-03-26T19:08:40.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:40 smithi097 ceph-mon[31955]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 679 op/s 2024-03-26T19:08:42.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:42 smithi144 ceph-mon[36053]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 772 op/s 2024-03-26T19:08:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.694+0000 7f1920c00640 1 -- 172.21.15.97:0/3837939215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f191c10a390 msgr2=0x7f191c110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.694+0000 7f1920c00640 1 --2- 172.21.15.97:0/3837939215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f191c10a390 0x7f191c110c20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1914009be0 tx=0x7f191402bef0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.695+0000 7f1920c00640 1 -- 172.21.15.97:0/3837939215 shutdown_connections 2024-03-26T19:08:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.695+0000 7f1920c00640 1 --2- 172.21.15.97:0/3837939215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f191c10a390 0x7f191c110c20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.695+0000 7f1920c00640 1 --2- 172.21.15.97:0/3837939215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f191c1099d0 0x7f191c109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.695+0000 7f1920c00640 1 --2- 172.21.15.97:0/3837939215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c108c60 0x7f191c109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.695+0000 7f1920c00640 1 -- 172.21.15.97:0/3837939215 >> 172.21.15.97:0/3837939215 conn(0x7f191c06ed20 msgr2=0x7f191c071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.695+0000 7f1920c00640 1 -- 172.21.15.97:0/3837939215 shutdown_connections 2024-03-26T19:08:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.695+0000 7f1920c00640 1 -- 172.21.15.97:0/3837939215 wait complete. 2024-03-26T19:08:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.697+0000 7f1920c00640 1 Processor -- start 2024-03-26T19:08:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.698+0000 7f1920c00640 1 -- start start 2024-03-26T19:08:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.698+0000 7f1920c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f191c108c60 0x7f191c1a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.698+0000 7f1920c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c1099d0 0x7f191c1a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.699+0000 7f1920c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f191c10a390 0x7f191c1ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.699+0000 7f1920c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f191c118040 con 0x7f191c1099d0 2024-03-26T19:08:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.699+0000 7f1920c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f191c117ec0 con 0x7f191c10a390 2024-03-26T19:08:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.699+0000 7f1920c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f191c1181c0 con 0x7f191c108c60 2024-03-26T19:08:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.699+0000 7f191a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f191c10a390 0x7f191c1ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.699+0000 7f1919e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f191c108c60 0x7f191c1a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.699+0000 7f191a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f191c10a390 0x7f191c1ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:55090/0 (socket says 172.21.15.97:55090) 2024-03-26T19:08:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.699+0000 7f191a800640 1 -- 172.21.15.97:0/198871198 learned_addr learned my addr 172.21.15.97:0/198871198 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.699+0000 7f1919400640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c1099d0 0x7f191c1a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.700+0000 7f191a800640 1 -- 172.21.15.97:0/198871198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f191c108c60 msgr2=0x7f191c1a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.700+0000 7f191a800640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f191c108c60 0x7f191c1a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.700+0000 7f191a800640 1 -- 172.21.15.97:0/198871198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c1099d0 msgr2=0x7f191c1a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.700+0000 7f191a800640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c1099d0 0x7f191c1a6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.702+0000 7f191a800640 1 -- 172.21.15.97:0/198871198 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c1ae4e0 con 0x7f191c10a390 2024-03-26T19:08:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.706+0000 7f191a800640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f191c10a390 0x7f191c1ade70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1914036040 tx=0x7f19140337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.708+0000 7f1912a00640 1 -- 172.21.15.97:0/198871198 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914043020 con 0x7f191c10a390 2024-03-26T19:08:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.708+0000 7f1912a00640 1 -- 172.21.15.97:0/198871198 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f191402ca60 con 0x7f191c10a390 2024-03-26T19:08:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.708+0000 7f1912a00640 1 -- 172.21.15.97:0/198871198 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914034690 con 0x7f191c10a390 2024-03-26T19:08:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.708+0000 7f1920c00640 1 -- 172.21.15.97:0/198871198 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191c1ae770 con 0x7f191c10a390 2024-03-26T19:08:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.708+0000 7f1920c00640 1 -- 172.21.15.97:0/198871198 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191c1aecc0 con 0x7f191c10a390 2024-03-26T19:08:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.714+0000 7f19237f5740 1 Processor -- start 2024-03-26T19:08:42.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:42 smithi105 ceph-mon[36270]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 772 op/s 2024-03-26T19:08:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.717+0000 7f1912a00640 1 -- 172.21.15.97:0/198871198 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f1914046020 con 0x7f191c10a390 2024-03-26T19:08:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.719+0000 7f19237f5740 1 -- start start 2024-03-26T19:08:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.719+0000 7f19237f5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42124fda0 0x55d4212501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.719+0000 7f19237f5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421250790 0x55d421255290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.719+0000 7f19237f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d421257d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.719+0000 7f19237f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d4211b4e80 con 0x55d421255900 2024-03-26T19:08:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.719+0000 7f19237f5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d42115d660 con 0x55d421250790 2024-03-26T19:08:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.719+0000 7f19237f5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4211b0e30 con 0x55d42124fda0 2024-03-26T19:08:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.720+0000 7f1919400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42124fda0 0x55d4212501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.720+0000 7f1919e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421250790 0x55d421255290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.720+0000 7f1919e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421250790 0x55d421255290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:55092/0 (socket says 172.21.15.97:55092) 2024-03-26T19:08:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.720+0000 7f1919400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42124fda0 0x55d4212501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:42302/0 (socket says 172.21.15.97:42302) 2024-03-26T19:08:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.720+0000 7f1919e00640 1 -- 172.21.15.97:0/2850867138 learned_addr learned my addr 172.21.15.97:0/2850867138 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.720+0000 7f1919400640 1 --2- 172.21.15.97:0/2850867138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d421257d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.721+0000 7f1919e00640 1 -- 172.21.15.97:0/2850867138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42124fda0 msgr2=0x55d4212501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.721+0000 7f1919e00640 1 --2- 172.21.15.97:0/2850867138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42124fda0 0x55d4212501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.721+0000 7f1919e00640 1 -- 172.21.15.97:0/2850867138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 msgr2=0x55d421257d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.721+0000 7f1919e00640 1 --2- 172.21.15.97:0/2850867138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d421257d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.721+0000 7f1919e00640 1 -- 172.21.15.97:0/2850867138 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d420ec9210 con 0x55d421250790 2024-03-26T19:08:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.722+0000 7f1912a00640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19000a3110 0x7f19000a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.722+0000 7f1912a00640 1 -- 172.21.15.97:0/198871198 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f19140f1b20 con 0x7f191c10a390 2024-03-26T19:08:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.723+0000 7f1919e00640 1 --2- 172.21.15.97:0/2850867138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421250790 0x55d421255290 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f190c009c20 tx=0x7f190c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503d770225abc57b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.723+0000 7f1907000640 1 -- 172.21.15.97:0/2850867138 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c012450 con 0x55d421250790 2024-03-26T19:08:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.724+0000 7f1907000640 1 -- 172.21.15.97:0/2850867138 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f190c012ad0 con 0x55d421250790 2024-03-26T19:08:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.724+0000 7f1907000640 1 -- 172.21.15.97:0/2850867138 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c00fb00 con 0x55d421250790 2024-03-26T19:08:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.725+0000 7f19237f5740 1 -- 172.21.15.97:0/2850867138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421250790 msgr2=0x55d421255290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.725+0000 7f19237f5740 1 --2- 172.21.15.97:0/2850867138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421250790 0x55d421255290 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f190c009c20 tx=0x7f190c0117b0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.726+0000 7f1919400640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19000a3110 0x7f19000a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.727+0000 7f1919400640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19000a3110 0x7f19000a55d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55d421250ff0 tx=0x7f1908009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.728+0000 7f19237f5740 1 -- 172.21.15.97:0/2850867138 shutdown_connections 2024-03-26T19:08:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.728+0000 7f19237f5740 1 --2- 172.21.15.97:0/2850867138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d421257d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.728+0000 7f19237f5740 1 --2- 172.21.15.97:0/2850867138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421250790 0x55d421255290 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.728+0000 7f19237f5740 1 --2- 172.21.15.97:0/2850867138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42124fda0 0x55d4212501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.728+0000 7f19237f5740 1 -- 172.21.15.97:0/2850867138 >> 172.21.15.97:0/2850867138 conn(0x55d42124b310 msgr2=0x55d42124b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.729+0000 7f19237f5740 1 -- 172.21.15.97:0/2850867138 shutdown_connections 2024-03-26T19:08:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.730+0000 7f19237f5740 1 -- 172.21.15.97:0/2850867138 wait complete. 2024-03-26T19:08:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.730+0000 7f19237f5740 1 -- start start 2024-03-26T19:08:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.731+0000 7f19237f5740 1 Processor -- start 2024-03-26T19:08:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.731+0000 7f19237f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d42124fda0 0x55d421259e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.731+0000 7f19237f5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.731+0000 7f19237f5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421255900 0x55d421261340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.731+0000 7f19237f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d4212588b0 con 0x55d42124fda0 2024-03-26T19:08:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.731+0000 7f1919400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.731+0000 7f19237f5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d421170ac0 con 0x55d421255900 2024-03-26T19:08:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.732+0000 7f19237f5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d421258a30 con 0x55d421250790 2024-03-26T19:08:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.732+0000 7f1919400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:42308/0 (socket says 172.21.15.97:42308) 2024-03-26T19:08:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.732+0000 7f1919400640 1 -- 172.21.15.97:0/20706944 learned_addr learned my addr 172.21.15.97:0/20706944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.732+0000 7f19237f5740 10 client.0 _ll_register_callbacks cb 0x55d421170490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:08:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.732+0000 7f1919400640 1 -- 172.21.15.97:0/20706944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421255900 msgr2=0x55d421261340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.732+0000 7f1919400640 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421255900 0x55d421261340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.732+0000 7f1919400640 1 -- 172.21.15.97:0/20706944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d42124fda0 msgr2=0x55d421259e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.732+0000 7f1919400640 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d42124fda0 0x55d421259e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.732+0000 7f1919400640 1 -- 172.21.15.97:0/20706944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d420f95620 con 0x55d421250790 2024-03-26T19:08:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.734+0000 7f1919400640 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125a370 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1908010fd0 tx=0x7f1908004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42863c226b9412a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.734+0000 7f18fb400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:08:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.735+0000 7f18fb400640 1 -- 172.21.15.97:0/20706944 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190802e060 con 0x55d421250790 2024-03-26T19:08:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.735+0000 7f18fb400640 1 -- 172.21.15.97:0/20706944 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f190801b030 con 0x55d421250790 2024-03-26T19:08:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.735+0000 7f19237f5740 1 -- 172.21.15.97:0/20706944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f190c012ad0 con 0x55d421250790 2024-03-26T19:08:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.738+0000 7f18fb400640 1 -- 172.21.15.97:0/20706944 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1908029410 con 0x55d421250790 2024-03-26T19:08:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.738+0000 7f19237f5740 10 client.25960 fetch_fsmap learned FSMap version 27 2024-03-26T19:08:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.738+0000 7f19237f5740 1 -- 172.21.15.97:0/20706944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d421264720 con 0x55d421250790 2024-03-26T19:08:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.738+0000 7f18fb400640 1 -- 172.21.15.97:0/20706944 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f190c012ad0 con 0x55d421250790 2024-03-26T19:08:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.738+0000 7f18fb400640 1 -- 172.21.15.97:0/20706944 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f19080283b0 con 0x55d421250790 2024-03-26T19:08:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.739+0000 7f19237f5740 10 client.25960 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:08:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.739+0000 7f19237f5740 10 client.25960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:08:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.739+0000 7f19237f5740 20 client.25960 populate_metadata read hostname 'smithi097' 2024-03-26T19:08:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.739+0000 7f19237f5740 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d421265e80 0x55d421268310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.739+0000 7f19237f5740 4 client.25960 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:08:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.739+0000 7f19237f5740 1 -- 172.21.15.97:0/20706944 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4211c23c0 con 0x55d421265e80 2024-03-26T19:08:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.739+0000 7f1919e00640 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d421265e80 0x55d421268310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.740+0000 7f1919e00640 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d421265e80 0x55d421268310 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7abb00e2ab8430c1 server_cookie=7a3daa91641d8831 in_seq=0 out_seq=0 2024-03-26T19:08:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.740+0000 7f18fb400640 10 client.25960 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.741+0000 7f18fb400640 1 -- 172.21.15.97:0/20706944 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d4211c23c0 con 0x55d421265e80 2024-03-26T19:08:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.742+0000 7f18fb400640 10 client.25960 handle_command_reply: tid=0 2024-03-26T19:08:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.742+0000 7f19237f5740 1 client.25960 shutdown 2024-03-26T19:08:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.742+0000 7f19237f5740 1 -- 172.21.15.97:0/20706944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 msgr2=0x55d42125a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.742+0000 7f19237f5740 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125a370 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1908010fd0 tx=0x7f1908004730 comp rx=0 tx=0).stop 2024-03-26T19:08:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.746+0000 7f19237f5740 1 -- 172.21.15.97:0/20706944 shutdown_connections 2024-03-26T19:08:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.746+0000 7f19237f5740 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d421265e80 0x55d421268310 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.746+0000 7f19237f5740 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d421255900 0x55d421261340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.746+0000 7f19237f5740 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125a370 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.746+0000 7f19237f5740 1 --2- 172.21.15.97:0/20706944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d42124fda0 0x55d421259e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.746+0000 7f19237f5740 1 -- 172.21.15.97:0/20706944 >> 172.21.15.97:0/20706944 conn(0x55d42124b310 msgr2=0x55d42124a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.746+0000 7f18fb400640 0 client.25960 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.746+0000 7f19237f5740 1 -- 172.21.15.97:0/20706944 shutdown_connections 2024-03-26T19:08:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.746+0000 7f19237f5740 1 -- 172.21.15.97:0/20706944 wait complete. 2024-03-26T19:08:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.747+0000 7f19237f5740 20 client.25960 trim_cache size 0 max 16384 2024-03-26T19:08:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.776+0000 7f19237f5740 1 Processor -- start 2024-03-26T19:08:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.777+0000 7f19237f5740 1 -- start start 2024-03-26T19:08:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.778+0000 7f19237f5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 0x55d42124ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.778+0000 7f19237f5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d421261300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.778+0000 7f19237f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d421249cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.778+0000 7f19237f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f191c1b7a20 con 0x55d421255900 2024-03-26T19:08:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.778+0000 7f19237f5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f191c1b9560 con 0x55d42124fda0 2024-03-26T19:08:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.778+0000 7f19237f5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f191c1b80f0 con 0x55d421250790 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.778+0000 7f1919400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d421261300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.778+0000 7f1919400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d421261300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:42312/0 (socket says 172.21.15.97:42312) 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.778+0000 7f1919400640 1 -- 172.21.15.97:0/3396425491 learned_addr learned my addr 172.21.15.97:0/3396425491 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.779+0000 7f191a800640 1 --2- 172.21.15.97:0/3396425491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d421249cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.779+0000 7f1919e00640 1 --2- 172.21.15.97:0/3396425491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 0x55d42124ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.779+0000 7f1919400640 1 -- 172.21.15.97:0/3396425491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 msgr2=0x55d42124ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.779+0000 7f1919400640 1 --2- 172.21.15.97:0/3396425491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 0x55d42124ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.779+0000 7f1919400640 1 -- 172.21.15.97:0/3396425491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 msgr2=0x55d421249cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.779+0000 7f1919400640 1 --2- 172.21.15.97:0/3396425491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d421249cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.779+0000 7f1919400640 1 -- 172.21.15.97:0/3396425491 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42124c080 con 0x55d421250790 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.779+0000 7f191a800640 1 --2- 172.21.15.97:0/3396425491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d421249cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:08:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.779+0000 7f1919400640 1 --2- 172.21.15.97:0/3396425491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d421261300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1908009d80 tx=0x7f1908009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aaa297528693499 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.780+0000 7f1906600640 1 -- 172.21.15.97:0/3396425491 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190802e070 con 0x55d421250790 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.780+0000 7f1906600640 1 -- 172.21.15.97:0/3396425491 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19080283b0 con 0x55d421250790 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.780+0000 7f1906600640 1 -- 172.21.15.97:0/3396425491 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908038080 con 0x55d421250790 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.781+0000 7f19237f5740 1 -- 172.21.15.97:0/3396425491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 msgr2=0x55d421261300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.781+0000 7f19237f5740 1 --2- 172.21.15.97:0/3396425491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d421261300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1908009d80 tx=0x7f1908009ae0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.781+0000 7f19237f5740 1 -- 172.21.15.97:0/3396425491 shutdown_connections 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.781+0000 7f19237f5740 1 --2- 172.21.15.97:0/3396425491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d421249cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.781+0000 7f19237f5740 1 --2- 172.21.15.97:0/3396425491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d421261300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.781+0000 7f19237f5740 1 --2- 172.21.15.97:0/3396425491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 0x55d42124ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.781+0000 7f19237f5740 1 -- 172.21.15.97:0/3396425491 >> 172.21.15.97:0/3396425491 conn(0x55d42124b310 msgr2=0x55d42124e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.781+0000 7f19237f5740 1 -- 172.21.15.97:0/3396425491 shutdown_connections 2024-03-26T19:08:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.781+0000 7f19237f5740 1 -- 172.21.15.97:0/3396425491 wait complete. 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.781+0000 7f19237f5740 1 -- start start 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.782+0000 7f19237f5740 1 Processor -- start 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.782+0000 7f19237f5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 0x55d42125e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.782+0000 7f19237f5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.782+0000 7f19237f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d42125fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.782+0000 7f19237f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f191c1b8e90 con 0x55d421255900 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.782+0000 7f19237f5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f191c1b87c0 con 0x55d42124fda0 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.782+0000 7f19237f5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f191c1b5a00 con 0x55d421250790 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.782+0000 7f19237f5740 10 client.0 _ll_register_callbacks cb 0x7f1908012a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 0x55d42125e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:42316/0 (socket says 172.21.15.97:42316) 2024-03-26T19:08:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919400640 1 -- 172.21.15.97:0/368643408 learned_addr learned my addr 172.21.15.97:0/368643408 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 0x55d42125e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:55130/0 (socket says 172.21.15.97:55130) 2024-03-26T19:08:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f191a800640 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d42125fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919400640 1 -- 172.21.15.97:0/368643408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 msgr2=0x55d42125e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919400640 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 0x55d42125e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919400640 1 -- 172.21.15.97:0/368643408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 msgr2=0x55d42125fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919400640 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d42125fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919400640 1 -- 172.21.15.97:0/368643408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4212603b0 con 0x55d421250790 2024-03-26T19:08:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f191a800640 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d42125fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:08:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f1919400640 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1908011010 tx=0x7f1908009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=170a9329126033a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.783+0000 7f18fb400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:08:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.784+0000 7f18fb400640 1 -- 172.21.15.97:0/368643408 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908041020 con 0x55d421250790 2024-03-26T19:08:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.784+0000 7f18fb400640 1 -- 172.21.15.97:0/368643408 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f19080295b0 con 0x55d421250790 2024-03-26T19:08:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.784+0000 7f19237f5740 1 -- 172.21.15.97:0/368643408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19080283b0 con 0x55d421250790 2024-03-26T19:08:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.785+0000 7f18fb400640 1 -- 172.21.15.97:0/368643408 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1908038040 con 0x55d421250790 2024-03-26T19:08:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.785+0000 7f19237f5740 10 client.25966 fetch_fsmap learned FSMap version 27 2024-03-26T19:08:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.785+0000 7f19237f5740 1 -- 172.21.15.97:0/368643408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d4212616c0 con 0x55d421250790 2024-03-26T19:08:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.785+0000 7f18fb400640 1 -- 172.21.15.97:0/368643408 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19080283b0 con 0x55d421250790 2024-03-26T19:08:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.786+0000 7f18fb400640 1 -- 172.21.15.97:0/368643408 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f190803d8e0 con 0x55d421250790 2024-03-26T19:08:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.786+0000 7f19237f5740 10 client.25966 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:08:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.786+0000 7f19237f5740 10 client.25966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:08:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.786+0000 7f19237f5740 20 client.25966 populate_metadata read hostname 'smithi097' 2024-03-26T19:08:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.786+0000 7f19237f5740 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d4212cc900 0x55d4212ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.786+0000 7f19237f5740 4 client.25966 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:08:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.786+0000 7f19237f5740 1 -- 172.21.15.97:0/368643408 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d4211bc710 con 0x55d4212cc900 2024-03-26T19:08:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.787+0000 7f1919e00640 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d4212cc900 0x55d4212ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.787+0000 7f18fb400640 10 client.25966 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.787+0000 7f1919e00640 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d4212cc900 0x55d4212ced20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1c9d03faa0d9ce server_cookie=5fb4a89f9c840705 in_seq=0 out_seq=0 2024-03-26T19:08:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.810+0000 7f18fb400640 1 -- 172.21.15.97:0/368643408 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d4211bc710 con 0x55d4212cc900 2024-03-26T19:08:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.810+0000 7f18fb400640 10 client.25966 handle_command_reply: tid=0 2024-03-26T19:08:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.811+0000 7f19237f5740 1 client.25966 shutdown 2024-03-26T19:08:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.811+0000 7f19237f5740 1 -- 172.21.15.97:0/368643408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 msgr2=0x55d42125edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.811+0000 7f19237f5740 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1908011010 tx=0x7f1908009f60 comp rx=0 tx=0).stop 2024-03-26T19:08:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.811+0000 7f19237f5740 1 -- 172.21.15.97:0/368643408 shutdown_connections 2024-03-26T19:08:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.811+0000 7f19237f5740 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55d4212cc900 0x55d4212ced20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.811+0000 7f19237f5740 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d421255900 0x55d42125fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.811+0000 7f19237f5740 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d421250790 0x55d42125edb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.811+0000 7f19237f5740 1 --2- 172.21.15.97:0/368643408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d42124fda0 0x55d42125e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.811+0000 7f19237f5740 1 -- 172.21.15.97:0/368643408 >> 172.21.15.97:0/368643408 conn(0x55d42124b310 msgr2=0x55d42124ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.811+0000 7f18fb400640 0 client.25966 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.812+0000 7f19237f5740 1 -- 172.21.15.97:0/368643408 shutdown_connections 2024-03-26T19:08:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.812+0000 7f19237f5740 1 -- 172.21.15.97:0/368643408 wait complete. 2024-03-26T19:08:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.812+0000 7f19237f5740 20 client.25966 trim_cache size 0 max 16384 2024-03-26T19:08:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:08:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:08:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:08:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:08:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.813+0000 7f1920c00640 1 -- 172.21.15.97:0/198871198 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19000a3110 msgr2=0x7f19000a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.813+0000 7f1920c00640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19000a3110 0x7f19000a55d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55d421250ff0 tx=0x7f1908009260 comp rx=0 tx=0).stop 2024-03-26T19:08:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.813+0000 7f1920c00640 1 -- 172.21.15.97:0/198871198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f191c10a390 msgr2=0x7f191c1ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.813+0000 7f1920c00640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f191c10a390 0x7f191c1ade70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1914036040 tx=0x7f19140337b0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.814+0000 7f1920c00640 1 -- 172.21.15.97:0/198871198 shutdown_connections 2024-03-26T19:08:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.814+0000 7f1920c00640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f19000a3110 0x7f19000a55d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.814+0000 7f1920c00640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f191c10a390 0x7f191c1ade70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.814+0000 7f1920c00640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c1099d0 0x7f191c1a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.814+0000 7f1920c00640 1 --2- 172.21.15.97:0/198871198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f191c108c60 0x7f191c1a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.814+0000 7f1920c00640 1 -- 172.21.15.97:0/198871198 >> 172.21.15.97:0/198871198 conn(0x7f191c06ed20 msgr2=0x7f191c1158b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.815+0000 7f1920c00640 1 -- 172.21.15.97:0/198871198 shutdown_connections 2024-03-26T19:08:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:42.815+0000 7f1920c00640 1 -- 172.21.15.97:0/198871198 wait complete. 2024-03-26T19:08:42.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:42 smithi097 ceph-mon[31955]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 772 op/s 2024-03-26T19:08:44.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:44 smithi105 ceph-mon[36270]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 8.2 MiB/s wr, 646 op/s 2024-03-26T19:08:44.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:44 smithi097 ceph-mon[31955]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 8.2 MiB/s wr, 646 op/s 2024-03-26T19:08:44.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:44 smithi144 ceph-mon[36053]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 8.2 MiB/s wr, 646 op/s 2024-03-26T19:08:45.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 560.0 s, 1411.9 tps, lat 70.395 ms stddev 78.685 2024-03-26T19:08:45.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:45 smithi105 ceph-mon[36270]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 686 op/s 2024-03-26T19:08:45.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:45 smithi097 ceph-mon[31955]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 686 op/s 2024-03-26T19:08:45.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:45 smithi144 ceph-mon[36053]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 686 op/s 2024-03-26T19:08:48.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:48 smithi097 ceph-mon[31955]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.7 MiB/s wr, 593 op/s 2024-03-26T19:08:48.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:48 smithi144 ceph-mon[36053]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.7 MiB/s wr, 593 op/s 2024-03-26T19:08:48.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:48 smithi105 ceph-mon[36270]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.7 MiB/s wr, 593 op/s 2024-03-26T19:08:50.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 565.0 s, 1658.8 tps, lat 59.851 ms stddev 83.389 2024-03-26T19:08:50.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:50 smithi097 ceph-mon[31955]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.2 MiB/s wr, 683 op/s 2024-03-26T19:08:50.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:50 smithi144 ceph-mon[36053]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.2 MiB/s wr, 683 op/s 2024-03-26T19:08:50.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:50 smithi105 ceph-mon[36270]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.2 MiB/s wr, 683 op/s 2024-03-26T19:08:52.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:52 smithi097 ceph-mon[31955]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 727 op/s 2024-03-26T19:08:52.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:52 smithi144 ceph-mon[36053]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 727 op/s 2024-03-26T19:08:52.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:52 smithi105 ceph-mon[36270]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 727 op/s 2024-03-26T19:08:54.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:54 smithi097 ceph-mon[31955]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 565 op/s 2024-03-26T19:08:54.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:54 smithi144 ceph-mon[36053]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 565 op/s 2024-03-26T19:08:54.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:54 smithi105 ceph-mon[36270]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 565 op/s 2024-03-26T19:08:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 570.0 s, 1614.4 tps, lat 62.977 ms stddev 78.624 2024-03-26T19:08:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.012+0000 7f18ec800640 1 -- 172.21.15.97:0/852528913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e40774c0 msgr2=0x7f18e4077920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.013+0000 7f18ec800640 1 --2- 172.21.15.97:0/852528913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e40774c0 0x7f18e4077920 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009900 tx=0x7f18d802bc30 comp rx=0 tx=0).stop 2024-03-26T19:08:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.013+0000 7f18ec800640 1 -- 172.21.15.97:0/852528913 shutdown_connections 2024-03-26T19:08:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.013+0000 7f18ec800640 1 --2- 172.21.15.97:0/852528913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e4077e60 0x7f18e4078700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.013+0000 7f18ec800640 1 --2- 172.21.15.97:0/852528913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e40774c0 0x7f18e4077920 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.013+0000 7f18ec800640 1 --2- 172.21.15.97:0/852528913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18e4076280 0x7f18e40766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.013+0000 7f18ec800640 1 -- 172.21.15.97:0/852528913 >> 172.21.15.97:0/852528913 conn(0x7f18e40ff0b0 msgr2=0x7f18e41014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.013+0000 7f18ec800640 1 -- 172.21.15.97:0/852528913 shutdown_connections 2024-03-26T19:08:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.014+0000 7f18ec800640 1 -- 172.21.15.97:0/852528913 wait complete. 2024-03-26T19:08:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.015+0000 7f18ec800640 1 Processor -- start 2024-03-26T19:08:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.016+0000 7f18ec800640 1 -- start start 2024-03-26T19:08:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.016+0000 7f18ec800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4076280 0x7f18e419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.016+0000 7f18ec800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e40774c0 0x7f18e419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.016+0000 7f18ec800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18e4077e60 0x7f18e41a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.016+0000 7f18ec800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4113bb0 con 0x7f18e4077e60 2024-03-26T19:08:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.016+0000 7f18ec800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4113a30 con 0x7f18e4076280 2024-03-26T19:08:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.016+0000 7f18ec800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4113d30 con 0x7f18e40774c0 2024-03-26T19:08:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.016+0000 7f18e9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4076280 0x7f18e419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18e9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e40774c0 0x7f18e419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18e9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4076280 0x7f18e419e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:42434/0 (socket says 172.21.15.97:42434) 2024-03-26T19:08:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18e9e00640 1 -- 172.21.15.97:0/402650078 learned_addr learned my addr 172.21.15.97:0/402650078 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18ea800640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18e4077e60 0x7f18e41a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18e9400640 1 -- 172.21.15.97:0/402650078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4076280 msgr2=0x7f18e419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18e9400640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4076280 0x7f18e419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18e9400640 1 -- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18e4077e60 msgr2=0x7f18e41a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18e9400640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18e4077e60 0x7f18e41a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18e9400640 1 -- 172.21.15.97:0/402650078 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e41a5d00 con 0x7f18e40774c0 2024-03-26T19:08:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18ea800640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18e4077e60 0x7f18e41a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:08:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.017+0000 7f18e9e00640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4076280 0x7f18e419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:08:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.018+0000 7f18e9400640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e40774c0 0x7f18e419e6e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18d8036040 tx=0x7f18d802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.018+0000 7f18dea00640 1 -- 172.21.15.97:0/402650078 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8043020 con 0x7f18e40774c0 2024-03-26T19:08:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.018+0000 7f18dea00640 1 -- 172.21.15.97:0/402650078 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d80054e0 con 0x7f18e40774c0 2024-03-26T19:08:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.018+0000 7f18dea00640 1 -- 172.21.15.97:0/402650078 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8003ca0 con 0x7f18e40774c0 2024-03-26T19:08:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.018+0000 7f18ec800640 1 -- 172.21.15.97:0/402650078 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e41a5f90 con 0x7f18e40774c0 2024-03-26T19:08:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.018+0000 7f18ec800640 1 -- 172.21.15.97:0/402650078 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e41a6400 con 0x7f18e40774c0 2024-03-26T19:08:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.022+0000 7f18dea00640 1 -- 172.21.15.97:0/402650078 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f18d8046020 con 0x7f18e40774c0 2024-03-26T19:08:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.025+0000 7f18dea00640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18cc0a3130 0x7f18cc0a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.025+0000 7f18dea00640 1 -- 172.21.15.97:0/402650078 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f18d80f1400 con 0x7f18e40774c0 2024-03-26T19:08:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.025+0000 7f18ef5f2740 1 Processor -- start 2024-03-26T19:08:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.026+0000 7f18ef5f2740 1 -- start start 2024-03-26T19:08:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.026+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 0x55dc9fdfe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.026+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 0x55dc9fe03290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.026+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fe05d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.026+0000 7f18ef5f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dc9fd62e80 con 0x55dc9fdfe790 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.026+0000 7f18ef5f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dc9fd0b660 con 0x55dc9fe03900 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18ef5f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dc9fd5ee30 con 0x55dc9fdfdda0 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18e9e00640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18cc0a3130 0x7f18cc0a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18ea800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fe05d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18ea800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fe05d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:42446/0 (socket says 172.21.15.97:42446) 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18ea800640 1 -- 172.21.15.97:0/3490926833 learned_addr learned my addr 172.21.15.97:0/3490926833 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18e9400640 1 --2- 172.21.15.97:0/3490926833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 0x55dc9fe03290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18ea800640 1 -- 172.21.15.97:0/3490926833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 msgr2=0x55dc9fdfe1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18ea800640 1 --2- 172.21.15.97:0/3490926833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 0x55dc9fdfe1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18ea800640 1 -- 172.21.15.97:0/3490926833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 msgr2=0x55dc9fe03290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18ea800640 1 --2- 172.21.15.97:0/3490926833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 0x55dc9fe03290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18ea800640 1 -- 172.21.15.97:0/3490926833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9fa77210 con 0x55dc9fe03900 2024-03-26T19:08:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.027+0000 7f18e9400640 1 --2- 172.21.15.97:0/3490926833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 0x55dc9fe03290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:08:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.028+0000 7f18ea800640 1 --2- 172.21.15.97:0/3490926833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fe05d90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009cc0 tx=0x7f18d40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5a51159799b9e7a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.028+0000 7f18e9e00640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18cc0a3130 0x7f18cc0a55f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18e0007fd0 tx=0x7f18e0007620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.028+0000 7f18d3000640 1 -- 172.21.15.97:0/3490926833 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4012450 con 0x55dc9fe03900 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.028+0000 7f18d3000640 1 -- 172.21.15.97:0/3490926833 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d4012ad0 con 0x55dc9fe03900 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.028+0000 7f18d3000640 1 -- 172.21.15.97:0/3490926833 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d400fc90 con 0x55dc9fe03900 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.029+0000 7f18ef5f2740 1 -- 172.21.15.97:0/3490926833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 msgr2=0x55dc9fe05d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.029+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/3490926833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fe05d90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009cc0 tx=0x7f18d40117b0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.029+0000 7f18ef5f2740 1 -- 172.21.15.97:0/3490926833 shutdown_connections 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.029+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/3490926833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fe05d90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.029+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/3490926833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 0x55dc9fe03290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.029+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/3490926833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 0x55dc9fdfe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.029+0000 7f18ef5f2740 1 -- 172.21.15.97:0/3490926833 >> 172.21.15.97:0/3490926833 conn(0x55dc9fdf9310 msgr2=0x55dc9fdf9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.030+0000 7f18ef5f2740 1 -- 172.21.15.97:0/3490926833 shutdown_connections 2024-03-26T19:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.030+0000 7f18ef5f2740 1 -- 172.21.15.97:0/3490926833 wait complete. 2024-03-26T19:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.030+0000 7f18ef5f2740 1 -- start start 2024-03-26T19:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.030+0000 7f18ef5f2740 1 Processor -- start 2024-03-26T19:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.032+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfdda0 0x55dc9fe18d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.032+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fdfe790 0x55dc9fe192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.032+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fe03900 0x55dc9fe20310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.032+0000 7f18ef5f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dc9fe178c0 con 0x55dc9fdfdda0 2024-03-26T19:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.032+0000 7f18ef5f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dc9fd1eac0 con 0x55dc9fdfe790 2024-03-26T19:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.032+0000 7f18ef5f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dc9fe17a40 con 0x55dc9fe03900 2024-03-26T19:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.032+0000 7f18ef5f2740 10 client.0 _ll_register_callbacks cb 0x55dc9fd1e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.033+0000 7f18ea800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fe03900 0x55dc9fe20310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.033+0000 7f18e9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfdda0 0x55dc9fe18d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.033+0000 7f18ea800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fe03900 0x55dc9fe20310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44152/0 (socket says 172.21.15.97:44152) 2024-03-26T19:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.033+0000 7f18ea800640 1 -- 172.21.15.97:0/207432766 learned_addr learned my addr 172.21.15.97:0/207432766 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.033+0000 7f18e9400640 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fdfe790 0x55dc9fe192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.034+0000 7f18ea800640 1 -- 172.21.15.97:0/207432766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fdfe790 msgr2=0x55dc9fe192d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.034+0000 7f18ea800640 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fdfe790 0x55dc9fe192d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.034+0000 7f18ea800640 1 -- 172.21.15.97:0/207432766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfdda0 msgr2=0x55dc9fe18d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.034+0000 7f18ea800640 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfdda0 0x55dc9fe18d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.034+0000 7f18ea800640 1 -- 172.21.15.97:0/207432766 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9fb43620 con 0x55dc9fe03900 2024-03-26T19:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.034+0000 7f18e9400640 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fdfe790 0x55dc9fe192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.034+0000 7f18ea800640 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fe03900 0x55dc9fe20310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009c90 tx=0x7f18d400ff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e221076644590f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.035+0000 7f18c7400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.035+0000 7f18c7400640 1 -- 172.21.15.97:0/207432766 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4028020 con 0x55dc9fe03900 2024-03-26T19:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.035+0000 7f18c7400640 1 -- 172.21.15.97:0/207432766 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f18d4012860 con 0x55dc9fe03900 2024-03-26T19:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.037+0000 7f18ef5f2740 1 -- 172.21.15.97:0/207432766 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18d4012ad0 con 0x55dc9fe03900 2024-03-26T19:08:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.037+0000 7f18c7400640 1 -- 172.21.15.97:0/207432766 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18d4005040 con 0x55dc9fe03900 2024-03-26T19:08:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.037+0000 7f18ef5f2740 10 client.25975 fetch_fsmap learned FSMap version 27 2024-03-26T19:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.037+0000 7f18ef5f2740 1 -- 172.21.15.97:0/207432766 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc9fe236d0 con 0x55dc9fe03900 2024-03-26T19:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.038+0000 7f18c7400640 1 -- 172.21.15.97:0/207432766 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d4012ad0 con 0x55dc9fe03900 2024-03-26T19:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.038+0000 7f18c7400640 1 -- 172.21.15.97:0/207432766 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f18d4004ad0 con 0x55dc9fe03900 2024-03-26T19:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.038+0000 7f18ef5f2740 10 client.25975 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.038+0000 7f18ef5f2740 10 client.25975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.038+0000 7f18ef5f2740 20 client.25975 populate_metadata read hostname 'smithi097' 2024-03-26T19:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.039+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dc9fe24ae0 0x55dc9fe26ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.039+0000 7f18ef5f2740 4 client.25975 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:08:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.039+0000 7f18ef5f2740 1 -- 172.21.15.97:0/207432766 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc9fd703c0 con 0x55dc9fe24ae0 2024-03-26T19:08:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.039+0000 7f18e9e00640 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dc9fe24ae0 0x55dc9fe26ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.040+0000 7f18e9e00640 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dc9fe24ae0 0x55dc9fe26ed0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29870f0e9dbf7396 server_cookie=d58e9d4b35ba85df in_seq=0 out_seq=0 2024-03-26T19:08:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.040+0000 7f18c7400640 10 client.25975 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.041+0000 7f18c7400640 1 -- 172.21.15.97:0/207432766 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dc9fd703c0 con 0x55dc9fe24ae0 2024-03-26T19:08:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.041+0000 7f18c7400640 10 client.25975 handle_command_reply: tid=0 2024-03-26T19:08:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.041+0000 7f18ef5f2740 1 client.25975 shutdown 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 1 -- 172.21.15.97:0/207432766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fe03900 msgr2=0x55dc9fe20310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fe03900 0x55dc9fe20310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009c90 tx=0x7f18d400ff60 comp rx=0 tx=0).stop 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 1 -- 172.21.15.97:0/207432766 shutdown_connections 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dc9fe24ae0 0x55dc9fe26ed0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fe03900 0x55dc9fe20310 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fdfe790 0x55dc9fe192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/207432766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfdda0 0x55dc9fe18d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18c7400640 0 client.25975 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 1 -- 172.21.15.97:0/207432766 >> 172.21.15.97:0/207432766 conn(0x55dc9fdf9310 msgr2=0x55dc9fdf8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 1 -- 172.21.15.97:0/207432766 shutdown_connections 2024-03-26T19:08:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 1 -- 172.21.15.97:0/207432766 wait complete. 2024-03-26T19:08:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.042+0000 7f18ef5f2740 20 client.25975 trim_cache size 0 max 16384 2024-03-26T19:08:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.068+0000 7f18ef5f2740 1 Processor -- start 2024-03-26T19:08:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.068+0000 7f18ef5f2740 1 -- start start 2024-03-26T19:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfdda0 0x55dc9fdfcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfe790 0x55dc9fe202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fdff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ef5f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41a6ab0 con 0x55dc9fdfdda0 2024-03-26T19:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ef5f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41a85f0 con 0x55dc9fe03900 2024-03-26T19:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ef5f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41a7180 con 0x55dc9fdfe790 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ea800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fdff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ea800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fdff020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:42464/0 (socket says 172.21.15.97:42464) 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ea800640 1 -- 172.21.15.97:0/1022258686 learned_addr learned my addr 172.21.15.97:0/1022258686 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18e9e00640 1 --2- 172.21.15.97:0/1022258686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfdda0 0x55dc9fdfcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18e9400640 1 --2- 172.21.15.97:0/1022258686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfe790 0x55dc9fe202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ea800640 1 -- 172.21.15.97:0/1022258686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfe790 msgr2=0x55dc9fe202d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ea800640 1 --2- 172.21.15.97:0/1022258686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfe790 0x55dc9fe202d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ea800640 1 -- 172.21.15.97:0/1022258686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfdda0 msgr2=0x55dc9fdfcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ea800640 1 --2- 172.21.15.97:0/1022258686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfdda0 0x55dc9fdfcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.069+0000 7f18ea800640 1 -- 172.21.15.97:0/1022258686 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9fdf7ab0 con 0x55dc9fe03900 2024-03-26T19:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.070+0000 7f18ea800640 1 --2- 172.21.15.97:0/1022258686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fdff020 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009be0 tx=0x7f18d400ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa27aca47cd2d0c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.070+0000 7f18d2600640 1 -- 172.21.15.97:0/1022258686 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4028040 con 0x55dc9fe03900 2024-03-26T19:08:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.070+0000 7f18d2600640 1 -- 172.21.15.97:0/1022258686 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d4012660 con 0x55dc9fe03900 2024-03-26T19:08:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.070+0000 7f18d2600640 1 -- 172.21.15.97:0/1022258686 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4005540 con 0x55dc9fe03900 2024-03-26T19:08:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.071+0000 7f18ef5f2740 1 -- 172.21.15.97:0/1022258686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 msgr2=0x55dc9fdff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.071+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/1022258686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fdff020 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009be0 tx=0x7f18d400ffd0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.071+0000 7f18ef5f2740 1 -- 172.21.15.97:0/1022258686 shutdown_connections 2024-03-26T19:08:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.071+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/1022258686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fdff020 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.071+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/1022258686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfe790 0x55dc9fe202d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.071+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/1022258686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfdda0 0x55dc9fdfcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.071+0000 7f18ef5f2740 1 -- 172.21.15.97:0/1022258686 >> 172.21.15.97:0/1022258686 conn(0x55dc9fdf9310 msgr2=0x55dc9fdfc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.071+0000 7f18ef5f2740 1 -- 172.21.15.97:0/1022258686 shutdown_connections 2024-03-26T19:08:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.071+0000 7f18ef5f2740 1 -- 172.21.15.97:0/1022258686 wait complete. 2024-03-26T19:08:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.071+0000 7f18ef5f2740 1 -- start start 2024-03-26T19:08:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.072+0000 7f18ef5f2740 1 Processor -- start 2024-03-26T19:08:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.072+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 0x55dc9fe1e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 0x55dc9fe1e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18ef5f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fe1f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18ef5f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41a7f20 con 0x55dc9fdfe790 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18e9e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 0x55dc9fe1e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18e9e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 0x55dc9fe1e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:44170/0 (socket says 172.21.15.97:44170) 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18e9e00640 1 -- 172.21.15.97:0/479375890 learned_addr learned my addr 172.21.15.97:0/479375890 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18ea800640 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fe1f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18ef5f2740 1 -- 172.21.15.97:0/479375890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41a7850 con 0x55dc9fe03900 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18ef5f2740 1 -- 172.21.15.97:0/479375890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18e410a0f0 con 0x55dc9fdfdda0 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18ef5f2740 10 client.0 _ll_register_callbacks cb 0x7f18d400fe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.073+0000 7f18e9400640 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 0x55dc9fe1e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.074+0000 7f18e9e00640 1 -- 172.21.15.97:0/479375890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 msgr2=0x55dc9fe1f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.074+0000 7f18e9e00640 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fe1f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.074+0000 7f18e9e00640 1 -- 172.21.15.97:0/479375890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 msgr2=0x55dc9fe1e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.074+0000 7f18e9e00640 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 0x55dc9fe1e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.074+0000 7f18e9e00640 1 -- 172.21.15.97:0/479375890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9fe19640 con 0x55dc9fdfdda0 2024-03-26T19:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.074+0000 7f18e9400640 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 0x55dc9fe1e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.074+0000 7f18e9e00640 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 0x55dc9fe1e060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f18e0013000 tx=0x7f18e0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9709d897b5b364 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.074+0000 7f18c7400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.075+0000 7f18c7400640 1 -- 172.21.15.97:0/479375890 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0004d20 con 0x55dc9fdfdda0 2024-03-26T19:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.075+0000 7f18c7400640 1 -- 172.21.15.97:0/479375890 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f18e00262e0 con 0x55dc9fdfdda0 2024-03-26T19:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.075+0000 7f18ef5f2740 1 -- 172.21.15.97:0/479375890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18d4012660 con 0x55dc9fdfdda0 2024-03-26T19:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.075+0000 7f18c7400640 1 -- 172.21.15.97:0/479375890 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18e00264e0 con 0x55dc9fdfdda0 2024-03-26T19:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.075+0000 7f18ef5f2740 10 client.25981 fetch_fsmap learned FSMap version 27 2024-03-26T19:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.075+0000 7f18ef5f2740 1 -- 172.21.15.97:0/479375890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc9fdfd9e0 con 0x55dc9fdfdda0 2024-03-26T19:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.076+0000 7f18c7400640 1 -- 172.21.15.97:0/479375890 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d4012660 con 0x55dc9fdfdda0 2024-03-26T19:08:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.076+0000 7f18c7400640 1 -- 172.21.15.97:0/479375890 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f18e0005120 con 0x55dc9fdfdda0 2024-03-26T19:08:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.077+0000 7f18ef5f2740 10 client.25981 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:08:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.077+0000 7f18ef5f2740 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:08:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.077+0000 7f18ef5f2740 20 client.25981 populate_metadata read hostname 'smithi097' 2024-03-26T19:08:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.077+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dc9fe8b960 0x55dc9fe8dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.077+0000 7f18ef5f2740 4 client.25981 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.077+0000 7f18ef5f2740 1 -- 172.21.15.97:0/479375890 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc9fd6a710 con 0x55dc9fe8b960 2024-03-26T19:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.077+0000 7f18e9400640 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dc9fe8b960 0x55dc9fe8dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.078+0000 7f18e9400640 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dc9fe8b960 0x55dc9fe8dd80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d3fb2be6cedc4f server_cookie=1d7e577fad4c2ae8 in_seq=0 out_seq=0 2024-03-26T19:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.078+0000 7f18c7400640 10 client.25981 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.099+0000 7f18c7400640 1 -- 172.21.15.97:0/479375890 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc9fd6a710 con 0x55dc9fe8b960 2024-03-26T19:08:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.099+0000 7f18c7400640 10 client.25981 handle_command_reply: tid=0 2024-03-26T19:08:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.099+0000 7f18ef5f2740 1 client.25981 shutdown 2024-03-26T19:08:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 1 -- 172.21.15.97:0/479375890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 msgr2=0x55dc9fe1e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 0x55dc9fe1e060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f18e0013000 tx=0x7f18e0004ab0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 1 -- 172.21.15.97:0/479375890 shutdown_connections 2024-03-26T19:08:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dc9fe8b960 0x55dc9fe8dd80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dc9fe03900 0x55dc9fe1f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc9fdfe790 0x55dc9fe1e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 1 --2- 172.21.15.97:0/479375890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc9fdfdda0 0x55dc9fe1e060 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 1 -- 172.21.15.97:0/479375890 >> 172.21.15.97:0/479375890 conn(0x55dc9fdf9310 msgr2=0x55dc9fdf7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18c7400640 0 client.25981 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 1 -- 172.21.15.97:0/479375890 shutdown_connections 2024-03-26T19:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 1 -- 172.21.15.97:0/479375890 wait complete. 2024-03-26T19:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.100+0000 7f18ef5f2740 20 client.25981 trim_cache size 0 max 16384 2024-03-26T19:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.101+0000 7f18ec800640 1 -- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18cc0a3130 msgr2=0x7f18cc0a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.101+0000 7f18ec800640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18cc0a3130 0x7f18cc0a55f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18e0007fd0 tx=0x7f18e0007620 comp rx=0 tx=0).stop 2024-03-26T19:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.101+0000 7f18ec800640 1 -- 172.21.15.97:0/402650078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e40774c0 msgr2=0x7f18e419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.101+0000 7f18ec800640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e40774c0 0x7f18e419e6e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18d8036040 tx=0x7f18d802c6c0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.101+0000 7f18ec800640 1 -- 172.21.15.97:0/402650078 shutdown_connections 2024-03-26T19:08:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.101+0000 7f18ec800640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18cc0a3130 0x7f18cc0a55f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.101+0000 7f18ec800640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18e4077e60 0x7f18e41a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.101+0000 7f18ec800640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e40774c0 0x7f18e419e6e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.101+0000 7f18ec800640 1 --2- 172.21.15.97:0/402650078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4076280 0x7f18e419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:08:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.101+0000 7f18ec800640 1 -- 172.21.15.97:0/402650078 >> 172.21.15.97:0/402650078 conn(0x7f18e40ff0b0 msgr2=0x7f18e4100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:08:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.102+0000 7f18ec800640 1 -- 172.21.15.97:0/402650078 shutdown_connections 2024-03-26T19:08:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:08:56.102+0000 7f18ec800640 1 -- 172.21.15.97:0/402650078 wait complete. 2024-03-26T19:08:56.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:56 smithi097 ceph-mon[31955]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 MiB/s wr, 669 op/s 2024-03-26T19:08:56.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:56 smithi144 ceph-mon[36053]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 MiB/s wr, 669 op/s 2024-03-26T19:08:56.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:56 smithi105 ceph-mon[36270]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 MiB/s wr, 669 op/s 2024-03-26T19:08:58.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:08:58 smithi097 ceph-mon[31955]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 571 op/s 2024-03-26T19:08:58.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:08:58 smithi144 ceph-mon[36053]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 571 op/s 2024-03-26T19:08:58.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:08:58 smithi105 ceph-mon[36270]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 571 op/s 2024-03-26T19:09:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 575.0 s, 1556.2 tps, lat 63.959 ms stddev 75.239 2024-03-26T19:09:00.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:00 smithi097 ceph-mon[31955]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 639 op/s 2024-03-26T19:09:00.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:00 smithi144 ceph-mon[36053]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 639 op/s 2024-03-26T19:09:00.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:00 smithi105 ceph-mon[36270]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 639 op/s 2024-03-26T19:09:02.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:02 smithi097 ceph-mon[31955]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 684 op/s 2024-03-26T19:09:02.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:02 smithi144 ceph-mon[36053]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 684 op/s 2024-03-26T19:09:02.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:02 smithi105 ceph-mon[36270]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 684 op/s 2024-03-26T19:09:04.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:04 smithi097 ceph-mon[31955]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 600 op/s 2024-03-26T19:09:04.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:04 smithi144 ceph-mon[36053]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 600 op/s 2024-03-26T19:09:04.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:04 smithi105 ceph-mon[36270]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 600 op/s 2024-03-26T19:09:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 580.0 s, 1224.6 tps, lat 80.881 ms stddev 96.077 2024-03-26T19:09:06.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:06 smithi097 ceph-mon[31955]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 729 op/s 2024-03-26T19:09:06.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:06 smithi144 ceph-mon[36053]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 729 op/s 2024-03-26T19:09:06.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:06 smithi105 ceph-mon[36270]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 729 op/s 2024-03-26T19:09:08.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:08 smithi144 ceph-mon[36053]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 588 op/s 2024-03-26T19:09:08.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:08 smithi105 ceph-mon[36270]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 588 op/s 2024-03-26T19:09:08.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:08 smithi097 ceph-mon[31955]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 588 op/s 2024-03-26T19:09:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.285+0000 7f3cb5000640 1 -- 172.21.15.97:0/2699969868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cb0109780 msgr2=0x7f3cb010bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.285+0000 7f3cb5000640 1 --2- 172.21.15.97:0/2699969868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cb0109780 0x7f3cb010bc20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009ca0 tx=0x7f3ca802c180 comp rx=0 tx=0).stop 2024-03-26T19:09:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.285+0000 7f3cb5000640 1 -- 172.21.15.97:0/2699969868 shutdown_connections 2024-03-26T19:09:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.285+0000 7f3cb5000640 1 --2- 172.21.15.97:0/2699969868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cb0109780 0x7f3cb010bc20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.285+0000 7f3cb5000640 1 --2- 172.21.15.97:0/2699969868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb00742c0 0x7f3cb0074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.285+0000 7f3cb5000640 1 --2- 172.21.15.97:0/2699969868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb00758a0 0x7f3cb0073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.285+0000 7f3cb5000640 1 -- 172.21.15.97:0/2699969868 >> 172.21.15.97:0/2699969868 conn(0x7f3cb00fce80 msgr2=0x7f3cb00ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.285+0000 7f3cb5000640 1 -- 172.21.15.97:0/2699969868 shutdown_connections 2024-03-26T19:09:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.286+0000 7f3cb5000640 1 -- 172.21.15.97:0/2699969868 wait complete. 2024-03-26T19:09:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.287+0000 7f3cb5000640 1 Processor -- start 2024-03-26T19:09:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.287+0000 7f3cb5000640 1 -- start start 2024-03-26T19:09:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.287+0000 7f3cb5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb00742c0 0x7f3cb0104c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.287+0000 7f3cb5000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cb00758a0 0x7f3cb01051b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.287+0000 7f3cb5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0109780 0x7f3cb01017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cb5000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb010f7c0 con 0x7f3cb00742c0 2024-03-26T19:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cb5000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb010f640 con 0x7f3cb00758a0 2024-03-26T19:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cb5000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb010f940 con 0x7f3cb0109780 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cad400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cb00758a0 0x7f3cb01051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cae800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0109780 0x7f3cb01017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cad400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cb00758a0 0x7f3cb01051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60954/0 (socket says 172.21.15.97:60954) 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cad400640 1 -- 172.21.15.97:0/1488080542 learned_addr learned my addr 172.21.15.97:0/1488080542 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cae800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0109780 0x7f3cb01017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60806/0 (socket says 172.21.15.97:60806) 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cade00640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb00742c0 0x7f3cb0104c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cad400640 1 -- 172.21.15.97:0/1488080542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0109780 msgr2=0x7f3cb01017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.288+0000 7f3cad400640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0109780 0x7f3cb01017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.289+0000 7f3cad400640 1 -- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb00742c0 msgr2=0x7f3cb0104c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.289+0000 7f3cad400640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb00742c0 0x7f3cb0104c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.289+0000 7f3cad400640 1 -- 172.21.15.97:0/1488080542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb01020a0 con 0x7f3cb00758a0 2024-03-26T19:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.289+0000 7f3cade00640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb00742c0 0x7f3cb0104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.289+0000 7f3cae800640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0109780 0x7f3cb01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.289+0000 7f3cad400640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cb00758a0 0x7f3cb01051b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00d950 tx=0x7f3c9c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.290+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1488080542 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c017070 con 0x7f3cb00758a0 2024-03-26T19:09:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.290+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1488080542 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c9c009c30 con 0x7f3cb00758a0 2024-03-26T19:09:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.290+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1488080542 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c00f620 con 0x7f3cb00758a0 2024-03-26T19:09:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.290+0000 7f3cb5000640 1 -- 172.21.15.97:0/1488080542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb01a61c0 con 0x7f3cb00758a0 2024-03-26T19:09:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.290+0000 7f3cb5000640 1 -- 172.21.15.97:0/1488080542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb01a6650 con 0x7f3cb00758a0 2024-03-26T19:09:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.294+0000 7f3cb7c46740 1 Processor -- start 2024-03-26T19:09:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.296+0000 7f3cb7c46740 1 -- start start 2024-03-26T19:09:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.296+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53906da0 0x560f539071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.296+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f53907790 0x560f5390c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.296+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f5390c900 0x560f5390ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.296+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560f5386be80 con 0x560f53907790 2024-03-26T19:09:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.296+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560f53814660 con 0x560f5390c900 2024-03-26T19:09:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.296+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560f53867e30 con 0x560f53906da0 2024-03-26T19:09:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.297+0000 7f3cae800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53906da0 0x560f539071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.297+0000 7f3cae800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53906da0 0x560f539071c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60822/0 (socket says 172.21.15.97:60822) 2024-03-26T19:09:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.297+0000 7f3cae800640 1 -- 172.21.15.97:0/3603444869 learned_addr learned my addr 172.21.15.97:0/3603444869 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.297+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1488080542 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f3c9c00f7c0 con 0x7f3cb00758a0 2024-03-26T19:09:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.297+0000 7f3cae800640 1 -- 172.21.15.97:0/3603444869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f5390c900 msgr2=0x560f5390ed90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:09:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.297+0000 7f3cae800640 1 --2- 172.21.15.97:0/3603444869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f5390c900 0x560f5390ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.297+0000 7f3cae800640 1 -- 172.21.15.97:0/3603444869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f53907790 msgr2=0x560f5390c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.297+0000 7f3cae800640 1 --2- 172.21.15.97:0/3603444869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f53907790 0x560f5390c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.297+0000 7f3cae800640 1 -- 172.21.15.97:0/3603444869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f53580210 con 0x560f53906da0 2024-03-26T19:09:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.298+0000 7f3cae800640 1 --2- 172.21.15.97:0/3603444869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53906da0 0x560f539071c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3ca802c790 tx=0x7f3ca80098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30240ac85bf06c23 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.298+0000 7f3c9b400640 1 -- 172.21.15.97:0/3603444869 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8043070 con 0x560f53906da0 2024-03-26T19:09:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.298+0000 7f3c9b400640 1 -- 172.21.15.97:0/3603444869 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca8034cd0 con 0x560f53906da0 2024-03-26T19:09:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.298+0000 7f3c9b400640 1 -- 172.21.15.97:0/3603444869 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8039040 con 0x560f53906da0 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3ca6a00640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3c940a3090 0x7f3c940a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3cb7c46740 1 -- 172.21.15.97:0/3603444869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53906da0 msgr2=0x560f539071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/3603444869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53906da0 0x560f539071c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3ca802c790 tx=0x7f3ca80098b0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1488080542 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3c9c0cefe0 con 0x7f3cb00758a0 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3cb7c46740 1 -- 172.21.15.97:0/3603444869 shutdown_connections 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/3603444869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f5390c900 0x560f5390ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/3603444869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f53907790 0x560f5390c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/3603444869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53906da0 0x560f539071c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3cb7c46740 1 -- 172.21.15.97:0/3603444869 >> 172.21.15.97:0/3603444869 conn(0x560f53902310 msgr2=0x560f53902710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3cade00640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3c940a3090 0x7f3c940a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.299+0000 7f3cb7c46740 1 -- 172.21.15.97:0/3603444869 shutdown_connections 2024-03-26T19:09:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.300+0000 7f3cb7c46740 1 -- 172.21.15.97:0/3603444869 wait complete. 2024-03-26T19:09:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.301+0000 7f3cb7c46740 1 -- start start 2024-03-26T19:09:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.301+0000 7f3cb7c46740 1 Processor -- start 2024-03-26T19:09:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cade00640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3c940a3090 0x7f3c940a5550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0006fd0 tx=0x7f3ca0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f53906da0 0x560f53910da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f5390c900 0x560f53918320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560f5390f850 con 0x560f53906da0 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560f53827ac0 con 0x560f5390c900 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560f5390f9d0 con 0x560f53907790 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cb7c46740 10 client.0 _ll_register_callbacks cb 0x560f53827490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cad400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cad400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60826/0 (socket says 172.21.15.97:60826) 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.302+0000 7f3cad400640 1 -- 172.21.15.97:0/446551451 learned_addr learned my addr 172.21.15.97:0/446551451 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.303+0000 7f3cad400640 1 -- 172.21.15.97:0/446551451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f5390c900 msgr2=0x560f53918320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.303+0000 7f3cad400640 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f5390c900 0x560f53918320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.303+0000 7f3cad400640 1 -- 172.21.15.97:0/446551451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f53906da0 msgr2=0x560f53910da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.303+0000 7f3cad400640 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f53906da0 0x560f53910da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.303+0000 7f3cad400640 1 -- 172.21.15.97:0/446551451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f5364c620 con 0x560f53907790 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.303+0000 7f3cad400640 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539112e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c097b60 tx=0x7f3c9c00b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3ff33dbff514701 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.303+0000 7f3c8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.304+0000 7f3c8fe00640 1 -- 172.21.15.97:0/446551451 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c020040 con 0x560f53907790 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.304+0000 7f3c8fe00640 1 -- 172.21.15.97:0/446551451 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3c9c097d70 con 0x560f53907790 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.304+0000 7f3cb7c46740 1 -- 172.21.15.97:0/446551451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca8034cd0 con 0x560f53907790 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.304+0000 7f3c8fe00640 1 -- 172.21.15.97:0/446551451 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c9c009de0 con 0x560f53907790 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.304+0000 7f3cb7c46740 10 client.25993 fetch_fsmap learned FSMap version 27 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.304+0000 7f3cb7c46740 1 -- 172.21.15.97:0/446551451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560f5391b7b0 con 0x560f53907790 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.305+0000 7f3c8fe00640 1 -- 172.21.15.97:0/446551451 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca8034cd0 con 0x560f53907790 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.305+0000 7f3c8fe00640 1 -- 172.21.15.97:0/446551451 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3c9c09abe0 con 0x560f53907790 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.305+0000 7f3cb7c46740 10 client.25993 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:09:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.305+0000 7f3cb7c46740 10 client.25993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:09:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.305+0000 7f3cb7c46740 20 client.25993 populate_metadata read hostname 'smithi097' 2024-03-26T19:09:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.305+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560f5391ce60 0x560f5391f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.305+0000 7f3cb7c46740 4 client.25993 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:09:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.305+0000 7f3cb7c46740 1 -- 172.21.15.97:0/446551451 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f538793c0 con 0x560f5391ce60 2024-03-26T19:09:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.305+0000 7f3cade00640 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560f5391ce60 0x560f5391f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.306+0000 7f3cade00640 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560f5391ce60 0x560f5391f250 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231575199f208d28 server_cookie=760715fe3e4f5861 in_seq=0 out_seq=0 2024-03-26T19:09:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.306+0000 7f3c8fe00640 10 client.25993 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.307+0000 7f3c8fe00640 1 -- 172.21.15.97:0/446551451 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560f538793c0 con 0x560f5391ce60 2024-03-26T19:09:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.307+0000 7f3c8fe00640 10 client.25993 handle_command_reply: tid=0 2024-03-26T19:09:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.308+0000 7f3cb7c46740 1 client.25993 shutdown 2024-03-26T19:09:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.308+0000 7f3cb7c46740 1 -- 172.21.15.97:0/446551451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 msgr2=0x560f539112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.308+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539112e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c097b60 tx=0x7f3c9c00b660 comp rx=0 tx=0).stop 2024-03-26T19:09:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.309+0000 7f3cb7c46740 1 -- 172.21.15.97:0/446551451 shutdown_connections 2024-03-26T19:09:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.309+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560f5391ce60 0x560f5391f250 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.309+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f5390c900 0x560f53918320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.309+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539112e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.309+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/446551451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f53906da0 0x560f53910da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.309+0000 7f3cb7c46740 1 -- 172.21.15.97:0/446551451 >> 172.21.15.97:0/446551451 conn(0x560f53902310 msgr2=0x560f53901540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.309+0000 7f3c8fe00640 0 client.25993 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.309+0000 7f3cb7c46740 1 -- 172.21.15.97:0/446551451 shutdown_connections 2024-03-26T19:09:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.310+0000 7f3cb7c46740 1 -- 172.21.15.97:0/446551451 wait complete. 2024-03-26T19:09:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.310+0000 7f3cb7c46740 20 client.25993 trim_cache size 0 max 16384 2024-03-26T19:09:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.341+0000 7f3cb7c46740 1 Processor -- start 2024-03-26T19:09:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.341+0000 7f3cb7c46740 1 -- start start 2024-03-26T19:09:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.341+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 0x560f5390be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.341+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.341+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 0x560f53900cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.341+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01af340 con 0x560f5390c900 2024-03-26T19:09:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.341+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01b0e80 con 0x560f53906da0 2024-03-26T19:09:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.341+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01afa10 con 0x560f53907790 2024-03-26T19:09:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.342+0000 7f3cade00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 0x560f5390be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.342+0000 7f3cade00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 0x560f5390be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60976/0 (socket says 172.21.15.97:60976) 2024-03-26T19:09:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.342+0000 7f3cad400640 1 --2- 172.21.15.97:0/1594933157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.342+0000 7f3cade00640 1 -- 172.21.15.97:0/1594933157 learned_addr learned my addr 172.21.15.97:0/1594933157 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.343+0000 7f3cae800640 1 --2- 172.21.15.97:0/1594933157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 0x560f53900cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.343+0000 7f3cad400640 1 -- 172.21.15.97:0/1594933157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 msgr2=0x560f5390be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.343+0000 7f3cad400640 1 --2- 172.21.15.97:0/1594933157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 0x560f5390be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.343+0000 7f3cad400640 1 -- 172.21.15.97:0/1594933157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 msgr2=0x560f53900cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.343+0000 7f3cad400640 1 --2- 172.21.15.97:0/1594933157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 0x560f53900cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.343+0000 7f3cad400640 1 -- 172.21.15.97:0/1594933157 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5391f830 con 0x560f53907790 2024-03-26T19:09:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.344+0000 7f3cad400640 1 --2- 172.21.15.97:0/1594933157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539182e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005fd0 tx=0x7f3c9c09a7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7de6ab5370ccd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.344+0000 7f3c9aa00640 1 -- 172.21.15.97:0/1594933157 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c020070 con 0x560f53907790 2024-03-26T19:09:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.344+0000 7f3c9aa00640 1 -- 172.21.15.97:0/1594933157 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c9c0ba240 con 0x560f53907790 2024-03-26T19:09:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.344+0000 7f3c9aa00640 1 -- 172.21.15.97:0/1594933157 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c09ab50 con 0x560f53907790 2024-03-26T19:09:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.345+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1594933157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 msgr2=0x560f539182e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.345+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/1594933157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539182e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005fd0 tx=0x7f3c9c09a7e0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.346+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1594933157 shutdown_connections 2024-03-26T19:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.346+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/1594933157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 0x560f53900cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.346+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/1594933157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f539182e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.346+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/1594933157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 0x560f5390be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.346+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1594933157 >> 172.21.15.97:0/1594933157 conn(0x560f53902310 msgr2=0x560f53905e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.346+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1594933157 shutdown_connections 2024-03-26T19:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.346+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1594933157 wait complete. 2024-03-26T19:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.346+0000 7f3cb7c46740 1 -- start start 2024-03-26T19:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.347+0000 7f3cb7c46740 1 Processor -- start 2024-03-26T19:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.347+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 0x560f53916100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.347+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f53916640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.347+0000 7f3cb7c46740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 0x560f53917570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.347+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01b07b0 con 0x560f5390c900 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.347+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01b00e0 con 0x560f53906da0 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.347+0000 7f3cb7c46740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01ad320 con 0x560f53907790 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.347+0000 7f3cb7c46740 10 client.0 _ll_register_callbacks cb 0x7f3c9c09b5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cade00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 0x560f53916100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cad400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f53916640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cade00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 0x560f53916100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60992/0 (socket says 172.21.15.97:60992) 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cade00640 1 -- 172.21.15.97:0/1519796843 learned_addr learned my addr 172.21.15.97:0/1519796843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cae800640 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 0x560f53917570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cad400640 1 -- 172.21.15.97:0/1519796843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 msgr2=0x560f53916100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cad400640 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 0x560f53916100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cad400640 1 -- 172.21.15.97:0/1519796843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 msgr2=0x560f53917570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cad400640 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 0x560f53917570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cad400640 1 -- 172.21.15.97:0/1519796843 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f53917b60 con 0x560f53907790 2024-03-26T19:09:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.348+0000 7f3cae800640 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 0x560f53917570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:09:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.349+0000 7f3cad400640 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f53916640 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0999b0 tx=0x7f3c9c00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5549fcdc03c909ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.349+0000 7f3c8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:09:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.349+0000 7f3c8fe00640 1 -- 172.21.15.97:0/1519796843 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c020070 con 0x560f53907790 2024-03-26T19:09:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.349+0000 7f3c8fe00640 1 -- 172.21.15.97:0/1519796843 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3c9c097660 con 0x560f53907790 2024-03-26T19:09:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.349+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1519796843 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c9c0ba240 con 0x560f53907790 2024-03-26T19:09:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.349+0000 7f3c8fe00640 1 -- 172.21.15.97:0/1519796843 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c9c09a8a0 con 0x560f53907790 2024-03-26T19:09:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.349+0000 7f3cb7c46740 10 client.25999 fetch_fsmap learned FSMap version 27 2024-03-26T19:09:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.349+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1519796843 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560f539947f0 con 0x560f53907790 2024-03-26T19:09:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.350+0000 7f3c8fe00640 1 -- 172.21.15.97:0/1519796843 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c9c0ba240 con 0x560f53907790 2024-03-26T19:09:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.350+0000 7f3c8fe00640 1 -- 172.21.15.97:0/1519796843 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3c9c09ca30 con 0x560f53907790 2024-03-26T19:09:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.351+0000 7f3cb7c46740 10 client.25999 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:09:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.351+0000 7f3cb7c46740 10 client.25999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:09:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.351+0000 7f3cb7c46740 20 client.25999 populate_metadata read hostname 'smithi097' 2024-03-26T19:09:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.351+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560f539950d0 0x560f539974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.351+0000 7f3cb7c46740 4 client.25999 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:09:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.351+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1519796843 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f53873710 con 0x560f539950d0 2024-03-26T19:09:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.352+0000 7f3cade00640 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560f539950d0 0x560f539974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.353+0000 7f3cade00640 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560f539950d0 0x560f539974c0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfedd092cc7ce944 server_cookie=f9667f180aadd6e5 in_seq=0 out_seq=0 2024-03-26T19:09:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.353+0000 7f3c8fe00640 10 client.25999 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.475+0000 7f3c8fe00640 1 -- 172.21.15.97:0/1519796843 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f53873710 con 0x560f539950d0 2024-03-26T19:09:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.475+0000 7f3c8fe00640 10 client.25999 handle_command_reply: tid=0 2024-03-26T19:09:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.475+0000 7f3cb7c46740 1 client.25999 shutdown 2024-03-26T19:09:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.476+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1519796843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 msgr2=0x560f53916640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.476+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f53916640 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0999b0 tx=0x7f3c9c00bf60 comp rx=0 tx=0).stop 2024-03-26T19:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.476+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1519796843 shutdown_connections 2024-03-26T19:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.476+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560f539950d0 0x560f539974c0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.476+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f5390c900 0x560f53917570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.476+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f53907790 0x560f53916640 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.476+0000 7f3cb7c46740 1 --2- 172.21.15.97:0/1519796843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f53906da0 0x560f53916100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.476+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1519796843 >> 172.21.15.97:0/1519796843 conn(0x560f53902310 msgr2=0x560f53908230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.476+0000 7f3c8fe00640 0 client.25999 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.476+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1519796843 shutdown_connections 2024-03-26T19:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.477+0000 7f3cb7c46740 1 -- 172.21.15.97:0/1519796843 wait complete. 2024-03-26T19:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.477+0000 7f3cb7c46740 20 client.25999 trim_cache size 0 max 16384 2024-03-26T19:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.478+0000 7f3cb5000640 1 -- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3c940a3090 msgr2=0x7f3c940a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.478+0000 7f3cb5000640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3c940a3090 0x7f3c940a5550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0006fd0 tx=0x7f3ca0008040 comp rx=0 tx=0).stop 2024-03-26T19:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.478+0000 7f3cb5000640 1 -- 172.21.15.97:0/1488080542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cb00758a0 msgr2=0x7f3cb01051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.478+0000 7f3cb5000640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cb00758a0 0x7f3cb01051b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00d950 tx=0x7f3c9c00de10 comp rx=0 tx=0).stop 2024-03-26T19:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.478+0000 7f3cb5000640 1 -- 172.21.15.97:0/1488080542 shutdown_connections 2024-03-26T19:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.478+0000 7f3cb5000640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3c940a3090 0x7f3c940a5550 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.478+0000 7f3cb5000640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0109780 0x7f3cb01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.479+0000 7f3cb5000640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cb00758a0 0x7f3cb01051b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.479+0000 7f3cb5000640 1 --2- 172.21.15.97:0/1488080542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb00742c0 0x7f3cb0104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.479+0000 7f3cb5000640 1 -- 172.21.15.97:0/1488080542 >> 172.21.15.97:0/1488080542 conn(0x7f3cb00fce80 msgr2=0x7f3cb0105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.480+0000 7f3cb5000640 1 -- 172.21.15.97:0/1488080542 shutdown_connections 2024-03-26T19:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:09.480+0000 7f3cb5000640 1 -- 172.21.15.97:0/1488080542 wait complete. 2024-03-26T19:09:10.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 585.0 s, 1416.8 tps, lat 70.676 ms stddev 101.656 2024-03-26T19:09:10.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:10 smithi144 ceph-mon[36053]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.2 MiB/s wr, 616 op/s 2024-03-26T19:09:10.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:10 smithi105 ceph-mon[36270]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.2 MiB/s wr, 616 op/s 2024-03-26T19:09:10.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:10 smithi097 ceph-mon[31955]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.2 MiB/s wr, 616 op/s 2024-03-26T19:09:12.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:12 smithi144 ceph-mon[36053]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 714 op/s 2024-03-26T19:09:12.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:12 smithi105 ceph-mon[36270]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 714 op/s 2024-03-26T19:09:12.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:12 smithi097 ceph-mon[31955]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 714 op/s 2024-03-26T19:09:14.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:14 smithi144 ceph-mon[36053]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 MiB/s wr, 583 op/s 2024-03-26T19:09:14.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:14 smithi105 ceph-mon[36270]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 MiB/s wr, 583 op/s 2024-03-26T19:09:14.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:14 smithi097 ceph-mon[31955]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 MiB/s wr, 583 op/s 2024-03-26T19:09:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 590.0 s, 1436.8 tps, lat 70.035 ms stddev 87.538 2024-03-26T19:09:16.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:16 smithi105 ceph-mon[36270]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 722 op/s 2024-03-26T19:09:16.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:16 smithi097 ceph-mon[31955]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 722 op/s 2024-03-26T19:09:16.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:16 smithi144 ceph-mon[36053]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 722 op/s 2024-03-26T19:09:18.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:18 smithi105 ceph-mon[36270]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.6 MiB/s wr, 580 op/s 2024-03-26T19:09:18.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:18 smithi097 ceph-mon[31955]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.6 MiB/s wr, 580 op/s 2024-03-26T19:09:18.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:18 smithi144 ceph-mon[36053]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.6 MiB/s wr, 580 op/s 2024-03-26T19:09:20.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 595.0 s, 2029.6 tps, lat 48.685 ms stddev 54.985 2024-03-26T19:09:20.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:20 smithi105 ceph-mon[36270]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.2 MiB/s wr, 630 op/s 2024-03-26T19:09:20.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:20 smithi097 ceph-mon[31955]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.2 MiB/s wr, 630 op/s 2024-03-26T19:09:20.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:20 smithi144 ceph-mon[36053]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.2 MiB/s wr, 630 op/s 2024-03-26T19:09:21.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:21 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:09:21.843 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:21 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:09:21.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:21 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.678+0000 7f4c43600640 1 -- 172.21.15.97:0/2704543794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c3c1088c0 msgr2=0x7f4c3c10acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.678+0000 7f4c43600640 1 --2- 172.21.15.97:0/2704543794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c3c1088c0 0x7f4c3c10acf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009860 tx=0x7f4c2802bb70 comp rx=0 tx=0).stop 2024-03-26T19:09:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.678+0000 7f4c43600640 1 -- 172.21.15.97:0/2704543794 shutdown_connections 2024-03-26T19:09:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.678+0000 7f4c43600640 1 --2- 172.21.15.97:0/2704543794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c3c10dbe0 0x7f4c3c10e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.678+0000 7f4c43600640 1 --2- 172.21.15.97:0/2704543794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c3c10b230 0x7f4c3c10d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.678+0000 7f4c43600640 1 --2- 172.21.15.97:0/2704543794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c3c1088c0 0x7f4c3c10acf0 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009860 tx=0x7f4c2802bb70 comp rx=0 tx=0).stop 2024-03-26T19:09:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.678+0000 7f4c43600640 1 -- 172.21.15.97:0/2704543794 >> 172.21.15.97:0/2704543794 conn(0x7f4c3c06e900 msgr2=0x7f4c3c070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.680+0000 7f4c43600640 1 -- 172.21.15.97:0/2704543794 shutdown_connections 2024-03-26T19:09:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.680+0000 7f4c43600640 1 -- 172.21.15.97:0/2704543794 wait complete. 2024-03-26T19:09:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.684+0000 7f4c43600640 1 Processor -- start 2024-03-26T19:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.685+0000 7f4c43600640 1 -- start start 2024-03-26T19:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.687+0000 7f4c43600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c3c10b230 0x7f4c3c19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.687+0000 7f4c43600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c3c10dbe0 0x7f4c3c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.687+0000 7f4c43600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c3c1a38a0 0x7f4c3c1a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.687+0000 7f4c43600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c111b40 con 0x7f4c3c10b230 2024-03-26T19:09:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.687+0000 7f4c43600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1119c0 con 0x7f4c3c10dbe0 2024-03-26T19:09:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.687+0000 7f4c43600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c111cc0 con 0x7f4c3c1a38a0 2024-03-26T19:09:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.687+0000 7f4c40c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c3c10b230 0x7f4c3c19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.688+0000 7f4c40c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c3c10b230 0x7f4c3c19e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43178/0 (socket says 172.21.15.97:43178) 2024-03-26T19:09:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.688+0000 7f4c40c00640 1 -- 172.21.15.97:0/1168851789 learned_addr learned my addr 172.21.15.97:0/1168851789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.688+0000 7f4c41600640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c3c1a38a0 0x7f4c3c1a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.688+0000 7f4c3be00640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c3c10dbe0 0x7f4c3c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.688+0000 7f4c40c00640 1 -- 172.21.15.97:0/1168851789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c3c1a38a0 msgr2=0x7f4c3c1a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.688+0000 7f4c40c00640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c3c1a38a0 0x7f4c3c1a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.688+0000 7f4c40c00640 1 -- 172.21.15.97:0/1168851789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c3c10dbe0 msgr2=0x7f4c3c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.688+0000 7f4c40c00640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c3c10dbe0 0x7f4c3c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.688+0000 7f4c40c00640 1 -- 172.21.15.97:0/1168851789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c1a6300 con 0x7f4c3c10b230 2024-03-26T19:09:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.688+0000 7f4c41600640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c3c1a38a0 0x7f4c3c1a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:09:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.689+0000 7f4c40c00640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c3c10b230 0x7f4c3c19e300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009780 tx=0x7f4c280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.690+0000 7f4c39600640 1 -- 172.21.15.97:0/1168851789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28041020 con 0x7f4c3c10b230 2024-03-26T19:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.690+0000 7f4c39600640 1 -- 172.21.15.97:0/1168851789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c280035f0 con 0x7f4c3c10b230 2024-03-26T19:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.690+0000 7f4c39600640 1 -- 172.21.15.97:0/1168851789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28039040 con 0x7f4c3c10b230 2024-03-26T19:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.691+0000 7f4c43600640 1 -- 172.21.15.97:0/1168851789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3c1a6590 con 0x7f4c3c10b230 2024-03-26T19:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.691+0000 7f4c43600640 1 -- 172.21.15.97:0/1168851789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3c1a6a30 con 0x7f4c3c10b230 2024-03-26T19:09:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.697+0000 7f4c39600640 1 -- 172.21.15.97:0/1168851789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f4c28044020 con 0x7f4c3c10b230 2024-03-26T19:09:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.699+0000 7f4c39600640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4c200a3130 0x7f4c200a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.699+0000 7f4c39600640 1 -- 172.21.15.97:0/1168851789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f4c280eff10 con 0x7f4c3c10b230 2024-03-26T19:09:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.699+0000 7f4c463e1740 1 Processor -- start 2024-03-26T19:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.701+0000 7f4c463e1740 1 -- start start 2024-03-26T19:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.701+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374446da0 0x5603744471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.701+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560374447790 0x56037444c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.701+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 0x56037444ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.701+0000 7f4c463e1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5603743abe80 con 0x560374447790 2024-03-26T19:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.701+0000 7f4c463e1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560374354660 con 0x56037444c900 2024-03-26T19:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.701+0000 7f4c463e1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5603743a7e30 con 0x560374446da0 2024-03-26T19:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.702+0000 7f4c40c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374446da0 0x5603744471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.702+0000 7f4c40c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374446da0 0x5603744471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:57176/0 (socket says 172.21.15.97:57176) 2024-03-26T19:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.702+0000 7f4c40c00640 1 -- 172.21.15.97:0/625923084 learned_addr learned my addr 172.21.15.97:0/625923084 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.702+0000 7f4c41600640 1 --2- 172.21.15.97:0/625923084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 0x56037444ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.703+0000 7f4c3be00640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4c200a3130 0x7f4c200a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.703+0000 7f4c41600640 1 -- 172.21.15.97:0/625923084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374446da0 msgr2=0x5603744471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.703+0000 7f4c41600640 1 --2- 172.21.15.97:0/625923084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374446da0 0x5603744471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.703+0000 7f4c41600640 1 -- 172.21.15.97:0/625923084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560374447790 msgr2=0x56037444c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.703+0000 7f4c41600640 1 --2- 172.21.15.97:0/625923084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560374447790 0x56037444c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.703+0000 7f4c41600640 1 -- 172.21.15.97:0/625923084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603740c0210 con 0x56037444c900 2024-03-26T19:09:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.704+0000 7f4c41600640 1 --2- 172.21.15.97:0/625923084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 0x56037444ed90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009f50 tx=0x7f4c2c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c349d2ce3a39b25a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.704+0000 7f4c31c00640 1 -- 172.21.15.97:0/625923084 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c01e070 con 0x56037444c900 2024-03-26T19:09:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.705+0000 7f4c31c00640 1 -- 172.21.15.97:0/625923084 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2c004310 con 0x56037444c900 2024-03-26T19:09:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.705+0000 7f4c31c00640 1 -- 172.21.15.97:0/625923084 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00f9e0 con 0x56037444c900 2024-03-26T19:09:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.706+0000 7f4c463e1740 1 -- 172.21.15.97:0/625923084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 msgr2=0x56037444ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.706+0000 7f4c463e1740 1 --2- 172.21.15.97:0/625923084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 0x56037444ed90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009f50 tx=0x7f4c2c0118d0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.706+0000 7f4c463e1740 1 -- 172.21.15.97:0/625923084 shutdown_connections 2024-03-26T19:09:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.707+0000 7f4c463e1740 1 --2- 172.21.15.97:0/625923084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 0x56037444ed90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.707+0000 7f4c463e1740 1 --2- 172.21.15.97:0/625923084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560374447790 0x56037444c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.707+0000 7f4c463e1740 1 --2- 172.21.15.97:0/625923084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374446da0 0x5603744471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.707+0000 7f4c463e1740 1 -- 172.21.15.97:0/625923084 >> 172.21.15.97:0/625923084 conn(0x560374442310 msgr2=0x560374442710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.707+0000 7f4c463e1740 1 -- 172.21.15.97:0/625923084 shutdown_connections 2024-03-26T19:09:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.708+0000 7f4c463e1740 1 -- 172.21.15.97:0/625923084 wait complete. 2024-03-26T19:09:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.708+0000 7f4c463e1740 1 -- start start 2024-03-26T19:09:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.708+0000 7f4c463e1740 1 Processor -- start 2024-03-26T19:09:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.709+0000 7f4c3be00640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4c200a3130 0x7f4c200a55f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x560374447b90 tx=0x7f4c3401aa00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.710+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 0x560374450db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.710+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 0x5603744512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.710+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 0x560374458330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.710+0000 7f4c463e1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56037444f8b0 con 0x56037444c900 2024-03-26T19:09:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.710+0000 7f4c463e1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560374367ac0 con 0x560374446da0 2024-03-26T19:09:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.710+0000 7f4c463e1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56037444fa30 con 0x560374447790 2024-03-26T19:09:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.710+0000 7f4c463e1740 10 client.0 _ll_register_callbacks cb 0x560374367490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:09:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.710+0000 7f4c40c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 0x560374450db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.710+0000 7f4c40c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 0x560374450db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34082/0 (socket says 172.21.15.97:34082) 2024-03-26T19:09:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.710+0000 7f4c40c00640 1 -- 172.21.15.97:0/3768148655 learned_addr learned my addr 172.21.15.97:0/3768148655 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.711+0000 7f4c41600640 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 0x560374458330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.711+0000 7f4c3be00640 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 0x5603744512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.713+0000 7f4c41600640 1 -- 172.21.15.97:0/3768148655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 msgr2=0x5603744512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.713+0000 7f4c41600640 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 0x5603744512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.713+0000 7f4c41600640 1 -- 172.21.15.97:0/3768148655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 msgr2=0x560374450db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.713+0000 7f4c41600640 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 0x560374450db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.713+0000 7f4c41600640 1 -- 172.21.15.97:0/3768148655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037418c620 con 0x56037444c900 2024-03-26T19:09:22.715 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:22 smithi105 ceph-mon[36270]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.3 MiB/s wr, 715 op/s 2024-03-26T19:09:22.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:22 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:22 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:22 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:22 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.716+0000 7f4c41600640 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 0x560374458330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c011dd0 tx=0x7f4c2c0056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58faa34ffa0686fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.716+0000 7f4c26000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T19:09:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.719+0000 7f4c26000640 1 -- 172.21.15.97:0/3768148655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c027020 con 0x56037444c900 2024-03-26T19:09:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.719+0000 7f4c463e1740 1 -- 172.21.15.97:0/3768148655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2c004310 con 0x56037444c900 2024-03-26T19:09:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.719+0000 7f4c26000640 1 -- 172.21.15.97:0/3768148655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f4c2c004110 con 0x56037444c900 2024-03-26T19:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.721+0000 7f4c26000640 1 -- 172.21.15.97:0/3768148655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c2c027020 con 0x56037444c900 2024-03-26T19:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.721+0000 7f4c463e1740 10 client.15669 fetch_fsmap learned FSMap version 27 2024-03-26T19:09:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.721+0000 7f4c463e1740 1 -- 172.21.15.97:0/3768148655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56037445b6f0 con 0x56037444c900 2024-03-26T19:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.723+0000 7f4c26000640 1 -- 172.21.15.97:0/3768148655 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2c004310 con 0x56037444c900 2024-03-26T19:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.724+0000 7f4c26000640 1 -- 172.21.15.97:0/3768148655 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4c2c00f9c0 con 0x56037444c900 2024-03-26T19:09:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.725+0000 7f4c463e1740 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:09:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.725+0000 7f4c463e1740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:09:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.725+0000 7f4c463e1740 20 client.15669 populate_metadata read hostname 'smithi097' 2024-03-26T19:09:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.725+0000 7f4c463e1740 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56037445caa0 0x56037445ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.725+0000 7f4c463e1740 4 client.15669 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:09:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.725+0000 7f4c463e1740 1 -- 172.21.15.97:0/3768148655 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603743b93c0 con 0x56037445caa0 2024-03-26T19:09:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.725+0000 7f4c40c00640 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56037445caa0 0x56037445ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.726+0000 7f4c26000640 10 client.15669 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.726+0000 7f4c40c00640 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56037445caa0 0x56037445ee90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2717b3e46e1b1dc server_cookie=14cc02ca6869559 in_seq=0 out_seq=0 2024-03-26T19:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.726+0000 7f4c26000640 1 -- 172.21.15.97:0/3768148655 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5603743b93c0 con 0x56037445caa0 2024-03-26T19:09:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.726+0000 7f4c26000640 10 client.15669 handle_command_reply: tid=0 2024-03-26T19:09:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.726+0000 7f4c463e1740 1 client.15669 shutdown 2024-03-26T19:09:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.730+0000 7f4c463e1740 1 -- 172.21.15.97:0/3768148655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 msgr2=0x560374458330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.730+0000 7f4c463e1740 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 0x560374458330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c011dd0 tx=0x7f4c2c0056c0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.732+0000 7f4c463e1740 1 -- 172.21.15.97:0/3768148655 shutdown_connections 2024-03-26T19:09:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.732+0000 7f4c463e1740 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56037445caa0 0x56037445ee90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.732+0000 7f4c463e1740 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 0x560374458330 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.732+0000 7f4c463e1740 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 0x5603744512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.732+0000 7f4c463e1740 1 --2- 172.21.15.97:0/3768148655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 0x560374450db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.732+0000 7f4c26000640 0 client.15669 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.732+0000 7f4c463e1740 1 -- 172.21.15.97:0/3768148655 >> 172.21.15.97:0/3768148655 conn(0x560374442310 msgr2=0x560374441540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.732+0000 7f4c463e1740 1 -- 172.21.15.97:0/3768148655 shutdown_connections 2024-03-26T19:09:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.733+0000 7f4c463e1740 1 -- 172.21.15.97:0/3768148655 wait complete. 2024-03-26T19:09:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.733+0000 7f4c463e1740 20 client.15669 trim_cache size 0 max 16384 2024-03-26T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.786+0000 7f4c463e1740 1 Processor -- start 2024-03-26T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.786+0000 7f4c463e1740 1 -- start start 2024-03-26T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.787+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 0x560374448940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.787+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 0x5603744582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.787+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 0x560374463790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.787+0000 7f4c3be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 0x5603744582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.787+0000 7f4c40c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 0x560374448940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.787+0000 7f4c3be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 0x5603744582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:57198/0 (socket says 172.21.15.97:57198) 2024-03-26T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.788+0000 7f4c40c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 0x560374448940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34092/0 (socket says 172.21.15.97:34092) 2024-03-26T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.788+0000 7f4c41600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 0x560374463790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.788+0000 7f4c3be00640 1 -- 172.21.15.97:0/2589814324 learned_addr learned my addr 172.21.15.97:0/2589814324 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.788+0000 7f4c463e1740 1 -- 172.21.15.97:0/2589814324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1afa70 con 0x56037444c900 2024-03-26T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.788+0000 7f4c463e1740 1 -- 172.21.15.97:0/2589814324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1b15b0 con 0x560374446da0 2024-03-26T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.788+0000 7f4c463e1740 1 -- 172.21.15.97:0/2589814324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1b0140 con 0x560374447790 2024-03-26T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.788+0000 7f4c3be00640 1 -- 172.21.15.97:0/2589814324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 msgr2=0x560374448940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.788+0000 7f4c3be00640 1 --2- 172.21.15.97:0/2589814324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 0x560374448940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.788+0000 7f4c3be00640 1 -- 172.21.15.97:0/2589814324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 msgr2=0x560374463790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.788+0000 7f4c3be00640 1 --2- 172.21.15.97:0/2589814324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 0x560374463790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.789+0000 7f4c3be00640 1 -- 172.21.15.97:0/2589814324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037444ba60 con 0x560374447790 2024-03-26T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.790+0000 7f4c3be00640 1 --2- 172.21.15.97:0/2589814324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 0x5603744582f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4c3402a6a0 tx=0x7f4c34004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97834518784fe98a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.791+0000 7f4c31200640 1 -- 172.21.15.97:0/2589814324 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c340355f0 con 0x560374447790 2024-03-26T19:09:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.791+0000 7f4c31200640 1 -- 172.21.15.97:0/2589814324 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c3403a070 con 0x560374447790 2024-03-26T19:09:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.791+0000 7f4c31200640 1 -- 172.21.15.97:0/2589814324 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34029ae0 con 0x560374447790 2024-03-26T19:09:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.792+0000 7f4c463e1740 1 -- 172.21.15.97:0/2589814324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 msgr2=0x5603744582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.792+0000 7f4c463e1740 1 --2- 172.21.15.97:0/2589814324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 0x5603744582f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4c3402a6a0 tx=0x7f4c34004ab0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.793+0000 7f4c463e1740 1 -- 172.21.15.97:0/2589814324 shutdown_connections 2024-03-26T19:09:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.793+0000 7f4c463e1740 1 --2- 172.21.15.97:0/2589814324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56037444c900 0x560374463790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.793+0000 7f4c463e1740 1 --2- 172.21.15.97:0/2589814324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374447790 0x5603744582f0 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4c3402a6a0 tx=0x7f4c34004ab0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.793+0000 7f4c463e1740 1 --2- 172.21.15.97:0/2589814324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560374446da0 0x560374448940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.793+0000 7f4c463e1740 1 -- 172.21.15.97:0/2589814324 >> 172.21.15.97:0/2589814324 conn(0x560374442310 msgr2=0x560374448230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.793+0000 7f4c463e1740 1 -- 172.21.15.97:0/2589814324 shutdown_connections 2024-03-26T19:09:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.793+0000 7f4c463e1740 1 -- 172.21.15.97:0/2589814324 wait complete. 2024-03-26T19:09:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.794+0000 7f4c463e1740 1 -- start start 2024-03-26T19:09:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.795+0000 7f4c463e1740 1 Processor -- start 2024-03-26T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.795+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560374446da0 0x560374456020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.795+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 0x560374456560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.795+0000 7f4c463e1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374457520 0x5603744579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.795+0000 7f4c463e1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1b0ee0 con 0x560374446da0 2024-03-26T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.795+0000 7f4c463e1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1b0810 con 0x56037444c900 2024-03-26T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.795+0000 7f4c463e1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1ad9f0 con 0x560374457520 2024-03-26T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.795+0000 7f4c463e1740 10 client.0 _ll_register_callbacks cb 0x7f4c2c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.796+0000 7f4c3be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 0x560374456560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.796+0000 7f4c40c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560374446da0 0x560374456020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.796+0000 7f4c41600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374457520 0x5603744579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.796+0000 7f4c40c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560374446da0 0x560374456020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43224/0 (socket says 172.21.15.97:43224) 2024-03-26T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.796+0000 7f4c40c00640 1 -- 172.21.15.97:0/2529092014 learned_addr learned my addr 172.21.15.97:0/2529092014 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.796+0000 7f4c41600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374457520 0x5603744579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:57204/0 (socket says 172.21.15.97:57204) 2024-03-26T19:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.796+0000 7f4c3be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 0x560374456560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34100/0 (socket says 172.21.15.97:34100) 2024-03-26T19:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.798+0000 7f4c41600640 1 -- 172.21.15.97:0/2529092014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 msgr2=0x560374456560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.798+0000 7f4c41600640 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 0x560374456560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.798+0000 7f4c41600640 1 -- 172.21.15.97:0/2529092014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560374446da0 msgr2=0x560374456020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.798+0000 7f4c41600640 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560374446da0 0x560374456020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.798+0000 7f4c41600640 1 -- 172.21.15.97:0/2529092014 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560374457ee0 con 0x560374457520 2024-03-26T19:09:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.799+0000 7f4c41600640 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374457520 0x5603744579a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00fdc0 tx=0x7f4c2c0047a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa36f99875ce248 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.799+0000 7f4c26000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:09:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.799+0000 7f4c26000640 1 -- 172.21.15.97:0/2529092014 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c027020 con 0x560374457520 2024-03-26T19:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.800+0000 7f4c26000640 1 -- 172.21.15.97:0/2529092014 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f4c2c004110 con 0x560374457520 2024-03-26T19:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.800+0000 7f4c463e1740 1 -- 172.21.15.97:0/2529092014 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c3403a070 con 0x560374457520 2024-03-26T19:09:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.805+0000 7f4c26000640 1 -- 172.21.15.97:0/2529092014 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c2c027020 con 0x560374457520 2024-03-26T19:09:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.805+0000 7f4c463e1740 10 client.26017 fetch_fsmap learned FSMap version 27 2024-03-26T19:09:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.805+0000 7f4c463e1740 1 -- 172.21.15.97:0/2529092014 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603744c3740 con 0x560374457520 2024-03-26T19:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.805+0000 7f4c26000640 1 -- 172.21.15.97:0/2529092014 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2c03a070 con 0x560374457520 2024-03-26T19:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.809+0000 7f4c26000640 1 -- 172.21.15.97:0/2529092014 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4c2c00f480 con 0x560374457520 2024-03-26T19:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.809+0000 7f4c463e1740 10 client.26017 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.809+0000 7f4c463e1740 10 client.26017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.809+0000 7f4c463e1740 20 client.26017 populate_metadata read hostname 'smithi097' 2024-03-26T19:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.809+0000 7f4c463e1740 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5603744c40b0 0x5603744c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.809+0000 7f4c463e1740 4 client.26017 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.809+0000 7f4c463e1740 1 -- 172.21.15.97:0/2529092014 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4c3403a070 con 0x5603744c40b0 2024-03-26T19:09:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.810+0000 7f4c40c00640 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5603744c40b0 0x5603744c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.810+0000 7f4c40c00640 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5603744c40b0 0x5603744c64d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c00339902766a912 server_cookie=6c538fbf92b84337 in_seq=0 out_seq=0 2024-03-26T19:09:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.810+0000 7f4c26000640 10 client.26017 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:22.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:22 smithi097 ceph-mon[31955]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.3 MiB/s wr, 715 op/s 2024-03-26T19:09:22.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:22 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:22 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:22 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:22 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:22 smithi144 ceph-mon[36053]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.3 MiB/s wr, 715 op/s 2024-03-26T19:09:22.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:22 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.918+0000 7f4c26000640 1 -- 172.21.15.97:0/2529092014 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4c3403a070 con 0x5603744c40b0 2024-03-26T19:09:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.918+0000 7f4c26000640 10 client.26017 handle_command_reply: tid=0 2024-03-26T19:09:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.918+0000 7f4c463e1740 1 client.26017 shutdown 2024-03-26T19:09:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.919+0000 7f4c463e1740 1 -- 172.21.15.97:0/2529092014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374457520 msgr2=0x5603744579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.920+0000 7f4c463e1740 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374457520 0x5603744579a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00fdc0 tx=0x7f4c2c0047a0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.920+0000 7f4c463e1740 1 -- 172.21.15.97:0/2529092014 shutdown_connections 2024-03-26T19:09:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.920+0000 7f4c463e1740 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5603744c40b0 0x5603744c64d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.920+0000 7f4c463e1740 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560374457520 0x5603744579a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.920+0000 7f4c463e1740 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56037444c900 0x560374456560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.920+0000 7f4c463e1740 1 --2- 172.21.15.97:0/2529092014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560374446da0 0x560374456020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.920+0000 7f4c463e1740 1 -- 172.21.15.97:0/2529092014 >> 172.21.15.97:0/2529092014 conn(0x560374442310 msgr2=0x560374441540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.920+0000 7f4c26000640 0 client.26017 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.920+0000 7f4c463e1740 1 -- 172.21.15.97:0/2529092014 shutdown_connections 2024-03-26T19:09:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.920+0000 7f4c463e1740 1 -- 172.21.15.97:0/2529092014 wait complete. 2024-03-26T19:09:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.921+0000 7f4c463e1740 20 client.26017 trim_cache size 0 max 16384 2024-03-26T19:09:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:09:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:09:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:09:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 -- 172.21.15.97:0/1168851789 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4c200a3130 msgr2=0x7f4c200a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4c200a3130 0x7f4c200a55f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x560374447b90 tx=0x7f4c3401aa00 comp rx=0 tx=0).stop 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 -- 172.21.15.97:0/1168851789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c3c10b230 msgr2=0x7f4c3c19e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c3c10b230 0x7f4c3c19e300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009780 tx=0x7f4c280042f0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 -- 172.21.15.97:0/1168851789 shutdown_connections 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f4c200a3130 0x7f4c200a55f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c3c1a38a0 0x7f4c3c1a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c3c10dbe0 0x7f4c3c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 --2- 172.21.15.97:0/1168851789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c3c10b230 0x7f4c3c19e300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 -- 172.21.15.97:0/1168851789 >> 172.21.15.97:0/1168851789 conn(0x7f4c3c06e900 msgr2=0x7f4c3c109030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.922+0000 7f4c43600640 1 -- 172.21.15.97:0/1168851789 shutdown_connections 2024-03-26T19:09:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:22.923+0000 7f4c43600640 1 -- 172.21.15.97:0/1168851789 wait complete. 2024-03-26T19:09:24.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:23 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:24.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:23 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:24.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:23 smithi097 ceph-mon[31955]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-03-26T19:09:24.123 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:23 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:24.123 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:23 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:24.123 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:23 smithi105 ceph-mon[36270]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-03-26T19:09:24.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:23 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:24.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:23 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:24.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:23 smithi144 ceph-mon[36053]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-03-26T19:09:25.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 600.0 s, 1378.0 tps, lat 73.518 ms stddev 101.554 2024-03-26T19:09:25.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:25 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:25.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:25 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:25.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:25 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:25.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:25 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:25.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:25 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:25.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:25 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:26.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:26 smithi144 ceph-mon[36053]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 722 op/s 2024-03-26T19:09:26.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:26.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:26 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:26.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:26 smithi105 ceph-mon[36270]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 722 op/s 2024-03-26T19:09:26.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:26.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:26 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:26.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:26 smithi097 ceph-mon[31955]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 722 op/s 2024-03-26T19:09:26.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:26.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:26 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:28.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:28 smithi144 ceph-mon[36053]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 555 op/s 2024-03-26T19:09:28.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:09:28.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:09:28.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:28.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:28 smithi105 ceph-mon[36270]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 555 op/s 2024-03-26T19:09:28.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:09:28.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:09:28.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:28.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:28 smithi097 ceph-mon[31955]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 555 op/s 2024-03-26T19:09:28.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:09:28.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:09:28.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:09:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 605.0 s, 2006.0 tps, lat 49.670 ms stddev 54.611 2024-03-26T19:09:30.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:30 smithi105 ceph-mon[36270]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 649 op/s 2024-03-26T19:09:30.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:30 smithi097 ceph-mon[31955]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 649 op/s 2024-03-26T19:09:30.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:30 smithi144 ceph-mon[36053]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 649 op/s 2024-03-26T19:09:32.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:32 smithi097 ceph-mon[31955]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-03-26T19:09:32.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:32 smithi105 ceph-mon[36270]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-03-26T19:09:32.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:32 smithi144 ceph-mon[36053]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-03-26T19:09:34.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:34 smithi105 ceph-mon[36270]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-03-26T19:09:34.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:34 smithi097 ceph-mon[31955]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-03-26T19:09:34.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:34 smithi144 ceph-mon[36053]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-03-26T19:09:35.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 610.0 s, 2016.2 tps, lat 49.614 ms stddev 58.100 2024-03-26T19:09:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.111+0000 7f6aab200640 1 -- 172.21.15.97:0/3614387565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa4100940 msgr2=0x7f6aa4100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.111+0000 7f6aab200640 1 --2- 172.21.15.97:0/3614387565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa4100940 0x7f6aa4100da0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009900 tx=0x7f6a9402bc40 comp rx=0 tx=0).stop 2024-03-26T19:09:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.112+0000 7f6aab200640 1 -- 172.21.15.97:0/3614387565 shutdown_connections 2024-03-26T19:09:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.112+0000 7f6aab200640 1 --2- 172.21.15.97:0/3614387565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa41012e0 0x7f6aa4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.112+0000 7f6aab200640 1 --2- 172.21.15.97:0/3614387565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa4100940 0x7f6aa4100da0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.112+0000 7f6aab200640 1 --2- 172.21.15.97:0/3614387565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa40fffe0 0x7f6aa4100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.112+0000 7f6aab200640 1 -- 172.21.15.97:0/3614387565 >> 172.21.15.97:0/3614387565 conn(0x7f6aa40fbba0 msgr2=0x7f6aa40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.112+0000 7f6aab200640 1 -- 172.21.15.97:0/3614387565 shutdown_connections 2024-03-26T19:09:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.112+0000 7f6aab200640 1 -- 172.21.15.97:0/3614387565 wait complete. 2024-03-26T19:09:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.113+0000 7f6aab200640 1 Processor -- start 2024-03-26T19:09:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.114+0000 7f6aab200640 1 -- start start 2024-03-26T19:09:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.114+0000 7f6aab200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa40fffe0 0x7f6aa4199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.114+0000 7f6aab200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa4100940 0x7f6aa419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aab200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa41012e0 0x7f6aa41a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aab200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa410f5a0 con 0x7f6aa41012e0 2024-03-26T19:09:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aab200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa410f420 con 0x7f6aa40fffe0 2024-03-26T19:09:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aab200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa410f720 con 0x7f6aa4100940 2024-03-26T19:09:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aa9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa40fffe0 0x7f6aa4199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aa9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa40fffe0 0x7f6aa4199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:45920/0 (socket says 172.21.15.97:45920) 2024-03-26T19:09:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aa9e00640 1 -- 172.21.15.97:0/1178410959 learned_addr learned my addr 172.21.15.97:0/1178410959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aa9400640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa4100940 0x7f6aa419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aaa800640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa41012e0 0x7f6aa41a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aa9e00640 1 -- 172.21.15.97:0/1178410959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa4100940 msgr2=0x7f6aa419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa4100940 0x7f6aa419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aa9e00640 1 -- 172.21.15.97:0/1178410959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa41012e0 msgr2=0x7f6aa41a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa41012e0 0x7f6aa41a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.115+0000 7f6aa9e00640 1 -- 172.21.15.97:0/1178410959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa41a1840 con 0x7f6aa40fffe0 2024-03-26T19:09:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.116+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa40fffe0 0x7f6aa4199c30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800e9f0 tx=0x7f6a9800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.116+0000 7f6a9ea00640 1 -- 172.21.15.97:0/1178410959 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800cd80 con 0x7f6aa40fffe0 2024-03-26T19:09:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.116+0000 7f6a9ea00640 1 -- 172.21.15.97:0/1178410959 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a98010910 con 0x7f6aa40fffe0 2024-03-26T19:09:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.116+0000 7f6aab200640 1 -- 172.21.15.97:0/1178410959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa41a1b30 con 0x7f6aa40fffe0 2024-03-26T19:09:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.116+0000 7f6a9ea00640 1 -- 172.21.15.97:0/1178410959 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98018990 con 0x7f6aa40fffe0 2024-03-26T19:09:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.116+0000 7f6aab200640 1 -- 172.21.15.97:0/1178410959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa41a2080 con 0x7f6aa40fffe0 2024-03-26T19:09:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.120+0000 7f6aaf57d740 1 Processor -- start 2024-03-26T19:09:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.120+0000 7f6a9ea00640 1 -- 172.21.15.97:0/1178410959 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f6a98010ac0 con 0x7f6aa40fffe0 2024-03-26T19:09:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.120+0000 7f6aaf57d740 1 -- start start 2024-03-26T19:09:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.121+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82666b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.121+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab8266bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.121+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab8266c2d0 0x55ab8266e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.121+0000 7f6aaf57d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ab825c2e80 con 0x55ab82666770 2024-03-26T19:09:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.121+0000 7f6aaf57d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ab8256b660 con 0x55ab82667160 2024-03-26T19:09:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.121+0000 7f6aaf57d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ab825bee30 con 0x55ab8266c2d0 2024-03-26T19:09:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.122+0000 7f6aaa800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82666b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.122+0000 7f6aaa800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82666b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51154/0 (socket says 172.21.15.97:51154) 2024-03-26T19:09:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.122+0000 7f6aaa800640 1 -- 172.21.15.97:0/4263122960 learned_addr learned my addr 172.21.15.97:0/4263122960 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.122+0000 7f6aa9400640 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab8266c2d0 0x55ab8266e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.123+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab8266bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.123+0000 7f6aa9400640 1 -- 172.21.15.97:0/4263122960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 msgr2=0x55ab8266bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.123+0000 7f6aa9400640 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab8266bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.123+0000 7f6aa9400640 1 -- 172.21.15.97:0/4263122960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 msgr2=0x55ab82666b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.123+0000 7f6aa9400640 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82666b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.123+0000 7f6aa9400640 1 -- 172.21.15.97:0/4263122960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab822d7210 con 0x55ab8266c2d0 2024-03-26T19:09:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.124+0000 7f6aaa800640 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82666b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:09:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.124+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab8266bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:09:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.124+0000 7f6aa9400640 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab8266c2d0 0x55ab8266e760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a94036040 tx=0x7f6a94005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5706f85c391c801b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.124+0000 7f6a9ea00640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6a8c0a2fd0 0x7f6a8c0a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.125+0000 7f6a9ea00640 1 -- 172.21.15.97:0/1178410959 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6a980cf850 con 0x7f6aa40fffe0 2024-03-26T19:09:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.125+0000 7f6a93000640 1 -- 172.21.15.97:0/4263122960 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9402cb90 con 0x55ab8266c2d0 2024-03-26T19:09:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.125+0000 7f6a93000640 1 -- 172.21.15.97:0/4263122960 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9402cd30 con 0x55ab8266c2d0 2024-03-26T19:09:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.125+0000 7f6a93000640 1 -- 172.21.15.97:0/4263122960 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94004900 con 0x55ab8266c2d0 2024-03-26T19:09:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.125+0000 7f6aaa800640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6a8c0a2fd0 0x7f6a8c0a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.126+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4263122960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab8266c2d0 msgr2=0x55ab8266e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.126+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab8266c2d0 0x55ab8266e760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a94036040 tx=0x7f6a94005c00 comp rx=0 tx=0).stop 2024-03-26T19:09:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.126+0000 7f6aaa800640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6a8c0a2fd0 0x7f6a8c0a5490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ab826679c0 tx=0x7f6aa00097e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.126+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4263122960 shutdown_connections 2024-03-26T19:09:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.126+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab8266c2d0 0x55ab8266e760 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.126+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab8266bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.126+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/4263122960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82666b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.126+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4263122960 >> 172.21.15.97:0/4263122960 conn(0x55ab826594d0 msgr2=0x55ab826598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.127+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4263122960 shutdown_connections 2024-03-26T19:09:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.127+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4263122960 wait complete. 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.127+0000 7f6aaf57d740 1 -- start start 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.127+0000 7f6aaf57d740 1 Processor -- start 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82670740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab82670c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 0x55ab82677cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aaf57d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ab8266f220 con 0x55ab82666770 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aaf57d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ab8257eac0 con 0x55ab8266c2d0 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aaf57d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ab8266f3a0 con 0x55ab82667160 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aaf57d740 10 client.0 _ll_register_callbacks cb 0x55ab8257e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab82670c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab82670c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:48694/0 (socket says 172.21.15.97:48694) 2024-03-26T19:09:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aa9400640 1 -- 172.21.15.97:0/1509683522 learned_addr learned my addr 172.21.15.97:0/1509683522 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aaa800640 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 0x55ab82677cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82670740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aa9400640 1 -- 172.21.15.97:0/1509683522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 msgr2=0x55ab82677cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.128+0000 7f6aa9400640 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 0x55ab82677cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.129+0000 7f6aa9400640 1 -- 172.21.15.97:0/1509683522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 msgr2=0x55ab82670740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.129+0000 7f6aa9400640 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82670740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.129+0000 7f6aa9400640 1 -- 172.21.15.97:0/1509683522 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab823a3620 con 0x55ab82667160 2024-03-26T19:09:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.129+0000 7f6aaa800640 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 0x55ab82677cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:09:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.129+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82670740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:09:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.130+0000 7f6aa9400640 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab82670c80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6a940096c0 tx=0x7f6a94004550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be42a8609b711f64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.130+0000 7f6a87400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:09:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.131+0000 7f6a87400640 1 -- 172.21.15.97:0/1509683522 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9404c070 con 0x55ab82667160 2024-03-26T19:09:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.131+0000 7f6a87400640 1 -- 172.21.15.97:0/1509683522 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6a94003ac0 con 0x55ab82667160 2024-03-26T19:09:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.131+0000 7f6aaf57d740 1 -- 172.21.15.97:0/1509683522 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ab825d03c0 con 0x55ab82667160 2024-03-26T19:09:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.131+0000 7f6a87400640 1 -- 172.21.15.97:0/1509683522 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a94003cc0 con 0x55ab82667160 2024-03-26T19:09:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.131+0000 7f6aaf57d740 10 client.26029 fetch_fsmap learned FSMap version 27 2024-03-26T19:09:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.132+0000 7f6aaf57d740 1 -- 172.21.15.97:0/1509683522 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab8266f270 con 0x55ab82667160 2024-03-26T19:09:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.132+0000 7f6a87400640 1 -- 172.21.15.97:0/1509683522 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ab825d03c0 con 0x55ab82667160 2024-03-26T19:09:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.132+0000 7f6a87400640 1 -- 172.21.15.97:0/1509683522 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a940345e0 con 0x55ab82667160 2024-03-26T19:09:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.132+0000 7f6aaf57d740 10 client.26029 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:09:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.132+0000 7f6aaf57d740 10 client.26029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:09:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.132+0000 7f6aaf57d740 20 client.26029 populate_metadata read hostname 'smithi097' 2024-03-26T19:09:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.132+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ab8267bf70 0x55ab8267e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.132+0000 7f6aaf57d740 4 client.26029 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:09:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.132+0000 7f6aaf57d740 1 -- 172.21.15.97:0/1509683522 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab825ca710 con 0x55ab8267bf70 2024-03-26T19:09:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.133+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ab8267bf70 0x55ab8267e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.133+0000 7f6a87400640 10 client.26029 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.133+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ab8267bf70 0x55ab8267e360 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5acd8af370bd2 server_cookie=cf3510340ca42cd in_seq=0 out_seq=0 2024-03-26T19:09:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6a87400640 1 -- 172.21.15.97:0/1509683522 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ab825ca710 con 0x55ab8267bf70 2024-03-26T19:09:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6a87400640 10 client.26029 handle_command_reply: tid=0 2024-03-26T19:09:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6aaf57d740 1 client.26029 shutdown 2024-03-26T19:09:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6aaf57d740 1 -- 172.21.15.97:0/1509683522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 msgr2=0x55ab82670c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab82670c80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6a940096c0 tx=0x7f6a94004550 comp rx=0 tx=0).stop 2024-03-26T19:09:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6aaf57d740 1 -- 172.21.15.97:0/1509683522 shutdown_connections 2024-03-26T19:09:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ab8267bf70 0x55ab8267e360 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6a87400640 0 client.26029 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 0x55ab82677cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab82670c80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/1509683522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82670740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.134+0000 7f6aaf57d740 1 -- 172.21.15.97:0/1509683522 >> 172.21.15.97:0/1509683522 conn(0x55ab826594d0 msgr2=0x55ab826586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.135+0000 7f6aaf57d740 1 -- 172.21.15.97:0/1509683522 shutdown_connections 2024-03-26T19:09:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.135+0000 7f6aaf57d740 1 -- 172.21.15.97:0/1509683522 wait complete. 2024-03-26T19:09:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.136+0000 7f6aaf57d740 20 client.26029 trim_cache size 0 max 16384 2024-03-26T19:09:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.176+0000 7f6aaf57d740 1 Processor -- start 2024-03-26T19:09:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.177+0000 7f6aaf57d740 1 -- start start 2024-03-26T19:09:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.177+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82666770 0x55ab8265ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.177+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab82678710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.177+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8266c2d0 0x55ab82677c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.177+0000 7f6aaf57d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41a6880 con 0x55ab8266c2d0 2024-03-26T19:09:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.177+0000 7f6aaf57d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41a83c0 con 0x55ab82667160 2024-03-26T19:09:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.177+0000 7f6aaf57d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41a6f50 con 0x55ab82666770 2024-03-26T19:09:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.178+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab82678710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.178+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab82678710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:45962/0 (socket says 172.21.15.97:45962) 2024-03-26T19:09:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.178+0000 7f6aa9400640 1 -- 172.21.15.97:0/3836536602 learned_addr learned my addr 172.21.15.97:0/3836536602 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.178+0000 7f6aaa800640 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8266c2d0 0x55ab82677c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.178+0000 7f6aa9400640 1 -- 172.21.15.97:0/3836536602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82666770 msgr2=0x55ab8265ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.178+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82666770 0x55ab8265ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.178+0000 7f6aa9400640 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82666770 0x55ab8265ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.179+0000 7f6aa9400640 1 -- 172.21.15.97:0/3836536602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8266c2d0 msgr2=0x55ab82677c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.179+0000 7f6aa9400640 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8266c2d0 0x55ab82677c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.179+0000 7f6aa9400640 1 -- 172.21.15.97:0/3836536602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab8265a240 con 0x55ab82667160 2024-03-26T19:09:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.180+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82666770 0x55ab8265ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:09:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.180+0000 7f6aaa800640 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8266c2d0 0x55ab82677c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:09:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.182+0000 7f6aa9400640 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab82678710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6a940349e0 tx=0x7f6a94009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef90dc6ee0b04cf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.183+0000 7f6a92600640 1 -- 172.21.15.97:0/3836536602 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94003e60 con 0x55ab82667160 2024-03-26T19:09:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.184+0000 7f6a92600640 1 -- 172.21.15.97:0/3836536602 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a940038c0 con 0x55ab82667160 2024-03-26T19:09:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.184+0000 7f6a92600640 1 -- 172.21.15.97:0/3836536602 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94005470 con 0x55ab82667160 2024-03-26T19:09:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.184+0000 7f6aaf57d740 1 -- 172.21.15.97:0/3836536602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 msgr2=0x55ab82678710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.184+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab82678710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6a940349e0 tx=0x7f6a94009780 comp rx=0 tx=0).stop 2024-03-26T19:09:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.185+0000 7f6aaf57d740 1 -- 172.21.15.97:0/3836536602 shutdown_connections 2024-03-26T19:09:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.185+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8266c2d0 0x55ab82677c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.185+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab82667160 0x55ab82678710 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.185+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/3836536602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82666770 0x55ab8265ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.185+0000 7f6aaf57d740 1 -- 172.21.15.97:0/3836536602 >> 172.21.15.97:0/3836536602 conn(0x55ab826594d0 msgr2=0x55ab8265c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.185+0000 7f6aaf57d740 1 -- 172.21.15.97:0/3836536602 shutdown_connections 2024-03-26T19:09:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.185+0000 7f6aaf57d740 1 -- 172.21.15.97:0/3836536602 wait complete. 2024-03-26T19:09:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.185+0000 7f6aaf57d740 1 -- start start 2024-03-26T19:09:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.186+0000 7f6aaf57d740 1 Processor -- start 2024-03-26T19:09:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.186+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82672c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.186+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab826731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.186+0000 7f6aaf57d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 0x55ab826f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aaf57d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41a7cf0 con 0x55ab82666770 2024-03-26T19:09:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab826731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aa9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82672c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aaa800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 0x55ab826f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab826731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:48704/0 (socket says 172.21.15.97:48704) 2024-03-26T19:09:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aa9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82672c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51176/0 (socket says 172.21.15.97:51176) 2024-03-26T19:09:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aa9400640 1 -- 172.21.15.97:0/4151446949 learned_addr learned my addr 172.21.15.97:0/4151446949 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4151446949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41a7620 con 0x55ab8266c2d0 2024-03-26T19:09:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4151446949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41a4880 con 0x55ab82667160 2024-03-26T19:09:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aaf57d740 10 client.0 _ll_register_callbacks cb 0x7f6a94004250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.187+0000 7f6aa9400640 1 -- 172.21.15.97:0/4151446949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 msgr2=0x55ab826f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.188+0000 7f6aa9400640 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 0x55ab826f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.188+0000 7f6aa9400640 1 -- 172.21.15.97:0/4151446949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 msgr2=0x55ab82672c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.188+0000 7f6aa9400640 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82672c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.188+0000 7f6aa9400640 1 -- 172.21.15.97:0/4151446949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab826f18b0 con 0x55ab82667160 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.188+0000 7f6aaa800640 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 0x55ab826f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.188+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82672c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.188+0000 7f6aa9400640 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab826731d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6a9402c7c0 tx=0x7f6a94004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6216f47e19c0d348 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.188+0000 7f6a87400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.188+0000 7f6a87400640 1 -- 172.21.15.97:0/4151446949 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a940053f0 con 0x55ab82667160 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.189+0000 7f6a87400640 1 -- 172.21.15.97:0/4151446949 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6a94005590 con 0x55ab82667160 2024-03-26T19:09:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.189+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4151446949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a940038c0 con 0x55ab82667160 2024-03-26T19:09:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.189+0000 7f6a87400640 1 -- 172.21.15.97:0/4151446949 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a94069050 con 0x55ab82667160 2024-03-26T19:09:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.189+0000 7f6aaf57d740 10 client.26032 fetch_fsmap learned FSMap version 27 2024-03-26T19:09:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.189+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4151446949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab826f2af0 con 0x55ab82667160 2024-03-26T19:09:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.189+0000 7f6a87400640 1 -- 172.21.15.97:0/4151446949 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a940038c0 con 0x55ab82667160 2024-03-26T19:09:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.190+0000 7f6a87400640 1 -- 172.21.15.97:0/4151446949 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a94046480 con 0x55ab82667160 2024-03-26T19:09:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.190+0000 7f6aaf57d740 10 client.26032 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:09:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.190+0000 7f6aaf57d740 10 client.26032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:09:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.190+0000 7f6aaf57d740 20 client.26032 populate_metadata read hostname 'smithi097' 2024-03-26T19:09:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.190+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ab8267bf70 0x55ab826f5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.190+0000 7f6aaf57d740 4 client.26032 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:09:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.190+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4151446949 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab82548630 con 0x55ab8267bf70 2024-03-26T19:09:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.191+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ab8267bf70 0x55ab826f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.192+0000 7f6aa9e00640 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ab8267bf70 0x55ab826f5320 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf497ad925807d server_cookie=37f798136064acd8 in_seq=0 out_seq=0 2024-03-26T19:09:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.192+0000 7f6a87400640 10 client.26032 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.219+0000 7f6a87400640 1 -- 172.21.15.97:0/4151446949 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab82548630 con 0x55ab8267bf70 2024-03-26T19:09:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.219+0000 7f6a87400640 10 client.26032 handle_command_reply: tid=0 2024-03-26T19:09:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.219+0000 7f6aaf57d740 1 client.26032 shutdown 2024-03-26T19:09:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.219+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4151446949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 msgr2=0x55ab826731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.219+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab826731d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6a9402c7c0 tx=0x7f6a94004530 comp rx=0 tx=0).stop 2024-03-26T19:09:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.220+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4151446949 shutdown_connections 2024-03-26T19:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.220+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ab8267bf70 0x55ab826f5320 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.220+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab8266c2d0 0x55ab826f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.220+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab82667160 0x55ab826731d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.220+0000 7f6aaf57d740 1 --2- 172.21.15.97:0/4151446949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab82666770 0x55ab82672c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.220+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4151446949 >> 172.21.15.97:0/4151446949 conn(0x55ab826594d0 msgr2=0x55ab82667c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.220+0000 7f6a87400640 0 client.26032 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.220+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4151446949 shutdown_connections 2024-03-26T19:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.220+0000 7f6aaf57d740 1 -- 172.21.15.97:0/4151446949 wait complete. 2024-03-26T19:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.221+0000 7f6aaf57d740 20 client.26032 trim_cache size 0 max 16384 2024-03-26T19:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.221+0000 7f6aab200640 1 -- 172.21.15.97:0/1178410959 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6a8c0a2fd0 msgr2=0x7f6a8c0a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.221+0000 7f6aab200640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6a8c0a2fd0 0x7f6a8c0a5490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ab826679c0 tx=0x7f6aa00097e0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.222+0000 7f6aab200640 1 -- 172.21.15.97:0/1178410959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa40fffe0 msgr2=0x7f6aa4199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.222+0000 7f6aab200640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa40fffe0 0x7f6aa4199c30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800e9f0 tx=0x7f6a9800eeb0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.222+0000 7f6aab200640 1 -- 172.21.15.97:0/1178410959 shutdown_connections 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.222+0000 7f6aab200640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f6a8c0a2fd0 0x7f6a8c0a5490 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.222+0000 7f6aab200640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa41012e0 0x7f6aa41a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.222+0000 7f6aab200640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa4100940 0x7f6aa419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.222+0000 7f6aab200640 1 --2- 172.21.15.97:0/1178410959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aa40fffe0 0x7f6aa4199c30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.222+0000 7f6aab200640 1 -- 172.21.15.97:0/1178410959 >> 172.21.15.97:0/1178410959 conn(0x7f6aa40fbba0 msgr2=0x7f6aa40fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.222+0000 7f6aab200640 1 -- 172.21.15.97:0/1178410959 shutdown_connections 2024-03-26T19:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:36.222+0000 7f6aab200640 1 -- 172.21.15.97:0/1178410959 wait complete. 2024-03-26T19:09:36.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:36 smithi105 ceph-mon[36270]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 720 op/s 2024-03-26T19:09:36.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:36 smithi097 ceph-mon[31955]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 720 op/s 2024-03-26T19:09:36.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:36 smithi144 ceph-mon[36053]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 720 op/s 2024-03-26T19:09:38.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:38 smithi105 ceph-mon[36270]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 605 op/s 2024-03-26T19:09:38.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:38 smithi097 ceph-mon[31955]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 605 op/s 2024-03-26T19:09:38.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:38 smithi144 ceph-mon[36053]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 605 op/s 2024-03-26T19:09:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 615.0 s, 1476.4 tps, lat 66.423 ms stddev 82.306 2024-03-26T19:09:40.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:40 smithi105 ceph-mon[36270]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 660 op/s 2024-03-26T19:09:40.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:40 smithi097 ceph-mon[31955]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 660 op/s 2024-03-26T19:09:40.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:40 smithi144 ceph-mon[36053]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 660 op/s 2024-03-26T19:09:41.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:41 smithi097 ceph-mon[31955]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 737 op/s 2024-03-26T19:09:41.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:41 smithi144 ceph-mon[36053]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 737 op/s 2024-03-26T19:09:41.966 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:41 smithi105 ceph-mon[36270]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 737 op/s 2024-03-26T19:09:44.584 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:44 smithi144 ceph-mon[36053]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 MiB/s wr, 634 op/s 2024-03-26T19:09:44.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:44 smithi097 ceph-mon[31955]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 MiB/s wr, 634 op/s 2024-03-26T19:09:44.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:44 smithi105 ceph-mon[36270]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 MiB/s wr, 634 op/s 2024-03-26T19:09:45.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 620.0 s, 998.4 tps, lat 102.398 ms stddev 121.101 2024-03-26T19:09:46.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:46 smithi097 ceph-mon[31955]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.1 MiB/s wr, 739 op/s 2024-03-26T19:09:46.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:46 smithi144 ceph-mon[36053]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.1 MiB/s wr, 739 op/s 2024-03-26T19:09:46.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:46 smithi105 ceph-mon[36270]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.1 MiB/s wr, 739 op/s 2024-03-26T19:09:48.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:48 smithi097 ceph-mon[31955]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.1 MiB/s wr, 564 op/s 2024-03-26T19:09:48.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:48 smithi144 ceph-mon[36053]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.1 MiB/s wr, 564 op/s 2024-03-26T19:09:48.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:48 smithi105 ceph-mon[36270]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.1 MiB/s wr, 564 op/s 2024-03-26T19:09:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.415+0000 7fd892600640 1 -- 172.21.15.97:0/506853860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c102ac0 msgr2=0x7fd88c104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.415+0000 7fd892600640 1 --2- 172.21.15.97:0/506853860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c102ac0 0x7fd88c104f30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd878009900 tx=0x7fd87802bc60 comp rx=0 tx=0).stop 2024-03-26T19:09:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.415+0000 7fd892600640 1 -- 172.21.15.97:0/506853860 shutdown_connections 2024-03-26T19:09:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.415+0000 7fd892600640 1 --2- 172.21.15.97:0/506853860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c105470 0x7fd88c1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.415+0000 7fd892600640 1 --2- 172.21.15.97:0/506853860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c102ac0 0x7fd88c104f30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.415+0000 7fd892600640 1 --2- 172.21.15.97:0/506853860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c100150 0x7fd88c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.415+0000 7fd892600640 1 -- 172.21.15.97:0/506853860 >> 172.21.15.97:0/506853860 conn(0x7fd88c0fbd50 msgr2=0x7fd88c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.416+0000 7fd892600640 1 -- 172.21.15.97:0/506853860 shutdown_connections 2024-03-26T19:09:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.416+0000 7fd892600640 1 -- 172.21.15.97:0/506853860 wait complete. 2024-03-26T19:09:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.417+0000 7fd892600640 1 Processor -- start 2024-03-26T19:09:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd892600640 1 -- start start 2024-03-26T19:09:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd892600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c100150 0x7fd88c10b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd892600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c102ac0 0x7fd88c10bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd892600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c105470 0x7fd88c1ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd892600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c113c00 con 0x7fd88c105470 2024-03-26T19:09:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd892600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c113a80 con 0x7fd88c100150 2024-03-26T19:09:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd892600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c113d80 con 0x7fd88c102ac0 2024-03-26T19:09:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd88b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c100150 0x7fd88c10b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd88b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c100150 0x7fd88c10b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44636/0 (socket says 172.21.15.97:44636) 2024-03-26T19:09:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd88b400640 1 -- 172.21.15.97:0/887060334 learned_addr learned my addr 172.21.15.97:0/887060334 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.418+0000 7fd88aa00640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c102ac0 0x7fd88c10bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.419+0000 7fd88be00640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c105470 0x7fd88c1ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.419+0000 7fd88b400640 1 -- 172.21.15.97:0/887060334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c102ac0 msgr2=0x7fd88c10bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.419+0000 7fd88b400640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c102ac0 0x7fd88c10bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.419+0000 7fd88b400640 1 -- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c105470 msgr2=0x7fd88c1ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.419+0000 7fd88b400640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c105470 0x7fd88c1ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.419+0000 7fd88b400640 1 -- 172.21.15.97:0/887060334 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c10ecd0 con 0x7fd88c100150 2024-03-26T19:09:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.419+0000 7fd88be00640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c105470 0x7fd88c1ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:09:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.419+0000 7fd88b400640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c100150 0x7fd88c10b610 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e9f0 tx=0x7fd87c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.419+0000 7fd883e00640 1 -- 172.21.15.97:0/887060334 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00cd80 con 0x7fd88c100150 2024-03-26T19:09:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.419+0000 7fd883e00640 1 -- 172.21.15.97:0/887060334 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87c010910 con 0x7fd88c100150 2024-03-26T19:09:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.420+0000 7fd883e00640 1 -- 172.21.15.97:0/887060334 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c018990 con 0x7fd88c100150 2024-03-26T19:09:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.420+0000 7fd892600640 1 -- 172.21.15.97:0/887060334 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c1ae8b0 con 0x7fd88c100150 2024-03-26T19:09:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.420+0000 7fd892600640 1 -- 172.21.15.97:0/887060334 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c1aed80 con 0x7fd88c100150 2024-03-26T19:09:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.423+0000 7fd894ffd740 1 Processor -- start 2024-03-26T19:09:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.424+0000 7fd894ffd740 1 -- start start 2024-03-26T19:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.425+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.425+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c537790 0x55a47c53c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.425+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c53ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.425+0000 7fd894ffd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a47c49be80 con 0x55a47c53c900 2024-03-26T19:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.425+0000 7fd894ffd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a47c444660 con 0x55a47c536da0 2024-03-26T19:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.425+0000 7fd894ffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a47c497e30 con 0x55a47c537790 2024-03-26T19:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.426+0000 7fd88be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.426+0000 7fd88be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5371c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44644/0 (socket says 172.21.15.97:44644) 2024-03-26T19:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.426+0000 7fd88be00640 1 -- 172.21.15.97:0/3462319632 learned_addr learned my addr 172.21.15.97:0/3462319632 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.426+0000 7fd88aa00640 1 --2- 172.21.15.97:0/3462319632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c53ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.426+0000 7fd88be00640 1 -- 172.21.15.97:0/3462319632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c537790 msgr2=0x55a47c53c290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.426+0000 7fd88be00640 1 --2- 172.21.15.97:0/3462319632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c537790 0x55a47c53c290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.426+0000 7fd88be00640 1 -- 172.21.15.97:0/3462319632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 msgr2=0x55a47c53ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.426+0000 7fd88be00640 1 --2- 172.21.15.97:0/3462319632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c53ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.426+0000 7fd88be00640 1 -- 172.21.15.97:0/3462319632 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a47c1b0210 con 0x55a47c536da0 2024-03-26T19:09:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.427+0000 7fd88be00640 1 --2- 172.21.15.97:0/3462319632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5371c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd884007f30 tx=0x7fd88400c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800bd0560f0889bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.427+0000 7fd880c00640 1 -- 172.21.15.97:0/3462319632 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884014070 con 0x55a47c536da0 2024-03-26T19:09:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.427+0000 7fd880c00640 1 -- 172.21.15.97:0/3462319632 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88400fd30 con 0x55a47c536da0 2024-03-26T19:09:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.427+0000 7fd880c00640 1 -- 172.21.15.97:0/3462319632 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88400ed90 con 0x55a47c536da0 2024-03-26T19:09:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.427+0000 7fd883e00640 1 -- 172.21.15.97:0/887060334 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fd87c010ac0 con 0x7fd88c100150 2024-03-26T19:09:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.428+0000 7fd894ffd740 1 -- 172.21.15.97:0/3462319632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 msgr2=0x55a47c5371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.428+0000 7fd894ffd740 1 --2- 172.21.15.97:0/3462319632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5371c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd884007f30 tx=0x7fd88400c750 comp rx=0 tx=0).stop 2024-03-26T19:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.428+0000 7fd894ffd740 1 -- 172.21.15.97:0/3462319632 shutdown_connections 2024-03-26T19:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.429+0000 7fd894ffd740 1 --2- 172.21.15.97:0/3462319632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c53ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.429+0000 7fd894ffd740 1 --2- 172.21.15.97:0/3462319632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c537790 0x55a47c53c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.429+0000 7fd894ffd740 1 --2- 172.21.15.97:0/3462319632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5371c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.429+0000 7fd894ffd740 1 -- 172.21.15.97:0/3462319632 >> 172.21.15.97:0/3462319632 conn(0x55a47c532310 msgr2=0x55a47c532710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.429+0000 7fd883e00640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8700a3040 0x7fd8700a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.429+0000 7fd883e00640 1 -- 172.21.15.97:0/887060334 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd87c0cf7e0 con 0x7fd88c100150 2024-03-26T19:09:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.430+0000 7fd894ffd740 1 -- 172.21.15.97:0/3462319632 shutdown_connections 2024-03-26T19:09:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.430+0000 7fd88aa00640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8700a3040 0x7fd8700a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.430+0000 7fd894ffd740 1 -- 172.21.15.97:0/3462319632 wait complete. 2024-03-26T19:09:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.430+0000 7fd894ffd740 1 -- start start 2024-03-26T19:09:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.430+0000 7fd88aa00640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8700a3040 0x7fd8700a5500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd87802c270 tx=0x7fd87802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd894ffd740 1 Processor -- start 2024-03-26T19:09:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c536da0 0x55a47c540da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c537790 0x55a47c5412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c53c900 0x55a47c5482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd894ffd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a47c53f8b0 con 0x55a47c536da0 2024-03-26T19:09:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd894ffd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a47c457ac0 con 0x55a47c537790 2024-03-26T19:09:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd894ffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a47c53fa30 con 0x55a47c53c900 2024-03-26T19:09:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd894ffd740 10 client.0 _ll_register_callbacks cb 0x55a47c457490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:09:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c537790 0x55a47c5412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c537790 0x55a47c5412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44646/0 (socket says 172.21.15.97:44646) 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.431+0000 7fd88aa00640 1 -- 172.21.15.97:0/3523368295 learned_addr learned my addr 172.21.15.97:0/3523368295 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd88be00640 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c53c900 0x55a47c5482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd88aa00640 1 -- 172.21.15.97:0/3523368295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c53c900 msgr2=0x55a47c5482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd88aa00640 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c53c900 0x55a47c5482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd88aa00640 1 -- 172.21.15.97:0/3523368295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c536da0 msgr2=0x55a47c540da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd88aa00640 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c536da0 0x55a47c540da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd88aa00640 1 -- 172.21.15.97:0/3523368295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a47c27c620 con 0x55a47c537790 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd88aa00640 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c537790 0x55a47c5412e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd87802e010 tx=0x7fd87802bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bb07a59efa122b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd875600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd875600640 1 -- 172.21.15.97:0/3523368295 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87804f020 con 0x55a47c537790 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd875600640 1 -- 172.21.15.97:0/3523368295 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd878005410 con 0x55a47c537790 2024-03-26T19:09:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd894ffd740 1 -- 172.21.15.97:0/3523368295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd88400fd30 con 0x55a47c537790 2024-03-26T19:09:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd875600640 1 -- 172.21.15.97:0/3523368295 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd878005610 con 0x55a47c537790 2024-03-26T19:09:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.432+0000 7fd894ffd740 10 client.26072 fetch_fsmap learned FSMap version 27 2024-03-26T19:09:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.433+0000 7fd894ffd740 1 -- 172.21.15.97:0/3523368295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a47c54b690 con 0x55a47c537790 2024-03-26T19:09:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.433+0000 7fd875600640 1 -- 172.21.15.97:0/3523368295 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88400fd30 con 0x55a47c537790 2024-03-26T19:09:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.433+0000 7fd875600640 1 -- 172.21.15.97:0/3523368295 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd878003b20 con 0x55a47c537790 2024-03-26T19:09:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.433+0000 7fd894ffd740 10 client.26072 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:09:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.433+0000 7fd894ffd740 10 client.26072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:09:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.433+0000 7fd894ffd740 20 client.26072 populate_metadata read hostname 'smithi097' 2024-03-26T19:09:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.433+0000 7fd894ffd740 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a47c54cd20 0x55a47c54f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.433+0000 7fd894ffd740 4 client.26072 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:09:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.433+0000 7fd894ffd740 1 -- 172.21.15.97:0/3523368295 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a47c4a93c0 con 0x55a47c54cd20 2024-03-26T19:09:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.434+0000 7fd88b400640 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a47c54cd20 0x55a47c54f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.435+0000 7fd88b400640 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a47c54cd20 0x55a47c54f170 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c6e249e74950650 server_cookie=552fa7b6829a3c3b in_seq=0 out_seq=0 2024-03-26T19:09:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.435+0000 7fd875600640 10 client.26072 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd875600640 1 -- 172.21.15.97:0/3523368295 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a47c4a93c0 con 0x55a47c54cd20 2024-03-26T19:09:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd875600640 10 client.26072 handle_command_reply: tid=0 2024-03-26T19:09:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd894ffd740 1 client.26072 shutdown 2024-03-26T19:09:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd894ffd740 1 -- 172.21.15.97:0/3523368295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c537790 msgr2=0x55a47c5412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd894ffd740 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c537790 0x55a47c5412e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd87802e010 tx=0x7fd87802bc60 comp rx=0 tx=0).stop 2024-03-26T19:09:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd894ffd740 1 -- 172.21.15.97:0/3523368295 shutdown_connections 2024-03-26T19:09:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd894ffd740 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a47c54cd20 0x55a47c54f170 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd894ffd740 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c53c900 0x55a47c5482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd894ffd740 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c537790 0x55a47c5412e0 secure :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd87802e010 tx=0x7fd87802bc60 comp rx=0 tx=0).stop 2024-03-26T19:09:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd894ffd740 1 --2- 172.21.15.97:0/3523368295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c536da0 0x55a47c540da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd894ffd740 1 -- 172.21.15.97:0/3523368295 >> 172.21.15.97:0/3523368295 conn(0x55a47c532310 msgr2=0x55a47c531540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.436+0000 7fd875600640 0 client.26072 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.437+0000 7fd894ffd740 1 -- 172.21.15.97:0/3523368295 shutdown_connections 2024-03-26T19:09:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.437+0000 7fd894ffd740 1 -- 172.21.15.97:0/3523368295 wait complete. 2024-03-26T19:09:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.437+0000 7fd894ffd740 20 client.26072 trim_cache size 0 max 16384 2024-03-26T19:09:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd894ffd740 1 Processor -- start 2024-03-26T19:09:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd894ffd740 1 -- start start 2024-03-26T19:09:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c536da0 0x55a47c535c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c548270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c530cb0 0x55a47c531110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd894ffd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1b7ae0 con 0x55a47c53c900 2024-03-26T19:09:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd894ffd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1b9620 con 0x55a47c530cb0 2024-03-26T19:09:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd894ffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1b81b0 con 0x55a47c536da0 2024-03-26T19:09:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd88be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c530cb0 0x55a47c531110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd88be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c530cb0 0x55a47c531110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44660/0 (socket says 172.21.15.97:44660) 2024-03-26T19:09:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.458+0000 7fd88be00640 1 -- 172.21.15.97:0/4125888003 learned_addr learned my addr 172.21.15.97:0/4125888003 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.459+0000 7fd88b400640 1 --2- 172.21.15.97:0/4125888003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c536da0 0x55a47c535c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.459+0000 7fd88be00640 1 -- 172.21.15.97:0/4125888003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c536da0 msgr2=0x55a47c535c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.459+0000 7fd88be00640 1 --2- 172.21.15.97:0/4125888003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c536da0 0x55a47c535c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.459+0000 7fd88be00640 1 -- 172.21.15.97:0/4125888003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 msgr2=0x55a47c548270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:09:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.459+0000 7fd88be00640 1 --2- 172.21.15.97:0/4125888003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c548270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.459+0000 7fd88be00640 1 -- 172.21.15.97:0/4125888003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a47c53ba20 con 0x55a47c530cb0 2024-03-26T19:09:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.459+0000 7fd88b400640 1 --2- 172.21.15.97:0/4125888003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c536da0 0x55a47c535c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:09:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.460+0000 7fd88be00640 1 --2- 172.21.15.97:0/4125888003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c530cb0 0x55a47c531110 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd884007e50 tx=0x7fd88400e530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c96de48d7863ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.461+0000 7fd877e00640 1 -- 172.21.15.97:0/4125888003 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884014040 con 0x55a47c530cb0 2024-03-26T19:09:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.461+0000 7fd877e00640 1 -- 172.21.15.97:0/4125888003 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd884028070 con 0x55a47c530cb0 2024-03-26T19:09:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.461+0000 7fd877e00640 1 -- 172.21.15.97:0/4125888003 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884014040 con 0x55a47c530cb0 2024-03-26T19:09:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.462+0000 7fd894ffd740 1 -- 172.21.15.97:0/4125888003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c530cb0 msgr2=0x55a47c531110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.462+0000 7fd894ffd740 1 --2- 172.21.15.97:0/4125888003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c530cb0 0x55a47c531110 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd884007e50 tx=0x7fd88400e530 comp rx=0 tx=0).stop 2024-03-26T19:09:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.462+0000 7fd894ffd740 1 -- 172.21.15.97:0/4125888003 shutdown_connections 2024-03-26T19:09:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.462+0000 7fd894ffd740 1 --2- 172.21.15.97:0/4125888003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c530cb0 0x55a47c531110 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.462+0000 7fd894ffd740 1 --2- 172.21.15.97:0/4125888003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c548270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.462+0000 7fd894ffd740 1 --2- 172.21.15.97:0/4125888003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c536da0 0x55a47c535c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.462+0000 7fd894ffd740 1 -- 172.21.15.97:0/4125888003 >> 172.21.15.97:0/4125888003 conn(0x55a47c532310 msgr2=0x55a47c535570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.463+0000 7fd894ffd740 1 -- 172.21.15.97:0/4125888003 shutdown_connections 2024-03-26T19:09:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.463+0000 7fd894ffd740 1 -- 172.21.15.97:0/4125888003 wait complete. 2024-03-26T19:09:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.463+0000 7fd894ffd740 1 -- start start 2024-03-26T19:09:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.464+0000 7fd894ffd740 1 Processor -- start 2024-03-26T19:09:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.464+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c530cb0 0x55a47c547c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.464+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.464+0000 7fd894ffd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c5462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.464+0000 7fd894ffd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1b8f50 con 0x55a47c53c900 2024-03-26T19:09:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.464+0000 7fd894ffd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1b8880 con 0x55a47c536da0 2024-03-26T19:09:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.464+0000 7fd894ffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1b5ac0 con 0x55a47c530cb0 2024-03-26T19:09:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.464+0000 7fd894ffd740 10 client.0 _ll_register_callbacks cb 0x7fd87802cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:09:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:44668/0 (socket says 172.21.15.97:44668) 2024-03-26T19:09:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88aa00640 1 -- 172.21.15.97:0/213143774 learned_addr learned my addr 172.21.15.97:0/213143774 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:09:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88b400640 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c530cb0 0x55a47c547c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88aa00640 1 -- 172.21.15.97:0/213143774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c530cb0 msgr2=0x55a47c547c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88aa00640 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c530cb0 0x55a47c547c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88aa00640 1 -- 172.21.15.97:0/213143774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 msgr2=0x55a47c5462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:09:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88be00640 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c5462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88aa00640 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c5462c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88aa00640 1 -- 172.21.15.97:0/213143774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a47c546af0 con 0x55a47c536da0 2024-03-26T19:09:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88be00640 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c5462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:09:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd88aa00640 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5481b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd87802c810 tx=0x7fd878005540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e406c9db635a6f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:09:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd875600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:09:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.465+0000 7fd875600640 1 -- 172.21.15.97:0/213143774 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87804f070 con 0x55a47c536da0 2024-03-26T19:09:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.466+0000 7fd875600640 1 -- 172.21.15.97:0/213143774 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd87803fde0 con 0x55a47c536da0 2024-03-26T19:09:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.466+0000 7fd894ffd740 1 -- 172.21.15.97:0/213143774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd884028070 con 0x55a47c536da0 2024-03-26T19:09:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.466+0000 7fd875600640 1 -- 172.21.15.97:0/213143774 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd878065070 con 0x55a47c536da0 2024-03-26T19:09:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.466+0000 7fd894ffd740 10 client.26078 fetch_fsmap learned FSMap version 27 2024-03-26T19:09:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.466+0000 7fd894ffd740 1 -- 172.21.15.97:0/213143774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a47c5b3c90 con 0x55a47c536da0 2024-03-26T19:09:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.466+0000 7fd875600640 1 -- 172.21.15.97:0/213143774 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd884028070 con 0x55a47c536da0 2024-03-26T19:09:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.466+0000 7fd875600640 1 -- 172.21.15.97:0/213143774 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd87805d030 con 0x55a47c536da0 2024-03-26T19:09:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.466+0000 7fd894ffd740 10 client.26078 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:09:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.467+0000 7fd894ffd740 10 client.26078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:09:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.467+0000 7fd894ffd740 20 client.26078 populate_metadata read hostname 'smithi097' 2024-03-26T19:09:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.467+0000 7fd894ffd740 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a47c5b4640 0x55a47c5b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:09:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.467+0000 7fd894ffd740 4 client.26078 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:09:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.467+0000 7fd894ffd740 1 -- 172.21.15.97:0/213143774 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a47c4a3710 con 0x55a47c5b4640 2024-03-26T19:09:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.467+0000 7fd88b400640 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a47c5b4640 0x55a47c5b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:09:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.467+0000 7fd88b400640 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a47c5b4640 0x55a47c5b6a60 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6274aa3028d16f1c server_cookie=f569c59410b0e1c in_seq=0 out_seq=0 2024-03-26T19:09:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.467+0000 7fd875600640 10 client.26078 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.486+0000 7fd875600640 1 -- 172.21.15.97:0/213143774 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a47c4a3710 con 0x55a47c5b4640 2024-03-26T19:09:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.486+0000 7fd875600640 10 client.26078 handle_command_reply: tid=0 2024-03-26T19:09:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.486+0000 7fd894ffd740 1 client.26078 shutdown 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 1 -- 172.21.15.97:0/213143774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 msgr2=0x55a47c5481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5481b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd87802c810 tx=0x7fd878005540 comp rx=0 tx=0).stop 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 1 -- 172.21.15.97:0/213143774 shutdown_connections 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a47c5b4640 0x55a47c5b6a60 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a47c53c900 0x55a47c5462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a47c536da0 0x55a47c5481b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 1 --2- 172.21.15.97:0/213143774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a47c530cb0 0x55a47c547c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 1 -- 172.21.15.97:0/213143774 >> 172.21.15.97:0/213143774 conn(0x55a47c532310 msgr2=0x55a47c535570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd875600640 0 client.26078 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 1 -- 172.21.15.97:0/213143774 shutdown_connections 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 1 -- 172.21.15.97:0/213143774 wait complete. 2024-03-26T19:09:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.487+0000 7fd894ffd740 20 client.26078 trim_cache size 0 max 16384 2024-03-26T19:09:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:09:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:09:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:09:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:09:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.488+0000 7fd892600640 1 -- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8700a3040 msgr2=0x7fd8700a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.488+0000 7fd892600640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8700a3040 0x7fd8700a5500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd87802c270 tx=0x7fd87802e040 comp rx=0 tx=0).stop 2024-03-26T19:09:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.488+0000 7fd892600640 1 -- 172.21.15.97:0/887060334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c100150 msgr2=0x7fd88c10b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:09:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.488+0000 7fd892600640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c100150 0x7fd88c10b610 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e9f0 tx=0x7fd87c00eeb0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.488+0000 7fd892600640 1 -- 172.21.15.97:0/887060334 shutdown_connections 2024-03-26T19:09:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.488+0000 7fd892600640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd8700a3040 0x7fd8700a5500 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.488+0000 7fd892600640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c105470 0x7fd88c1ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.488+0000 7fd892600640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c102ac0 0x7fd88c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.488+0000 7fd892600640 1 --2- 172.21.15.97:0/887060334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c100150 0x7fd88c10b610 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:09:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.488+0000 7fd892600640 1 -- 172.21.15.97:0/887060334 >> 172.21.15.97:0/887060334 conn(0x7fd88c0fbd50 msgr2=0x7fd88c100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:09:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.489+0000 7fd892600640 1 -- 172.21.15.97:0/887060334 shutdown_connections 2024-03-26T19:09:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:09:49.489+0000 7fd892600640 1 -- 172.21.15.97:0/887060334 wait complete. 2024-03-26T19:09:50.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 625.0 s, 1081.2 tps, lat 92.016 ms stddev 102.724 2024-03-26T19:09:50.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:50 smithi097 ceph-mon[31955]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 MiB/s wr, 621 op/s 2024-03-26T19:09:50.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:50 smithi144 ceph-mon[36053]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 MiB/s wr, 621 op/s 2024-03-26T19:09:50.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:50 smithi105 ceph-mon[36270]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 MiB/s wr, 621 op/s 2024-03-26T19:09:52.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:52 smithi097 ceph-mon[31955]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 675 op/s 2024-03-26T19:09:52.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:52 smithi144 ceph-mon[36053]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 675 op/s 2024-03-26T19:09:52.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:52 smithi105 ceph-mon[36270]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 675 op/s 2024-03-26T19:09:54.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:54 smithi097 ceph-mon[31955]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 MiB/s wr, 527 op/s 2024-03-26T19:09:54.634 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:54 smithi144 ceph-mon[36053]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 MiB/s wr, 527 op/s 2024-03-26T19:09:54.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:54 smithi105 ceph-mon[36270]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 MiB/s wr, 527 op/s 2024-03-26T19:09:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 630.0 s, 1032.4 tps, lat 95.997 ms stddev 111.323 2024-03-26T19:09:56.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:56 smithi097 ceph-mon[31955]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 660 op/s 2024-03-26T19:09:56.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:56 smithi144 ceph-mon[36053]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 660 op/s 2024-03-26T19:09:56.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:56 smithi105 ceph-mon[36270]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 660 op/s 2024-03-26T19:09:58.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:09:58 smithi097 ceph-mon[31955]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.6 MiB/s wr, 525 op/s 2024-03-26T19:09:58.631 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:09:58 smithi144 ceph-mon[36053]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.6 MiB/s wr, 525 op/s 2024-03-26T19:09:58.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:09:58 smithi105 ceph-mon[36270]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.6 MiB/s wr, 525 op/s 2024-03-26T19:10:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 635.0 s, 1349.4 tps, lat 74.941 ms stddev 95.031 2024-03-26T19:10:00.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:00 smithi097 ceph-mon[31955]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 MiB/s wr, 572 op/s 2024-03-26T19:10:00.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:00 smithi097 ceph-mon[31955]: overall HEALTH_OK 2024-03-26T19:10:00.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:00 smithi144 ceph-mon[36053]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 MiB/s wr, 572 op/s 2024-03-26T19:10:00.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:00 smithi144 ceph-mon[36053]: overall HEALTH_OK 2024-03-26T19:10:00.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:00 smithi105 ceph-mon[36270]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 MiB/s wr, 572 op/s 2024-03-26T19:10:00.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:00 smithi105 ceph-mon[36270]: overall HEALTH_OK 2024-03-26T19:10:02.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:02 smithi144 ceph-mon[36053]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 668 op/s 2024-03-26T19:10:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.684+0000 7f8b2d400640 1 -- 172.21.15.97:0/2416648046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b28075900 msgr2=0x7f8b28073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.684+0000 7f8b2d400640 1 --2- 172.21.15.97:0/2416648046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b28075900 0x7f8b28073d50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b18009900 tx=0x7f8b1802bc30 comp rx=0 tx=0).stop 2024-03-26T19:10:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.684+0000 7f8b2d400640 1 -- 172.21.15.97:0/2416648046 shutdown_connections 2024-03-26T19:10:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.684+0000 7f8b2d400640 1 --2- 172.21.15.97:0/2416648046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b281097b0 0x7f8b2810bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.684+0000 7f8b2d400640 1 --2- 172.21.15.97:0/2416648046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b28074320 0x7f8b280747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.684+0000 7f8b2d400640 1 --2- 172.21.15.97:0/2416648046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b28075900 0x7f8b28073d50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.684+0000 7f8b2d400640 1 -- 172.21.15.97:0/2416648046 >> 172.21.15.97:0/2416648046 conn(0x7f8b280fce80 msgr2=0x7f8b280ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.684+0000 7f8b2d400640 1 -- 172.21.15.97:0/2416648046 shutdown_connections 2024-03-26T19:10:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.684+0000 7f8b2d400640 1 -- 172.21.15.97:0/2416648046 wait complete. 2024-03-26T19:10:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.685+0000 7f8b2d400640 1 Processor -- start 2024-03-26T19:10:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.686+0000 7f8b2d400640 1 -- start start 2024-03-26T19:10:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.686+0000 7f8b2d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b28074320 0x7f8b2819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.686+0000 7f8b2d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b28075900 0x7f8b2819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.686+0000 7f8b2d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b281097b0 0x7f8b281a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.686+0000 7f8b2d400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2810f780 con 0x7f8b28075900 2024-03-26T19:10:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.686+0000 7f8b2d400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2810f600 con 0x7f8b281097b0 2024-03-26T19:10:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.686+0000 7f8b2d400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2810f900 con 0x7f8b28074320 2024-03-26T19:10:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.687+0000 7f8b26800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b28074320 0x7f8b2819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.687+0000 7f8b26800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b28074320 0x7f8b2819e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60090/0 (socket says 172.21.15.97:60090) 2024-03-26T19:10:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.687+0000 7f8b26800640 1 -- 172.21.15.97:0/4072048942 learned_addr learned my addr 172.21.15.97:0/4072048942 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.688+0000 7f8b25e00640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b28075900 0x7f8b2819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.688+0000 7f8b27200640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b281097b0 0x7f8b281a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.688+0000 7f8b26800640 1 -- 172.21.15.97:0/4072048942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b281097b0 msgr2=0x7f8b281a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.688+0000 7f8b26800640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b281097b0 0x7f8b281a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.688+0000 7f8b26800640 1 -- 172.21.15.97:0/4072048942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b28075900 msgr2=0x7f8b2819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.688+0000 7f8b26800640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b28075900 0x7f8b2819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.688+0000 7f8b26800640 1 -- 172.21.15.97:0/4072048942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b281a5cd0 con 0x7f8b28074320 2024-03-26T19:10:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.689+0000 7f8b26800640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b28074320 0x7f8b2819e0b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8b18005ab0 tx=0x7f8b1802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.691+0000 7f8b1f400640 1 -- 172.21.15.97:0/4072048942 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18043020 con 0x7f8b28074320 2024-03-26T19:10:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.691+0000 7f8b1f400640 1 -- 172.21.15.97:0/4072048942 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b180054e0 con 0x7f8b28074320 2024-03-26T19:10:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.691+0000 7f8b1f400640 1 -- 172.21.15.97:0/4072048942 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18003d70 con 0x7f8b28074320 2024-03-26T19:10:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.692+0000 7f8b2d400640 1 -- 172.21.15.97:0/4072048942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b281a5f60 con 0x7f8b28074320 2024-03-26T19:10:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.693+0000 7f8b2d400640 1 -- 172.21.15.97:0/4072048942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b281a6370 con 0x7f8b28074320 2024-03-26T19:10:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.697+0000 7f8b1f400640 1 -- 172.21.15.97:0/4072048942 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f8b18046020 con 0x7f8b28074320 2024-03-26T19:10:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.698+0000 7f8b3019c740 1 Processor -- start 2024-03-26T19:10:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.698+0000 7f8b3019c740 1 -- start start 2024-03-26T19:10:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.699+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd05da0 0x559f6bd061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.699+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd0b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.699+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 0x559f6bd0dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.700+0000 7f8b1f400640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8b0c0a31b0 0x7f8b0c0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.700+0000 7f8b3019c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559f6bc6ae80 con 0x559f6bd06790 2024-03-26T19:10:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.700+0000 7f8b3019c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559f6bc13660 con 0x559f6bd05da0 2024-03-26T19:10:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.700+0000 7f8b3019c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f6bc66e30 con 0x559f6bd0b900 2024-03-26T19:10:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.700+0000 7f8b27200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd05da0 0x559f6bd061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.700+0000 7f8b1f400640 1 -- 172.21.15.97:0/4072048942 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8b180f0ed0 con 0x7f8b28074320 2024-03-26T19:10:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.700+0000 7f8b27200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd05da0 0x559f6bd061c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57468/0 (socket says 172.21.15.97:57468) 2024-03-26T19:10:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.700+0000 7f8b27200640 1 -- 172.21.15.97:0/3974049569 learned_addr learned my addr 172.21.15.97:0/3974049569 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.700+0000 7f8b27200640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8b0c0a31b0 0x7f8b0c0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.700+0000 7f8b26800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd0b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.702+0000 7f8b25e00640 1 --2- 172.21.15.97:0/3974049569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 0x559f6bd0dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.702+0000 7f8b26800640 1 -- 172.21.15.97:0/3974049569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 msgr2=0x559f6bd0dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.702+0000 7f8b26800640 1 --2- 172.21.15.97:0/3974049569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 0x559f6bd0dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.702+0000 7f8b26800640 1 -- 172.21.15.97:0/3974049569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd05da0 msgr2=0x559f6bd061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.702+0000 7f8b26800640 1 --2- 172.21.15.97:0/3974049569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd05da0 0x559f6bd061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.702+0000 7f8b26800640 1 -- 172.21.15.97:0/3974049569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6b97f210 con 0x559f6bd06790 2024-03-26T19:10:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.702+0000 7f8b27200640 1 --2- 172.21.15.97:0/3974049569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd05da0 0x559f6bd061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:10:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.702+0000 7f8b27200640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8b0c0a31b0 0x7f8b0c0a5670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8b20007fd0 tx=0x7f8b20007b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.702+0000 7f8b26800640 1 --2- 172.21.15.97:0/3974049569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd0b290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8b180ee9a0 tx=0x7f8b18003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af00a2cb697ff206 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.703+0000 7f8b13400640 1 -- 172.21.15.97:0/3974049569 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b180d5070 con 0x559f6bd06790 2024-03-26T19:10:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.703+0000 7f8b13400640 1 -- 172.21.15.97:0/3974049569 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b18004c80 con 0x559f6bd06790 2024-03-26T19:10:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.703+0000 7f8b13400640 1 -- 172.21.15.97:0/3974049569 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b180d5070 con 0x559f6bd06790 2024-03-26T19:10:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.704+0000 7f8b3019c740 1 -- 172.21.15.97:0/3974049569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 msgr2=0x559f6bd0b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.704+0000 7f8b3019c740 1 --2- 172.21.15.97:0/3974049569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd0b290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8b180ee9a0 tx=0x7f8b18003840 comp rx=0 tx=0).stop 2024-03-26T19:10:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.704+0000 7f8b3019c740 1 -- 172.21.15.97:0/3974049569 shutdown_connections 2024-03-26T19:10:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.704+0000 7f8b3019c740 1 --2- 172.21.15.97:0/3974049569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 0x559f6bd0dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.704+0000 7f8b3019c740 1 --2- 172.21.15.97:0/3974049569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd0b290 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.704+0000 7f8b3019c740 1 --2- 172.21.15.97:0/3974049569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd05da0 0x559f6bd061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.704+0000 7f8b3019c740 1 -- 172.21.15.97:0/3974049569 >> 172.21.15.97:0/3974049569 conn(0x559f6bd01310 msgr2=0x559f6bd01710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.704+0000 7f8b3019c740 1 -- 172.21.15.97:0/3974049569 shutdown_connections 2024-03-26T19:10:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.704+0000 7f8b3019c740 1 -- 172.21.15.97:0/3974049569 wait complete. 2024-03-26T19:10:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.705+0000 7f8b3019c740 1 -- start start 2024-03-26T19:10:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.705+0000 7f8b3019c740 1 Processor -- start 2024-03-26T19:10:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd0f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd0fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 0x559f6bd16e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b3019c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559f6bc6ae80 con 0x559f6bd06790 2024-03-26T19:10:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b3019c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559f6bc26ac0 con 0x559f6bd0b900 2024-03-26T19:10:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b3019c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f6bd0e670 con 0x559f6bd05da0 2024-03-26T19:10:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b3019c740 10 client.0 _ll_register_callbacks cb 0x559f6bc26490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:10:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b26800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd0f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b26800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd0f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60112/0 (socket says 172.21.15.97:60112) 2024-03-26T19:10:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b26800640 1 -- 172.21.15.97:0/1765113272 learned_addr learned my addr 172.21.15.97:0/1765113272 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b27200640 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 0x559f6bd16e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.706+0000 7f8b25e00640 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd0fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.707+0000 7f8b26800640 1 -- 172.21.15.97:0/1765113272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 msgr2=0x559f6bd16e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.707+0000 7f8b26800640 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 0x559f6bd16e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.707+0000 7f8b26800640 1 -- 172.21.15.97:0/1765113272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 msgr2=0x559f6bd0fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.707+0000 7f8b26800640 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd0fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.707+0000 7f8b26800640 1 -- 172.21.15.97:0/1765113272 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6ba4b620 con 0x559f6bd05da0 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.707+0000 7f8b27200640 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 0x559f6bd16e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.707+0000 7f8b25e00640 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd0fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.707+0000 7f8b26800640 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd0f890 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8b180b9a30 tx=0x7f8b180c1720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c5e78e64b49dcb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.707+0000 7f8b07e00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.707+0000 7f8b07e00640 1 -- 172.21.15.97:0/1765113272 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b180d5070 con 0x559f6bd05da0 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.708+0000 7f8b07e00640 1 -- 172.21.15.97:0/1765113272 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8b180bb8d0 con 0x559f6bd05da0 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.708+0000 7f8b3019c740 1 -- 172.21.15.97:0/1765113272 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b18004c80 con 0x559f6bd05da0 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.708+0000 7f8b07e00640 1 -- 172.21.15.97:0/1765113272 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b180eee60 con 0x559f6bd05da0 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.708+0000 7f8b3019c740 10 client.26056 fetch_fsmap learned FSMap version 27 2024-03-26T19:10:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.708+0000 7f8b3019c740 1 -- 172.21.15.97:0/1765113272 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f6bd1a180 con 0x559f6bd05da0 2024-03-26T19:10:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.708+0000 7f8b07e00640 1 -- 172.21.15.97:0/1765113272 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b18004c80 con 0x559f6bd05da0 2024-03-26T19:10:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.709+0000 7f8b07e00640 1 -- 172.21.15.97:0/1765113272 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8b180c1170 con 0x559f6bd05da0 2024-03-26T19:10:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.709+0000 7f8b3019c740 10 client.26056 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:10:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.709+0000 7f8b3019c740 10 client.26056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.709+0000 7f8b3019c740 20 client.26056 populate_metadata read hostname 'smithi097' 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.709+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559f6bd1b860 0x559f6bd1dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.709+0000 7f8b3019c740 4 client.26056 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.709+0000 7f8b3019c740 1 -- 172.21.15.97:0/1765113272 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f6bc783c0 con 0x559f6bd1b860 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.709+0000 7f8b25e00640 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559f6bd1b860 0x559f6bd1dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.710+0000 7f8b25e00640 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559f6bd1b860 0x559f6bd1dc90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbecacd20edccc5e server_cookie=910821a615940c4b in_seq=0 out_seq=0 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.710+0000 7f8b07e00640 10 client.26056 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.710+0000 7f8b07e00640 1 -- 172.21.15.97:0/1765113272 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559f6bc783c0 con 0x559f6bd1b860 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.710+0000 7f8b07e00640 10 client.26056 handle_command_reply: tid=0 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.710+0000 7f8b3019c740 1 client.26056 shutdown 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 1 -- 172.21.15.97:0/1765113272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 msgr2=0x559f6bd0f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd0f890 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8b180b9a30 tx=0x7f8b180c1720 comp rx=0 tx=0).stop 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 1 -- 172.21.15.97:0/1765113272 shutdown_connections 2024-03-26T19:10:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559f6bd1b860 0x559f6bd1dc90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 0x559f6bd16e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd0fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1765113272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd0f890 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 1 -- 172.21.15.97:0/1765113272 >> 172.21.15.97:0/1765113272 conn(0x559f6bd01310 msgr2=0x559f6bd00540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b07e00640 0 client.26056 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 1 -- 172.21.15.97:0/1765113272 shutdown_connections 2024-03-26T19:10:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 1 -- 172.21.15.97:0/1765113272 wait complete. 2024-03-26T19:10:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.711+0000 7f8b3019c740 20 client.26056 trim_cache size 0 max 16384 2024-03-26T19:10:02.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:02 smithi105 ceph-mon[36270]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 668 op/s 2024-03-26T19:10:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.739+0000 7f8b3019c740 1 Processor -- start 2024-03-26T19:10:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.739+0000 7f8b3019c740 1 -- start start 2024-03-26T19:10:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.740+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd05da0 0x559f6bd07020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.740+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd06790 0x559f6bd16da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.740+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 0x559f6bd07560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.740+0000 7f8b3019c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b281af110 con 0x559f6bd05da0 2024-03-26T19:10:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.740+0000 7f8b3019c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b281b0c50 con 0x559f6bd06790 2024-03-26T19:10:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.740+0000 7f8b3019c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8b281af7e0 con 0x559f6bd0b900 2024-03-26T19:10:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.741+0000 7f8b26800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd05da0 0x559f6bd07020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.741+0000 7f8b27200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 0x559f6bd07560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.741+0000 7f8b27200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 0x559f6bd07560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60122/0 (socket says 172.21.15.97:60122) 2024-03-26T19:10:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.741+0000 7f8b27200640 1 -- 172.21.15.97:0/95977743 learned_addr learned my addr 172.21.15.97:0/95977743 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.742+0000 7f8b26800640 1 -- 172.21.15.97:0/95977743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 msgr2=0x559f6bd07560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.742+0000 7f8b26800640 1 --2- 172.21.15.97:0/95977743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 0x559f6bd07560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.742+0000 7f8b26800640 1 -- 172.21.15.97:0/95977743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd06790 msgr2=0x559f6bd16da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.742+0000 7f8b26800640 1 --2- 172.21.15.97:0/95977743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd06790 0x559f6bd16da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.742+0000 7f8b26800640 1 -- 172.21.15.97:0/95977743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6bd1e2e0 con 0x559f6bd05da0 2024-03-26T19:10:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.742+0000 7f8b26800640 1 --2- 172.21.15.97:0/95977743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd05da0 0x559f6bd07020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8b180bd560 tx=0x7f8b18009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6e17e91812357f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.743+0000 7f8b12a00640 1 -- 172.21.15.97:0/95977743 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b180d5070 con 0x559f6bd05da0 2024-03-26T19:10:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.743+0000 7f8b12a00640 1 -- 172.21.15.97:0/95977743 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b18004c80 con 0x559f6bd05da0 2024-03-26T19:10:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.743+0000 7f8b12a00640 1 -- 172.21.15.97:0/95977743 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b180c1170 con 0x559f6bd05da0 2024-03-26T19:10:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.744+0000 7f8b3019c740 1 -- 172.21.15.97:0/95977743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd05da0 msgr2=0x559f6bd07020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.744+0000 7f8b3019c740 1 --2- 172.21.15.97:0/95977743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd05da0 0x559f6bd07020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8b180bd560 tx=0x7f8b18009260 comp rx=0 tx=0).stop 2024-03-26T19:10:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.744+0000 7f8b3019c740 1 -- 172.21.15.97:0/95977743 shutdown_connections 2024-03-26T19:10:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.744+0000 7f8b3019c740 1 --2- 172.21.15.97:0/95977743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd0b900 0x559f6bd07560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.744+0000 7f8b3019c740 1 --2- 172.21.15.97:0/95977743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd06790 0x559f6bd16da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.744+0000 7f8b3019c740 1 --2- 172.21.15.97:0/95977743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd05da0 0x559f6bd07020 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.744+0000 7f8b3019c740 1 -- 172.21.15.97:0/95977743 >> 172.21.15.97:0/95977743 conn(0x559f6bd01310 msgr2=0x559f6bd04e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.745+0000 7f8b3019c740 1 -- 172.21.15.97:0/95977743 shutdown_connections 2024-03-26T19:10:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.745+0000 7f8b3019c740 1 -- 172.21.15.97:0/95977743 wait complete. 2024-03-26T19:10:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.745+0000 7f8b3019c740 1 -- start start 2024-03-26T19:10:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.746+0000 7f8b3019c740 1 Processor -- start 2024-03-26T19:10:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.749+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd12640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.749+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd12b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.749+0000 7f8b3019c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 0x559f6bd15b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.750+0000 7f8b25e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd12b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.750+0000 7f8b25e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd12b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47398/0 (socket says 172.21.15.97:47398) 2024-03-26T19:10:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.753+0000 7f8b26800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd12640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.753+0000 7f8b25e00640 1 -- 172.21.15.97:0/1288915191 learned_addr learned my addr 172.21.15.97:0/1288915191 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.753+0000 7f8b27200640 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 0x559f6bd15b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.753+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b281b0580 con 0x559f6bd06790 2024-03-26T19:10:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.753+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b281afeb0 con 0x559f6bd0b900 2024-03-26T19:10:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.753+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8b281ad0f0 con 0x559f6bd05da0 2024-03-26T19:10:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.753+0000 7f8b3019c740 10 client.0 _ll_register_callbacks cb 0x7f8b180b9870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:10:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.753+0000 7f8b26800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd12640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60138/0 (socket says 172.21.15.97:60138) 2024-03-26T19:10:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.754+0000 7f8b26800640 1 -- 172.21.15.97:0/1288915191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 msgr2=0x559f6bd15b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.754+0000 7f8b26800640 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 0x559f6bd15b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.754+0000 7f8b26800640 1 -- 172.21.15.97:0/1288915191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 msgr2=0x559f6bd12b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.754+0000 7f8b26800640 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd12b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.754+0000 7f8b26800640 1 -- 172.21.15.97:0/1288915191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6bcfc730 con 0x559f6bd05da0 2024-03-26T19:10:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.757+0000 7f8b26800640 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd12640 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x559f6bc137b0 tx=0x7f8b180c1650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e31fe1acf7ba56e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.757+0000 7f8b07e00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:10:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.757+0000 7f8b07e00640 1 -- 172.21.15.97:0/1288915191 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b180a0070 con 0x559f6bd05da0 2024-03-26T19:10:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.757+0000 7f8b07e00640 1 -- 172.21.15.97:0/1288915191 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8b180c5d60 con 0x559f6bd05da0 2024-03-26T19:10:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.757+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b18004c80 con 0x559f6bd05da0 2024-03-26T19:10:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.757+0000 7f8b07e00640 1 -- 172.21.15.97:0/1288915191 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b180d5050 con 0x559f6bd05da0 2024-03-26T19:10:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.758+0000 7f8b3019c740 10 client.26062 fetch_fsmap learned FSMap version 27 2024-03-26T19:10:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.758+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f6bcffab0 con 0x559f6bd05da0 2024-03-26T19:10:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.758+0000 7f8b07e00640 1 -- 172.21.15.97:0/1288915191 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b18004c80 con 0x559f6bd05da0 2024-03-26T19:10:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.758+0000 7f8b07e00640 1 -- 172.21.15.97:0/1288915191 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8b180bca20 con 0x559f6bd05da0 2024-03-26T19:10:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.759+0000 7f8b3019c740 10 client.26062 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:10:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.759+0000 7f8b3019c740 10 client.26062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:10:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.759+0000 7f8b3019c740 20 client.26062 populate_metadata read hostname 'smithi097' 2024-03-26T19:10:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.759+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559f6bd92f40 0x559f6bd95330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.759+0000 7f8b3019c740 4 client.26062 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:10:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.759+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f6bc72710 con 0x559f6bd92f40 2024-03-26T19:10:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.759+0000 7f8b25e00640 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559f6bd92f40 0x559f6bd95330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.760+0000 7f8b25e00640 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559f6bd92f40 0x559f6bd95330 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a786179b0ad9dcd5 server_cookie=f443215ef57f0f26 in_seq=0 out_seq=0 2024-03-26T19:10:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.760+0000 7f8b07e00640 10 client.26062 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.856+0000 7f8b07e00640 1 -- 172.21.15.97:0/1288915191 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f6bc72710 con 0x559f6bd92f40 2024-03-26T19:10:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.856+0000 7f8b07e00640 10 client.26062 handle_command_reply: tid=0 2024-03-26T19:10:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.856+0000 7f8b3019c740 1 client.26062 shutdown 2024-03-26T19:10:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.857+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 msgr2=0x559f6bd12640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.857+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd12640 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x559f6bc137b0 tx=0x7f8b180c1650 comp rx=0 tx=0).stop 2024-03-26T19:10:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.857+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 shutdown_connections 2024-03-26T19:10:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.857+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x559f6bd92f40 0x559f6bd95330 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.857+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f6bd0b900 0x559f6bd15b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.857+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f6bd06790 0x559f6bd12b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.857+0000 7f8b3019c740 1 --2- 172.21.15.97:0/1288915191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6bd05da0 0x559f6bd12640 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.857+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 >> 172.21.15.97:0/1288915191 conn(0x559f6bd01310 msgr2=0x559f6bcffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.857+0000 7f8b07e00640 0 client.26062 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.858+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 shutdown_connections 2024-03-26T19:10:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.858+0000 7f8b3019c740 1 -- 172.21.15.97:0/1288915191 wait complete. 2024-03-26T19:10:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.859+0000 7f8b3019c740 20 client.26062 trim_cache size 0 max 16384 2024-03-26T19:10:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:10:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:10:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:10:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:10:02.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:02 smithi097 ceph-mon[31955]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 668 op/s 2024-03-26T19:10:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.860+0000 7f8b2d400640 1 -- 172.21.15.97:0/4072048942 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8b0c0a31b0 msgr2=0x7f8b0c0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.860+0000 7f8b2d400640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8b0c0a31b0 0x7f8b0c0a5670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8b20007fd0 tx=0x7f8b20007b10 comp rx=0 tx=0).stop 2024-03-26T19:10:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.860+0000 7f8b2d400640 1 -- 172.21.15.97:0/4072048942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b28074320 msgr2=0x7f8b2819e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.860+0000 7f8b2d400640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b28074320 0x7f8b2819e0b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8b18005ab0 tx=0x7f8b1802c6c0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.861+0000 7f8b2d400640 1 -- 172.21.15.97:0/4072048942 shutdown_connections 2024-03-26T19:10:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.861+0000 7f8b2d400640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f8b0c0a31b0 0x7f8b0c0a5670 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.861+0000 7f8b2d400640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b281097b0 0x7f8b281a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.861+0000 7f8b2d400640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b28075900 0x7f8b2819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.861+0000 7f8b2d400640 1 --2- 172.21.15.97:0/4072048942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b28074320 0x7f8b2819e0b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.861+0000 7f8b2d400640 1 -- 172.21.15.97:0/4072048942 >> 172.21.15.97:0/4072048942 conn(0x7f8b280fce80 msgr2=0x7f8b281012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.861+0000 7f8b2d400640 1 -- 172.21.15.97:0/4072048942 shutdown_connections 2024-03-26T19:10:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:02.861+0000 7f8b2d400640 1 -- 172.21.15.97:0/4072048942 wait complete. 2024-03-26T19:10:04.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:04 smithi105 ceph-mon[36270]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-03-26T19:10:04.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:04 smithi097 ceph-mon[31955]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-03-26T19:10:04.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:04 smithi144 ceph-mon[36053]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-03-26T19:10:05.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 640.0 s, 1516.6 tps, lat 65.843 ms stddev 82.541 2024-03-26T19:10:06.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:06 smithi105 ceph-mon[36270]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 693 op/s 2024-03-26T19:10:06.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:06 smithi097 ceph-mon[31955]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 693 op/s 2024-03-26T19:10:06.881 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:06 smithi144 ceph-mon[36053]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 693 op/s 2024-03-26T19:10:08.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:08 smithi105 ceph-mon[36270]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-03-26T19:10:08.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:08 smithi097 ceph-mon[31955]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-03-26T19:10:08.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:08 smithi144 ceph-mon[36053]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-03-26T19:10:10.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 645.0 s, 1595.2 tps, lat 62.560 ms stddev 82.392 2024-03-26T19:10:10.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:10 smithi105 ceph-mon[36270]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.3 MiB/s wr, 652 op/s 2024-03-26T19:10:10.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:10 smithi097 ceph-mon[31955]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.3 MiB/s wr, 652 op/s 2024-03-26T19:10:10.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:10 smithi144 ceph-mon[36053]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.3 MiB/s wr, 652 op/s 2024-03-26T19:10:12.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:12 smithi105 ceph-mon[36270]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 724 op/s 2024-03-26T19:10:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:12 smithi097 ceph-mon[31955]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 724 op/s 2024-03-26T19:10:12.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:12 smithi144 ceph-mon[36053]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 724 op/s 2024-03-26T19:10:14.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:14 smithi144 ceph-mon[36053]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 590 op/s 2024-03-26T19:10:14.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:14 smithi105 ceph-mon[36270]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 590 op/s 2024-03-26T19:10:14.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:14 smithi097 ceph-mon[31955]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 590 op/s 2024-03-26T19:10:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 650.0 s, 1011.6 tps, lat 98.057 ms stddev 104.593 2024-03-26T19:10:15.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:15 smithi097 ceph-mon[31955]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.3 MiB/s wr, 701 op/s 2024-03-26T19:10:15.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:15 smithi144 ceph-mon[36053]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.3 MiB/s wr, 701 op/s 2024-03-26T19:10:15.966 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:15 smithi105 ceph-mon[36270]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.3 MiB/s wr, 701 op/s 2024-03-26T19:10:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.041+0000 7fd554000640 1 -- 172.21.15.97:0/2034402932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd54c102b00 msgr2=0x7fd54c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.041+0000 7fd554000640 1 --2- 172.21.15.97:0/2034402932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd54c102b00 0x7fd54c102f80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd538009900 tx=0x7fd53802bc50 comp rx=0 tx=0).stop 2024-03-26T19:10:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.042+0000 7fd554000640 1 -- 172.21.15.97:0/2034402932 shutdown_connections 2024-03-26T19:10:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.042+0000 7fd554000640 1 --2- 172.21.15.97:0/2034402932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c1034c0 0x7fd54c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.042+0000 7fd554000640 1 --2- 172.21.15.97:0/2034402932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd54c102b00 0x7fd54c102f80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.042+0000 7fd554000640 1 --2- 172.21.15.97:0/2034402932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c101910 0x7fd54c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.042+0000 7fd554000640 1 -- 172.21.15.97:0/2034402932 >> 172.21.15.97:0/2034402932 conn(0x7fd54c0fce80 msgr2=0x7fd54c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.042+0000 7fd554000640 1 -- 172.21.15.97:0/2034402932 shutdown_connections 2024-03-26T19:10:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.042+0000 7fd554000640 1 -- 172.21.15.97:0/2034402932 wait complete. 2024-03-26T19:10:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.043+0000 7fd554000640 1 Processor -- start 2024-03-26T19:10:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.043+0000 7fd554000640 1 -- start start 2024-03-26T19:10:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.044+0000 7fd554000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c101910 0x7fd54c19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.044+0000 7fd554000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd54c102b00 0x7fd54c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.044+0000 7fd554000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c1034c0 0x7fd54c1a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.044+0000 7fd554000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d5c0 con 0x7fd54c1034c0 2024-03-26T19:10:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.044+0000 7fd554000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d440 con 0x7fd54c102b00 2024-03-26T19:10:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.044+0000 7fd554000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d740 con 0x7fd54c101910 2024-03-26T19:10:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.044+0000 7fd551600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c101910 0x7fd54c19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.044+0000 7fd551600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c101910 0x7fd54c19e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:57222/0 (socket says 172.21.15.97:57222) 2024-03-26T19:10:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.044+0000 7fd551600640 1 -- 172.21.15.97:0/2904100232 learned_addr learned my addr 172.21.15.97:0/2904100232 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.045+0000 7fd550c00640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd54c102b00 0x7fd54c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.045+0000 7fd551600640 1 -- 172.21.15.97:0/2904100232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd54c102b00 msgr2=0x7fd54c19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.045+0000 7fd551600640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd54c102b00 0x7fd54c19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.045+0000 7fd551600640 1 -- 172.21.15.97:0/2904100232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c1034c0 msgr2=0x7fd54c1a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.045+0000 7fd551600640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c1034c0 0x7fd54c1a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.045+0000 7fd551600640 1 -- 172.21.15.97:0/2904100232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c1a5dd0 con 0x7fd54c101910 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.046+0000 7fd551600640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c101910 0x7fd54c19e1b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd54000e9c0 tx=0x7fd54000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.046+0000 7fd54a000640 1 -- 172.21.15.97:0/2904100232 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54000cde0 con 0x7fd54c101910 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.046+0000 7fd54a000640 1 -- 172.21.15.97:0/2904100232 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd540010910 con 0x7fd54c101910 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.046+0000 7fd554000640 1 -- 172.21.15.97:0/2904100232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c1a60c0 con 0x7fd54c101910 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.046+0000 7fd54a000640 1 -- 172.21.15.97:0/2904100232 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540018a70 con 0x7fd54c101910 2024-03-26T19:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.046+0000 7fd554000640 1 -- 172.21.15.97:0/2904100232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c1a6630 con 0x7fd54c101910 2024-03-26T19:10:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.050+0000 7fd54a000640 1 -- 172.21.15.97:0/2904100232 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fd54001f020 con 0x7fd54c101910 2024-03-26T19:10:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.051+0000 7fd556d68740 1 Processor -- start 2024-03-26T19:10:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.052+0000 7fd556d68740 1 -- start start 2024-03-26T19:10:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.052+0000 7fd54a000640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd5340a30c0 0x7fd5340a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.052+0000 7fd54a000640 1 -- 172.21.15.97:0/2904100232 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd5400cf780 con 0x7fd54c101910 2024-03-26T19:10:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.053+0000 7fd556d68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c53da0 0x560b51c541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.053+0000 7fd556d68740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c54790 0x560b51c59290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.053+0000 7fd556d68740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c59900 0x560b51c5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.053+0000 7fd556d68740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560b51bb8e80 con 0x560b51c59900 2024-03-26T19:10:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.053+0000 7fd556d68740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560b51b61660 con 0x560b51c53da0 2024-03-26T19:10:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.053+0000 7fd556d68740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560b51bb4e30 con 0x560b51c54790 2024-03-26T19:10:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.053+0000 7fd551600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c53da0 0x560b51c541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.053+0000 7fd551600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c53da0 0x560b51c541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:48680/0 (socket says 172.21.15.97:48680) 2024-03-26T19:10:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.053+0000 7fd551600640 1 -- 172.21.15.97:0/1258688556 learned_addr learned my addr 172.21.15.97:0/1258688556 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.053+0000 7fd550c00640 1 --2- 172.21.15.97:0/1258688556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c54790 0x560b51c59290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.054+0000 7fd552000640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd5340a30c0 0x7fd5340a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.054+0000 7fd551600640 1 -- 172.21.15.97:0/1258688556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c54790 msgr2=0x560b51c59290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.054+0000 7fd551600640 1 --2- 172.21.15.97:0/1258688556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c54790 0x560b51c59290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.054+0000 7fd551600640 1 -- 172.21.15.97:0/1258688556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c59900 msgr2=0x560b51c5bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.054+0000 7fd551600640 1 --2- 172.21.15.97:0/1258688556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c59900 0x560b51c5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.054+0000 7fd551600640 1 -- 172.21.15.97:0/1258688556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b518cd210 con 0x560b51c53da0 2024-03-26T19:10:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.054+0000 7fd551600640 1 --2- 172.21.15.97:0/1258688556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c53da0 0x560b51c541c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd5400a4360 tx=0x7fd54009c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44af1a99f53bcb91 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.055+0000 7fd53e600640 1 -- 172.21.15.97:0/1258688556 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400bc070 con 0x560b51c53da0 2024-03-26T19:10:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.055+0000 7fd53e600640 1 -- 172.21.15.97:0/1258688556 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5400a4890 con 0x560b51c53da0 2024-03-26T19:10:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.055+0000 7fd53e600640 1 -- 172.21.15.97:0/1258688556 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54009b2d0 con 0x560b51c53da0 2024-03-26T19:10:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.056+0000 7fd556d68740 1 -- 172.21.15.97:0/1258688556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c53da0 msgr2=0x560b51c541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.056+0000 7fd556d68740 1 --2- 172.21.15.97:0/1258688556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c53da0 0x560b51c541c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd5400a4360 tx=0x7fd54009c640 comp rx=0 tx=0).stop 2024-03-26T19:10:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.056+0000 7fd552000640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd5340a30c0 0x7fd5340a5580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd544005fd0 tx=0x7fd5440097b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.056+0000 7fd556d68740 1 -- 172.21.15.97:0/1258688556 shutdown_connections 2024-03-26T19:10:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.056+0000 7fd556d68740 1 --2- 172.21.15.97:0/1258688556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c59900 0x560b51c5bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.056+0000 7fd556d68740 1 --2- 172.21.15.97:0/1258688556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c54790 0x560b51c59290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.056+0000 7fd556d68740 1 --2- 172.21.15.97:0/1258688556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c53da0 0x560b51c541c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.056+0000 7fd556d68740 1 -- 172.21.15.97:0/1258688556 >> 172.21.15.97:0/1258688556 conn(0x560b51c4f310 msgr2=0x560b51c4f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.057+0000 7fd556d68740 1 -- 172.21.15.97:0/1258688556 shutdown_connections 2024-03-26T19:10:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.057+0000 7fd556d68740 1 -- 172.21.15.97:0/1258688556 wait complete. 2024-03-26T19:10:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.057+0000 7fd556d68740 1 -- start start 2024-03-26T19:10:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.057+0000 7fd556d68740 1 Processor -- start 2024-03-26T19:10:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.058+0000 7fd556d68740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.058+0000 7fd556d68740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 0x560b51c5e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.058+0000 7fd556d68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 0x560b51c65360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.058+0000 7fd556d68740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560b51c5c8b0 con 0x560b51c54790 2024-03-26T19:10:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.058+0000 7fd556d68740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560b51b74ac0 con 0x560b51c59900 2024-03-26T19:10:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.058+0000 7fd556d68740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560b51c5ca30 con 0x560b51c53da0 2024-03-26T19:10:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.058+0000 7fd556d68740 10 client.0 _ll_register_callbacks cb 0x560b51b74490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:10:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.058+0000 7fd551600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd551600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:57236/0 (socket says 172.21.15.97:57236) 2024-03-26T19:10:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd551600640 1 -- 172.21.15.97:0/2456669335 learned_addr learned my addr 172.21.15.97:0/2456669335 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd552000640 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 0x560b51c65360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd551600640 1 -- 172.21.15.97:0/2456669335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 msgr2=0x560b51c65360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd551600640 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 0x560b51c65360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd551600640 1 -- 172.21.15.97:0/2456669335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 msgr2=0x560b51c5e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd550c00640 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 0x560b51c5e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd551600640 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 0x560b51c5e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd551600640 1 -- 172.21.15.97:0/2456669335 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b51999620 con 0x560b51c53da0 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd552000640 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 0x560b51c65360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd550c00640 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 0x560b51c5e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd551600640 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5dde0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd5400982e0 tx=0x7fd54009b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d86f4a2983bfd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.059+0000 7fd532a00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.060+0000 7fd532a00640 1 -- 172.21.15.97:0/2456669335 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400b4070 con 0x560b51c53da0 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.060+0000 7fd532a00640 1 -- 172.21.15.97:0/2456669335 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd540010430 con 0x560b51c53da0 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.060+0000 7fd556d68740 1 -- 172.21.15.97:0/2456669335 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5400a4890 con 0x560b51c53da0 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.060+0000 7fd532a00640 1 -- 172.21.15.97:0/2456669335 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd5400106b0 con 0x560b51c53da0 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.060+0000 7fd556d68740 10 client.26074 fetch_fsmap learned FSMap version 27 2024-03-26T19:10:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.060+0000 7fd556d68740 1 -- 172.21.15.97:0/2456669335 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b51c5c900 con 0x560b51c53da0 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.060+0000 7fd532a00640 1 -- 172.21.15.97:0/2456669335 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5400a4890 con 0x560b51c53da0 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.060+0000 7fd532a00640 1 -- 172.21.15.97:0/2456669335 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd5400a1160 con 0x560b51c53da0 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.061+0000 7fd556d68740 10 client.26074 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.061+0000 7fd556d68740 10 client.26074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.061+0000 7fd556d68740 20 client.26074 populate_metadata read hostname 'smithi097' 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.061+0000 7fd556d68740 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560b51c698c0 0x560b51c6bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.061+0000 7fd556d68740 4 client.26074 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.061+0000 7fd556d68740 1 -- 172.21.15.97:0/2456669335 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b51bc63c0 con 0x560b51c698c0 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.061+0000 7fd550c00640 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560b51c698c0 0x560b51c6bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.061+0000 7fd532a00640 10 client.26074 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.061+0000 7fd550c00640 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560b51c698c0 0x560b51c6bcb0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9c4fa8b7f3424c server_cookie=1f54d629781d8649 in_seq=0 out_seq=0 2024-03-26T19:10:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.062+0000 7fd532a00640 1 -- 172.21.15.97:0/2456669335 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560b51bc63c0 con 0x560b51c698c0 2024-03-26T19:10:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.062+0000 7fd532a00640 10 client.26074 handle_command_reply: tid=0 2024-03-26T19:10:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.062+0000 7fd556d68740 1 client.26074 shutdown 2024-03-26T19:10:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.062+0000 7fd556d68740 1 -- 172.21.15.97:0/2456669335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 msgr2=0x560b51c5dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.062+0000 7fd556d68740 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5dde0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd5400982e0 tx=0x7fd54009b6d0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.063+0000 7fd556d68740 1 -- 172.21.15.97:0/2456669335 shutdown_connections 2024-03-26T19:10:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.063+0000 7fd556d68740 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560b51c698c0 0x560b51c6bcb0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.063+0000 7fd556d68740 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 0x560b51c65360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.063+0000 7fd556d68740 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 0x560b51c5e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.063+0000 7fd556d68740 1 --2- 172.21.15.97:0/2456669335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5dde0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.063+0000 7fd532a00640 0 client.26074 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.063+0000 7fd556d68740 1 -- 172.21.15.97:0/2456669335 >> 172.21.15.97:0/2456669335 conn(0x560b51c4f310 msgr2=0x560b51c4e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.063+0000 7fd556d68740 1 -- 172.21.15.97:0/2456669335 shutdown_connections 2024-03-26T19:10:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.063+0000 7fd556d68740 1 -- 172.21.15.97:0/2456669335 wait complete. 2024-03-26T19:10:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.063+0000 7fd556d68740 20 client.26074 trim_cache size 0 max 16384 2024-03-26T19:10:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.097+0000 7fd556d68740 1 Processor -- start 2024-03-26T19:10:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.097+0000 7fd556d68740 1 -- start start 2024-03-26T19:10:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.098+0000 7fd556d68740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.098+0000 7fd556d68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c54790 0x560b51c4dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.098+0000 7fd556d68740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c59900 0x560b51c4e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.098+0000 7fd556d68740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1af230 con 0x560b51c59900 2024-03-26T19:10:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.098+0000 7fd556d68740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1b0d70 con 0x560b51c54790 2024-03-26T19:10:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.098+0000 7fd556d68740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1af900 con 0x560b51c53da0 2024-03-26T19:10:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.098+0000 7fd550c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c54790 0x560b51c4dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.098+0000 7fd551600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.098+0000 7fd551600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:57250/0 (socket says 172.21.15.97:57250) 2024-03-26T19:10:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.098+0000 7fd551600640 1 -- 172.21.15.97:0/3958413228 learned_addr learned my addr 172.21.15.97:0/3958413228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.099+0000 7fd551600640 1 -- 172.21.15.97:0/3958413228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c54790 msgr2=0x560b51c4dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.099+0000 7fd551600640 1 --2- 172.21.15.97:0/3958413228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c54790 0x560b51c4dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.099+0000 7fd551600640 1 -- 172.21.15.97:0/3958413228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c59900 msgr2=0x560b51c4e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.099+0000 7fd551600640 1 --2- 172.21.15.97:0/3958413228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c59900 0x560b51c4e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.099+0000 7fd551600640 1 -- 172.21.15.97:0/3958413228 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b51c6c020 con 0x560b51c53da0 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.100+0000 7fd551600640 1 --2- 172.21.15.97:0/3958413228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5e100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd54000c6a0 tx=0x7fd54000efb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7295ab4835eb5a3d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.100+0000 7fd53dc00640 1 -- 172.21.15.97:0/3958413228 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400b4070 con 0x560b51c53da0 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.100+0000 7fd53dc00640 1 -- 172.21.15.97:0/3958413228 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5400a1160 con 0x560b51c53da0 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.100+0000 7fd53dc00640 1 -- 172.21.15.97:0/3958413228 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54009ad40 con 0x560b51c53da0 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.101+0000 7fd556d68740 1 -- 172.21.15.97:0/3958413228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 msgr2=0x560b51c5e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.101+0000 7fd556d68740 1 --2- 172.21.15.97:0/3958413228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5e100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd54000c6a0 tx=0x7fd54000efb0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.101+0000 7fd556d68740 1 -- 172.21.15.97:0/3958413228 shutdown_connections 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.101+0000 7fd556d68740 1 --2- 172.21.15.97:0/3958413228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c59900 0x560b51c4e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.101+0000 7fd556d68740 1 --2- 172.21.15.97:0/3958413228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c54790 0x560b51c4dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.101+0000 7fd556d68740 1 --2- 172.21.15.97:0/3958413228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c5e100 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.101+0000 7fd556d68740 1 -- 172.21.15.97:0/3958413228 >> 172.21.15.97:0/3958413228 conn(0x560b51c4f310 msgr2=0x560b51c5d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.102+0000 7fd556d68740 1 -- 172.21.15.97:0/3958413228 shutdown_connections 2024-03-26T19:10:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.102+0000 7fd556d68740 1 -- 172.21.15.97:0/3958413228 wait complete. 2024-03-26T19:10:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.102+0000 7fd556d68740 1 -- start start 2024-03-26T19:10:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.103+0000 7fd556d68740 1 Processor -- start 2024-03-26T19:10:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.103+0000 7fd556d68740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c60630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.104+0000 7fd556d68740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 0x560b51c60b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.104+0000 7fd551600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c60630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.104+0000 7fd551600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c60630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:57258/0 (socket says 172.21.15.97:57258) 2024-03-26T19:10:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.104+0000 7fd551600640 1 -- 172.21.15.97:0/3176512598 learned_addr learned my addr 172.21.15.97:0/3176512598 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.104+0000 7fd556d68740 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 0x560b51c63b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.104+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1b06a0 con 0x560b51c54790 2024-03-26T19:10:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.104+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1affd0 con 0x560b51c59900 2024-03-26T19:10:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.104+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1ad210 con 0x560b51c53da0 2024-03-26T19:10:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.105+0000 7fd556d68740 10 client.0 _ll_register_callbacks cb 0x7fd54009b5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:10:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.105+0000 7fd550c00640 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 0x560b51c60b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.105+0000 7fd552000640 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 0x560b51c63b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.106+0000 7fd551600640 1 -- 172.21.15.97:0/3176512598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 msgr2=0x560b51c63b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.106+0000 7fd551600640 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 0x560b51c63b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.106+0000 7fd551600640 1 -- 172.21.15.97:0/3176512598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 msgr2=0x560b51c60b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.106+0000 7fd551600640 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 0x560b51c60b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.107+0000 7fd551600640 1 -- 172.21.15.97:0/3176512598 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b51c64180 con 0x560b51c53da0 2024-03-26T19:10:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.107+0000 7fd552000640 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 0x560b51c63b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:10:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.107+0000 7fd551600640 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c60630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd54009be10 tx=0x7fd540098780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d662f33e9b2f97e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.107+0000 7fd532a00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:10:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.107+0000 7fd532a00640 1 -- 172.21.15.97:0/3176512598 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400b4070 con 0x560b51c53da0 2024-03-26T19:10:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.108+0000 7fd532a00640 1 -- 172.21.15.97:0/3176512598 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd540097bd0 con 0x560b51c53da0 2024-03-26T19:10:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.108+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5400a1160 con 0x560b51c53da0 2024-03-26T19:10:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.108+0000 7fd532a00640 1 -- 172.21.15.97:0/3176512598 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd5400b4070 con 0x560b51c53da0 2024-03-26T19:10:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.108+0000 7fd556d68740 10 client.26080 fetch_fsmap learned FSMap version 27 2024-03-26T19:10:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.108+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b51c64180 con 0x560b51c53da0 2024-03-26T19:10:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.109+0000 7fd532a00640 1 -- 172.21.15.97:0/3176512598 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5400a1160 con 0x560b51c53da0 2024-03-26T19:10:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.109+0000 7fd532a00640 1 -- 172.21.15.97:0/3176512598 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd540085030 con 0x560b51c53da0 2024-03-26T19:10:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.109+0000 7fd556d68740 10 client.26080 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:10:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.109+0000 7fd556d68740 10 client.26080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:10:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.110+0000 7fd556d68740 20 client.26080 populate_metadata read hostname 'smithi097' 2024-03-26T19:10:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.110+0000 7fd556d68740 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560b51ce1010 0x560b51ce3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.110+0000 7fd556d68740 4 client.26080 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:10:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.110+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b51bc0710 con 0x560b51ce1010 2024-03-26T19:10:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.110+0000 7fd550c00640 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560b51ce1010 0x560b51ce3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.111+0000 7fd550c00640 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560b51ce1010 0x560b51ce3430 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30fe6372d66d4dcc server_cookie=be8057cd3e0a5a80 in_seq=0 out_seq=0 2024-03-26T19:10:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.111+0000 7fd532a00640 10 client.26080 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.264+0000 7fd532a00640 1 -- 172.21.15.97:0/3176512598 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b51bc0710 con 0x560b51ce1010 2024-03-26T19:10:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.264+0000 7fd532a00640 10 client.26080 handle_command_reply: tid=0 2024-03-26T19:10:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.265+0000 7fd556d68740 1 client.26080 shutdown 2024-03-26T19:10:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.265+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 msgr2=0x560b51c60630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.265+0000 7fd556d68740 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c60630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd54009be10 tx=0x7fd540098780 comp rx=0 tx=0).stop 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.265+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 shutdown_connections 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.265+0000 7fd556d68740 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x560b51ce1010 0x560b51ce3430 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.265+0000 7fd532a00640 0 client.26080 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.265+0000 7fd556d68740 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560b51c59900 0x560b51c63b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.265+0000 7fd556d68740 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b51c54790 0x560b51c60b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.265+0000 7fd556d68740 1 --2- 172.21.15.97:0/3176512598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b51c53da0 0x560b51c60630 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.265+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 >> 172.21.15.97:0/3176512598 conn(0x560b51c4f310 msgr2=0x560b51c55190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.266+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 shutdown_connections 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.266+0000 7fd556d68740 1 -- 172.21.15.97:0/3176512598 wait complete. 2024-03-26T19:10:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.266+0000 7fd556d68740 20 client.26080 trim_cache size 0 max 16384 2024-03-26T19:10:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:10:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:10:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:10:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.267+0000 7fd554000640 1 -- 172.21.15.97:0/2904100232 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd5340a30c0 msgr2=0x7fd5340a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.267+0000 7fd554000640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd5340a30c0 0x7fd5340a5580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd544005fd0 tx=0x7fd5440097b0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.267+0000 7fd554000640 1 -- 172.21.15.97:0/2904100232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c101910 msgr2=0x7fd54c19e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.267+0000 7fd554000640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c101910 0x7fd54c19e1b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd54000e9c0 tx=0x7fd54000ee80 comp rx=0 tx=0).stop 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.268+0000 7fd554000640 1 -- 172.21.15.97:0/2904100232 shutdown_connections 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.268+0000 7fd554000640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd5340a30c0 0x7fd5340a5580 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.268+0000 7fd554000640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd54c1034c0 0x7fd54c1a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.268+0000 7fd554000640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd54c102b00 0x7fd54c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.268+0000 7fd554000640 1 --2- 172.21.15.97:0/2904100232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c101910 0x7fd54c19e1b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.268+0000 7fd554000640 1 -- 172.21.15.97:0/2904100232 >> 172.21.15.97:0/2904100232 conn(0x7fd54c0fce80 msgr2=0x7fd54c103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.268+0000 7fd554000640 1 -- 172.21.15.97:0/2904100232 shutdown_connections 2024-03-26T19:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:16.268+0000 7fd554000640 1 -- 172.21.15.97:0/2904100232 wait complete. 2024-03-26T19:10:18.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:18 smithi097 ceph-mon[31955]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 577 op/s 2024-03-26T19:10:18.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:18 smithi144 ceph-mon[36053]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 577 op/s 2024-03-26T19:10:18.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:18 smithi105 ceph-mon[36270]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 577 op/s 2024-03-26T19:10:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 655.0 s, 1378.0 tps, lat 73.379 ms stddev 89.865 2024-03-26T19:10:20.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:20 smithi097 ceph-mon[31955]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 616 op/s 2024-03-26T19:10:20.673 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:20 smithi144 ceph-mon[36053]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 616 op/s 2024-03-26T19:10:20.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:20 smithi105 ceph-mon[36270]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 616 op/s 2024-03-26T19:10:22.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:22 smithi097 ceph-mon[31955]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 680 op/s 2024-03-26T19:10:22.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:22 smithi144 ceph-mon[36053]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 680 op/s 2024-03-26T19:10:22.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:22 smithi105 ceph-mon[36270]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 680 op/s 2024-03-26T19:10:24.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:24 smithi097 ceph-mon[31955]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 559 op/s 2024-03-26T19:10:24.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:24 smithi144 ceph-mon[36053]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 559 op/s 2024-03-26T19:10:24.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:24 smithi105 ceph-mon[36270]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 559 op/s 2024-03-26T19:10:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 660.0 s, 1523.2 tps, lat 65.617 ms stddev 81.203 2024-03-26T19:10:26.608 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:26 smithi097 ceph-mon[31955]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 687 op/s 2024-03-26T19:10:26.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:26 smithi144 ceph-mon[36053]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 687 op/s 2024-03-26T19:10:26.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:26 smithi105 ceph-mon[36270]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 687 op/s 2024-03-26T19:10:28.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:28 smithi144 ceph-mon[36053]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 575 op/s 2024-03-26T19:10:28.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:28 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:10:28.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:28 smithi105 ceph-mon[36270]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 575 op/s 2024-03-26T19:10:28.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:28 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:10:28.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:28 smithi097 ceph-mon[31955]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 575 op/s 2024-03-26T19:10:28.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:28 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:10:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.474+0000 7f3b74c00640 1 -- 172.21.15.97:0/1642385315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b701073f0 msgr2=0x7f3b70109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.474+0000 7f3b74c00640 1 --2- 172.21.15.97:0/1642385315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b701073f0 0x7f3b70109840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009ca0 tx=0x7f3b6802c0f0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.475+0000 7f3b74c00640 1 -- 172.21.15.97:0/1642385315 shutdown_connections 2024-03-26T19:10:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.476+0000 7f3b74c00640 1 --2- 172.21.15.97:0/1642385315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b701073f0 0x7f3b70109840 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.476+0000 7f3b74c00640 1 --2- 172.21.15.97:0/1642385315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b70104ac0 0x7f3b70106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.476+0000 7f3b74c00640 1 --2- 172.21.15.97:0/1642385315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b70102190 0x7f3b70104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.476+0000 7f3b74c00640 1 -- 172.21.15.97:0/1642385315 >> 172.21.15.97:0/1642385315 conn(0x7f3b700fbba0 msgr2=0x7f3b700fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.476+0000 7f3b74c00640 1 -- 172.21.15.97:0/1642385315 shutdown_connections 2024-03-26T19:10:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.477+0000 7f3b74c00640 1 -- 172.21.15.97:0/1642385315 wait complete. 2024-03-26T19:10:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.478+0000 7f3b74c00640 1 Processor -- start 2024-03-26T19:10:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.478+0000 7f3b74c00640 1 -- start start 2024-03-26T19:10:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b70102190 0x7f3b70110be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b70104ac0 0x7f3b70111120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b701073f0 0x7f3b701adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b74c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70101d20 con 0x7f3b70102190 2024-03-26T19:10:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b701073f0 0x7f3b701adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b701073f0 0x7f3b701adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57542/0 (socket says 172.21.15.97:57542) 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b6fe00640 1 -- 172.21.15.97:0/710687237 learned_addr learned my addr 172.21.15.97:0/710687237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b70104ac0 0x7f3b70111120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b74c00640 1 -- 172.21.15.97:0/710687237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70101ba0 con 0x7f3b701073f0 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b74c00640 1 -- 172.21.15.97:0/710687237 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70101ea0 con 0x7f3b70104ac0 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.479+0000 7f3b6f400640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b70102190 0x7f3b70110be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.480+0000 7f3b6ea00640 1 -- 172.21.15.97:0/710687237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b701073f0 msgr2=0x7f3b701adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.480+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b701073f0 0x7f3b701adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.480+0000 7f3b6ea00640 1 -- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b70102190 msgr2=0x7f3b70110be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.480+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b70102190 0x7f3b70110be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.480+0000 7f3b6ea00640 1 -- 172.21.15.97:0/710687237 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b701ae540 con 0x7f3b70104ac0 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.480+0000 7f3b6f400640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b70102190 0x7f3b70110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:10:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.480+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b70104ac0 0x7f3b70111120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00e9c0 tx=0x7f3b5c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.480+0000 7f3b6fe00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b701073f0 0x7f3b701adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.481+0000 7f3b67e00640 1 -- 172.21.15.97:0/710687237 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00ccc0 con 0x7f3b70104ac0 2024-03-26T19:10:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.481+0000 7f3b67e00640 1 -- 172.21.15.97:0/710687237 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c005020 con 0x7f3b70104ac0 2024-03-26T19:10:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.481+0000 7f3b67e00640 1 -- 172.21.15.97:0/710687237 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c004a00 con 0x7f3b70104ac0 2024-03-26T19:10:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.481+0000 7f3b74c00640 1 -- 172.21.15.97:0/710687237 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b701ae830 con 0x7f3b70104ac0 2024-03-26T19:10:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.481+0000 7f3b74c00640 1 -- 172.21.15.97:0/710687237 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b701aed30 con 0x7f3b70104ac0 2024-03-26T19:10:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.486+0000 7f3b79014740 1 Processor -- start 2024-03-26T19:10:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.486+0000 7f3b67e00640 1 -- 172.21.15.97:0/710687237 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f3b5c005560 con 0x7f3b70104ac0 2024-03-26T19:10:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.488+0000 7f3b79014740 1 -- start start 2024-03-26T19:10:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.488+0000 7f3b79014740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318f9f60 0x55ed318fa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.488+0000 7f3b79014740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 0x55ed318ff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.488+0000 7f3b79014740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed31901f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.488+0000 7f3b79014740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed3185ee80 con 0x55ed318f9f60 2024-03-26T19:10:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.488+0000 7f3b79014740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ed31807660 con 0x55ed318ffac0 2024-03-26T19:10:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.488+0000 7f3b79014740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ed3185ae30 con 0x55ed318fa950 2024-03-26T19:10:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.489+0000 7f3b6ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed31901f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.489+0000 7f3b6ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed31901f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57546/0 (socket says 172.21.15.97:57546) 2024-03-26T19:10:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.489+0000 7f3b6ea00640 1 -- 172.21.15.97:0/3483616698 learned_addr learned my addr 172.21.15.97:0/3483616698 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.489+0000 7f3b6f400640 1 --2- 172.21.15.97:0/3483616698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 0x55ed318ff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.489+0000 7f3b6ea00640 1 -- 172.21.15.97:0/3483616698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 msgr2=0x55ed318ff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.489+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/3483616698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 0x55ed318ff450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.489+0000 7f3b6ea00640 1 -- 172.21.15.97:0/3483616698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318f9f60 msgr2=0x55ed318fa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.489+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/3483616698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318f9f60 0x55ed318fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.489+0000 7f3b6ea00640 1 -- 172.21.15.97:0/3483616698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed31573210 con 0x55ed318ffac0 2024-03-26T19:10:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.489+0000 7f3b6f400640 1 --2- 172.21.15.97:0/3483616698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 0x55ed318ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.490+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/3483616698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed31901f50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0d7590 tx=0x7f3b5c0d7680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64dfeb61be58acc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.491+0000 7f3b5be00640 1 -- 172.21.15.97:0/3483616698 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0d1520 con 0x55ed318ffac0 2024-03-26T19:10:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.491+0000 7f3b5be00640 1 -- 172.21.15.97:0/3483616698 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c0cfd20 con 0x55ed318ffac0 2024-03-26T19:10:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.492+0000 7f3b5be00640 1 -- 172.21.15.97:0/3483616698 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0d1520 con 0x55ed318ffac0 2024-03-26T19:10:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.492+0000 7f3b67e00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b540a3010 0x7f3b540a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.493+0000 7f3b67e00640 1 -- 172.21.15.97:0/710687237 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3b5c0cf490 con 0x7f3b70104ac0 2024-03-26T19:10:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.493+0000 7f3b79014740 1 -- 172.21.15.97:0/3483616698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 msgr2=0x55ed31901f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.493+0000 7f3b79014740 1 --2- 172.21.15.97:0/3483616698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed31901f50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0d7590 tx=0x7f3b5c0d7680 comp rx=0 tx=0).stop 2024-03-26T19:10:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.493+0000 7f3b79014740 1 -- 172.21.15.97:0/3483616698 shutdown_connections 2024-03-26T19:10:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.493+0000 7f3b79014740 1 --2- 172.21.15.97:0/3483616698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed31901f50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.493+0000 7f3b79014740 1 --2- 172.21.15.97:0/3483616698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 0x55ed318ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.493+0000 7f3b79014740 1 --2- 172.21.15.97:0/3483616698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318f9f60 0x55ed318fa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.493+0000 7f3b79014740 1 -- 172.21.15.97:0/3483616698 >> 172.21.15.97:0/3483616698 conn(0x55ed318f54d0 msgr2=0x55ed318f58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.494+0000 7f3b79014740 1 -- 172.21.15.97:0/3483616698 shutdown_connections 2024-03-26T19:10:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.494+0000 7f3b79014740 1 -- 172.21.15.97:0/3483616698 wait complete. 2024-03-26T19:10:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.494+0000 7f3b6fe00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b540a3010 0x7f3b540a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.494+0000 7f3b79014740 1 -- start start 2024-03-26T19:10:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.495+0000 7f3b79014740 1 Processor -- start 2024-03-26T19:10:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.496+0000 7f3b6fe00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b540a3010 0x7f3b540a54d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3b68006010 tx=0x7f3b680099a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.496+0000 7f3b79014740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 0x55ed31903f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.496+0000 7f3b79014740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318fa950 0x55ed319044c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.496+0000 7f3b79014740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed3190b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.496+0000 7f3b79014740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed31902a70 con 0x55ed318fa950 2024-03-26T19:10:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.496+0000 7f3b79014740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ed3181aac0 con 0x55ed318ffac0 2024-03-26T19:10:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.497+0000 7f3b79014740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ed31902bf0 con 0x55ed318f9f60 2024-03-26T19:10:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.497+0000 7f3b79014740 10 client.0 _ll_register_callbacks cb 0x55ed3181a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:10:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.497+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 0x55ed31903f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.497+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed3190b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.498+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed3190b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57560/0 (socket says 172.21.15.97:57560) 2024-03-26T19:10:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.498+0000 7f3b6fe00640 1 -- 172.21.15.97:0/2592121236 learned_addr learned my addr 172.21.15.97:0/2592121236 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.498+0000 7f3b6fe00640 1 -- 172.21.15.97:0/2592121236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 msgr2=0x55ed31903f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.498+0000 7f3b6fe00640 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 0x55ed31903f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.499+0000 7f3b6fe00640 1 -- 172.21.15.97:0/2592121236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318fa950 msgr2=0x55ed319044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.499+0000 7f3b6fe00640 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318fa950 0x55ed319044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.499+0000 7f3b6fe00640 1 -- 172.21.15.97:0/2592121236 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed3163f620 con 0x55ed318ffac0 2024-03-26T19:10:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.500+0000 7f3b6fe00640 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed3190b500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b680326c0 tx=0x7f3b6802c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc0e9a7c759a04 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.500+0000 7f3b58c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:10:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.500+0000 7f3b58c00640 1 -- 172.21.15.97:0/2592121236 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68045050 con 0x55ed318ffac0 2024-03-26T19:10:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.500+0000 7f3b58c00640 1 -- 172.21.15.97:0/2592121236 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3b6803f670 con 0x55ed318ffac0 2024-03-26T19:10:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.500+0000 7f3b79014740 1 -- 172.21.15.97:0/2592121236 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b5c0cfd20 con 0x55ed318ffac0 2024-03-26T19:10:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.501+0000 7f3b58c00640 1 -- 172.21.15.97:0/2592121236 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b68039040 con 0x55ed318ffac0 2024-03-26T19:10:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.501+0000 7f3b79014740 10 client.26114 fetch_fsmap learned FSMap version 27 2024-03-26T19:10:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.501+0000 7f3b79014740 1 -- 172.21.15.97:0/2592121236 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed3191f860 con 0x55ed318ffac0 2024-03-26T19:10:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.502+0000 7f3b58c00640 1 -- 172.21.15.97:0/2592121236 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c0cfd20 con 0x55ed318ffac0 2024-03-26T19:10:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.502+0000 7f3b58c00640 1 -- 172.21.15.97:0/2592121236 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3b6804ab30 con 0x55ed318ffac0 2024-03-26T19:10:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.502+0000 7f3b79014740 10 client.26114 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:10:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.502+0000 7f3b79014740 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:10:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.502+0000 7f3b79014740 20 client.26114 populate_metadata read hostname 'smithi097' 2024-03-26T19:10:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.503+0000 7f3b79014740 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ed31920f40 0x55ed31923370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.503+0000 7f3b79014740 4 client.26114 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:10:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.503+0000 7f3b79014740 1 -- 172.21.15.97:0/2592121236 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed3186c3c0 con 0x55ed31920f40 2024-03-26T19:10:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.503+0000 7f3b6f400640 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ed31920f40 0x55ed31923370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.504+0000 7f3b58c00640 10 client.26114 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.504+0000 7f3b6f400640 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ed31920f40 0x55ed31923370 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7326beb79cccc5e7 server_cookie=1c7f512fe35966c2 in_seq=0 out_seq=0 2024-03-26T19:10:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b58c00640 1 -- 172.21.15.97:0/2592121236 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ed3186c3c0 con 0x55ed31920f40 2024-03-26T19:10:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b58c00640 10 client.26114 handle_command_reply: tid=0 2024-03-26T19:10:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b79014740 1 client.26114 shutdown 2024-03-26T19:10:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b79014740 1 -- 172.21.15.97:0/2592121236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 msgr2=0x55ed3190b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b79014740 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed3190b500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b680326c0 tx=0x7f3b6802c0f0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b79014740 1 -- 172.21.15.97:0/2592121236 shutdown_connections 2024-03-26T19:10:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b79014740 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ed31920f40 0x55ed31923370 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b79014740 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed3190b500 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b79014740 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318fa950 0x55ed319044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b79014740 1 --2- 172.21.15.97:0/2592121236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 0x55ed31903f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b58c00640 0 client.26114 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.505+0000 7f3b79014740 1 -- 172.21.15.97:0/2592121236 >> 172.21.15.97:0/2592121236 conn(0x55ed318f54d0 msgr2=0x55ed318f46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.506+0000 7f3b79014740 1 -- 172.21.15.97:0/2592121236 shutdown_connections 2024-03-26T19:10:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.506+0000 7f3b79014740 1 -- 172.21.15.97:0/2592121236 wait complete. 2024-03-26T19:10:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.507+0000 7f3b79014740 20 client.26114 trim_cache size 0 max 16384 2024-03-26T19:10:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.543+0000 7f3b79014740 1 Processor -- start 2024-03-26T19:10:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.544+0000 7f3b79014740 1 -- start start 2024-03-26T19:10:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.544+0000 7f3b79014740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 0x55ed318f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.544+0000 7f3b79014740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318fa950 0x55ed3190b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.544+0000 7f3b79014740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed318f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.544+0000 7f3b79014740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701af230 con 0x55ed318fa950 2024-03-26T19:10:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.544+0000 7f3b79014740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701b0d70 con 0x55ed318ffac0 2024-03-26T19:10:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.544+0000 7f3b79014740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701af900 con 0x55ed318f9f60 2024-03-26T19:10:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.545+0000 7f3b6ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318fa950 0x55ed3190b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.545+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 0x55ed318f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.545+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 0x55ed318f3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:38718/0 (socket says 172.21.15.97:38718) 2024-03-26T19:10:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.545+0000 7f3b6f400640 1 -- 172.21.15.97:0/301323195 learned_addr learned my addr 172.21.15.97:0/301323195 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.545+0000 7f3b6fe00640 1 --2- 172.21.15.97:0/301323195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed318f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.546+0000 7f3b6f400640 1 -- 172.21.15.97:0/301323195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 msgr2=0x55ed318f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.546+0000 7f3b6f400640 1 --2- 172.21.15.97:0/301323195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed318f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.546+0000 7f3b6f400640 1 -- 172.21.15.97:0/301323195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318fa950 msgr2=0x55ed3190b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.546+0000 7f3b6f400640 1 --2- 172.21.15.97:0/301323195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318fa950 0x55ed3190b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.546+0000 7f3b6f400640 1 -- 172.21.15.97:0/301323195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed318f0ae0 con 0x55ed318f9f60 2024-03-26T19:10:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.546+0000 7f3b6fe00640 1 --2- 172.21.15.97:0/301323195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed318f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:10:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.547+0000 7f3b6f400640 1 --2- 172.21.15.97:0/301323195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 0x55ed318f3e60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55ed31923bf0 tx=0x7f3b600093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e4209a88a09ad23 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.547+0000 7f3b5b400640 1 -- 172.21.15.97:0/301323195 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000f040 con 0x55ed318f9f60 2024-03-26T19:10:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.547+0000 7f3b5b400640 1 -- 172.21.15.97:0/301323195 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b6001f030 con 0x55ed318f9f60 2024-03-26T19:10:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.547+0000 7f3b5b400640 1 -- 172.21.15.97:0/301323195 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b600143b0 con 0x55ed318f9f60 2024-03-26T19:10:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.548+0000 7f3b79014740 1 -- 172.21.15.97:0/301323195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 msgr2=0x55ed318f3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.548+0000 7f3b79014740 1 --2- 172.21.15.97:0/301323195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 0x55ed318f3e60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55ed31923bf0 tx=0x7f3b600093e0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.548+0000 7f3b79014740 1 -- 172.21.15.97:0/301323195 shutdown_connections 2024-03-26T19:10:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.548+0000 7f3b79014740 1 --2- 172.21.15.97:0/301323195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318ffac0 0x55ed318f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.548+0000 7f3b79014740 1 --2- 172.21.15.97:0/301323195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318fa950 0x55ed3190b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.548+0000 7f3b79014740 1 --2- 172.21.15.97:0/301323195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318f9f60 0x55ed318f3e60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.548+0000 7f3b79014740 1 -- 172.21.15.97:0/301323195 >> 172.21.15.97:0/301323195 conn(0x55ed318f54d0 msgr2=0x55ed318f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.549+0000 7f3b79014740 1 -- 172.21.15.97:0/301323195 shutdown_connections 2024-03-26T19:10:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.549+0000 7f3b79014740 1 -- 172.21.15.97:0/301323195 wait complete. 2024-03-26T19:10:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.549+0000 7f3b79014740 1 -- start start 2024-03-26T19:10:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.549+0000 7f3b79014740 1 Processor -- start 2024-03-26T19:10:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.550+0000 7f3b79014740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318f9f60 0x55ed31908ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.550+0000 7f3b79014740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 0x55ed319091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.550+0000 7f3b79014740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318ffac0 0x55ed3190a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.550+0000 7f3b79014740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701b06a0 con 0x55ed318ffac0 2024-03-26T19:10:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.550+0000 7f3b79014740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701affd0 con 0x55ed318f9f60 2024-03-26T19:10:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.550+0000 7f3b79014740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70071d10 con 0x55ed318fa950 2024-03-26T19:10:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.550+0000 7f3b79014740 10 client.0 _ll_register_callbacks cb 0x7f3b68009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:10:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.550+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318f9f60 0x55ed31908ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.550+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318f9f60 0x55ed31908ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57580/0 (socket says 172.21.15.97:57580) 2024-03-26T19:10:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.550+0000 7f3b6f400640 1 -- 172.21.15.97:0/3239977292 learned_addr learned my addr 172.21.15.97:0/3239977292 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.551+0000 7f3b6fe00640 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318ffac0 0x55ed3190a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.551+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 0x55ed319091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.551+0000 7f3b6f400640 1 -- 172.21.15.97:0/3239977292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 msgr2=0x55ed319091e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.551+0000 7f3b6f400640 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 0x55ed319091e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.551+0000 7f3b6f400640 1 -- 172.21.15.97:0/3239977292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318ffac0 msgr2=0x55ed3190a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.551+0000 7f3b6f400640 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318ffac0 0x55ed3190a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.551+0000 7f3b6f400640 1 -- 172.21.15.97:0/3239977292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed3190a7e0 con 0x55ed318f9f60 2024-03-26T19:10:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.552+0000 7f3b6fe00640 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318ffac0 0x55ed3190a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.552+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 0x55ed319091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.552+0000 7f3b6f400640 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318f9f60 0x55ed31908ca0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009800 tx=0x7f3b60023e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2278a9e8b5d3bd5d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.552+0000 7f3b58c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:10:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.556+0000 7f3b58c00640 1 -- 172.21.15.97:0/3239977292 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60016c30 con 0x55ed318f9f60 2024-03-26T19:10:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.556+0000 7f3b58c00640 1 -- 172.21.15.97:0/3239977292 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3b600233d0 con 0x55ed318f9f60 2024-03-26T19:10:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.557+0000 7f3b79014740 1 -- 172.21.15.97:0/3239977292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b6001f030 con 0x55ed318f9f60 2024-03-26T19:10:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.559+0000 7f3b58c00640 1 -- 172.21.15.97:0/3239977292 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b60016dd0 con 0x55ed318f9f60 2024-03-26T19:10:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.563+0000 7f3b79014740 10 client.26120 fetch_fsmap learned FSMap version 27 2024-03-26T19:10:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.563+0000 7f3b79014740 1 -- 172.21.15.97:0/3239977292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed31987370 con 0x55ed318f9f60 2024-03-26T19:10:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.563+0000 7f3b58c00640 1 -- 172.21.15.97:0/3239977292 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b6001f030 con 0x55ed318f9f60 2024-03-26T19:10:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.563+0000 7f3b58c00640 1 -- 172.21.15.97:0/3239977292 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3b6002e3e0 con 0x55ed318f9f60 2024-03-26T19:10:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.564+0000 7f3b79014740 10 client.26120 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:10:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.564+0000 7f3b79014740 10 client.26120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:10:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.564+0000 7f3b79014740 20 client.26120 populate_metadata read hostname 'smithi097' 2024-03-26T19:10:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.564+0000 7f3b79014740 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ed31987c90 0x55ed3198a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.564+0000 7f3b79014740 4 client.26120 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:10:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.564+0000 7f3b79014740 1 -- 172.21.15.97:0/3239977292 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed31866710 con 0x55ed31987c90 2024-03-26T19:10:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.564+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ed31987c90 0x55ed3198a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.565+0000 7f3b58c00640 10 client.26120 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:29.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.565+0000 7f3b6ea00640 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ed31987c90 0x55ed3198a0b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90203846261e759d server_cookie=41ac4980256b08f6 in_seq=0 out_seq=0 2024-03-26T19:10:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.674+0000 7f3b58c00640 1 -- 172.21.15.97:0/3239977292 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed31866710 con 0x55ed31987c90 2024-03-26T19:10:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.674+0000 7f3b58c00640 10 client.26120 handle_command_reply: tid=0 2024-03-26T19:10:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.674+0000 7f3b79014740 1 client.26120 shutdown 2024-03-26T19:10:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.675+0000 7f3b79014740 1 -- 172.21.15.97:0/3239977292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318f9f60 msgr2=0x55ed31908ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.675+0000 7f3b79014740 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318f9f60 0x55ed31908ca0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009800 tx=0x7f3b60023e50 comp rx=0 tx=0).stop 2024-03-26T19:10:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.675+0000 7f3b79014740 1 -- 172.21.15.97:0/3239977292 shutdown_connections 2024-03-26T19:10:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.675+0000 7f3b79014740 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ed31987c90 0x55ed3198a0b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.675+0000 7f3b79014740 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed318ffac0 0x55ed3190a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.675+0000 7f3b79014740 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed318fa950 0x55ed319091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.675+0000 7f3b58c00640 0 client.26120 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.675+0000 7f3b79014740 1 --2- 172.21.15.97:0/3239977292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed318f9f60 0x55ed31908ca0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.675+0000 7f3b79014740 1 -- 172.21.15.97:0/3239977292 >> 172.21.15.97:0/3239977292 conn(0x55ed318f54d0 msgr2=0x55ed318fb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.676+0000 7f3b79014740 1 -- 172.21.15.97:0/3239977292 shutdown_connections 2024-03-26T19:10:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.676+0000 7f3b79014740 1 -- 172.21.15.97:0/3239977292 wait complete. 2024-03-26T19:10:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.677+0000 7f3b79014740 20 client.26120 trim_cache size 0 max 16384 2024-03-26T19:10:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:10:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:10:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:10:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:10:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.678+0000 7f3b74c00640 1 -- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b540a3010 msgr2=0x7f3b540a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.678+0000 7f3b74c00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b540a3010 0x7f3b540a54d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3b68006010 tx=0x7f3b680099a0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.678+0000 7f3b74c00640 1 -- 172.21.15.97:0/710687237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b70104ac0 msgr2=0x7f3b70111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.678+0000 7f3b74c00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b70104ac0 0x7f3b70111120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00e9c0 tx=0x7f3b5c00ee80 comp rx=0 tx=0).stop 2024-03-26T19:10:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.678+0000 7f3b74c00640 1 -- 172.21.15.97:0/710687237 shutdown_connections 2024-03-26T19:10:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.678+0000 7f3b74c00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b540a3010 0x7f3b540a54d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.679+0000 7f3b74c00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b701073f0 0x7f3b701adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.679+0000 7f3b74c00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b70104ac0 0x7f3b70111120 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.679+0000 7f3b74c00640 1 --2- 172.21.15.97:0/710687237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b70102190 0x7f3b70110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.679+0000 7f3b74c00640 1 -- 172.21.15.97:0/710687237 >> 172.21.15.97:0/710687237 conn(0x7f3b700fbba0 msgr2=0x7f3b70102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.679+0000 7f3b74c00640 1 -- 172.21.15.97:0/710687237 shutdown_connections 2024-03-26T19:10:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:29.680+0000 7f3b74c00640 1 -- 172.21.15.97:0/710687237 wait complete. 2024-03-26T19:10:30.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 665.0 s, 1544.2 tps, lat 64.368 ms stddev 78.289 2024-03-26T19:10:30.132 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:10:30.132 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:10:30.132 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:29 smithi144 ceph-mon[36053]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 582 op/s 2024-03-26T19:10:30.216 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:10:30.216 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:10:30.216 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:29 smithi105 ceph-mon[36270]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 582 op/s 2024-03-26T19:10:30.358 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:10:30.358 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:10:30.359 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:29 smithi097 ceph-mon[31955]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 582 op/s 2024-03-26T19:10:31.216 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:10:31.216 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:10:31.216 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:10:31.359 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:10:31.359 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:10:31.359 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:10:31.382 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:10:31.382 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:10:31.382 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:10:32.216 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:31 smithi105 ceph-mon[36270]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 690 op/s 2024-03-26T19:10:32.261 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:31 smithi097 ceph-mon[31955]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 690 op/s 2024-03-26T19:10:32.382 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:31 smithi144 ceph-mon[36053]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 690 op/s 2024-03-26T19:10:34.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:34 smithi097 ceph-mon[31955]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 MiB/s wr, 551 op/s 2024-03-26T19:10:34.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:34 smithi144 ceph-mon[36053]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 MiB/s wr, 551 op/s 2024-03-26T19:10:34.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:34 smithi105 ceph-mon[36270]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 MiB/s wr, 551 op/s 2024-03-26T19:10:35.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 670.0 s, 1349.6 tps, lat 73.289 ms stddev 89.340 2024-03-26T19:10:36.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:36 smithi144 ceph-mon[36053]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 MiB/s wr, 698 op/s 2024-03-26T19:10:36.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:36 smithi105 ceph-mon[36270]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 MiB/s wr, 698 op/s 2024-03-26T19:10:36.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:36 smithi097 ceph-mon[31955]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 MiB/s wr, 698 op/s 2024-03-26T19:10:38.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:38 smithi144 ceph-mon[36053]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.3 MiB/s wr, 552 op/s 2024-03-26T19:10:38.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:38 smithi105 ceph-mon[36270]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.3 MiB/s wr, 552 op/s 2024-03-26T19:10:38.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:38 smithi097 ceph-mon[31955]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.3 MiB/s wr, 552 op/s 2024-03-26T19:10:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 675.0 s, 1142.3 tps, lat 88.718 ms stddev 103.100 2024-03-26T19:10:40.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:40 smithi144 ceph-mon[36053]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 627 op/s 2024-03-26T19:10:40.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:40 smithi105 ceph-mon[36270]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 627 op/s 2024-03-26T19:10:40.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:40 smithi097 ceph-mon[31955]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 627 op/s 2024-03-26T19:10:42.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:42 smithi144 ceph-mon[36053]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 713 op/s 2024-03-26T19:10:42.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:42 smithi105 ceph-mon[36270]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 713 op/s 2024-03-26T19:10:42.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:42 smithi097 ceph-mon[31955]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 713 op/s 2024-03-26T19:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.864+0000 7efc3a000640 1 -- 172.21.15.97:0/20601565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc34075900 msgr2=0x7efc34073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.864+0000 7efc3a000640 1 --2- 172.21.15.97:0/20601565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc34075900 0x7efc34073d50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efc24009ca0 tx=0x7efc2402c120 comp rx=0 tx=0).stop 2024-03-26T19:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.864+0000 7efc3a000640 1 -- 172.21.15.97:0/20601565 shutdown_connections 2024-03-26T19:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.864+0000 7efc3a000640 1 --2- 172.21.15.97:0/20601565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc3410dec0 0x7efc341102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.864+0000 7efc3a000640 1 --2- 172.21.15.97:0/20601565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc34074320 0x7efc340747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.864+0000 7efc3a000640 1 --2- 172.21.15.97:0/20601565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc34075900 0x7efc34073d50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.864+0000 7efc3a000640 1 -- 172.21.15.97:0/20601565 >> 172.21.15.97:0/20601565 conn(0x7efc340fce80 msgr2=0x7efc340ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.864+0000 7efc3a000640 1 -- 172.21.15.97:0/20601565 shutdown_connections 2024-03-26T19:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.864+0000 7efc3a000640 1 -- 172.21.15.97:0/20601565 wait complete. 2024-03-26T19:10:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.866+0000 7efc3a000640 1 Processor -- start 2024-03-26T19:10:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.867+0000 7efc3a000640 1 -- start start 2024-03-26T19:10:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.867+0000 7efc3a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc34074320 0x7efc3419e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.867+0000 7efc3a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc34075900 0x7efc3419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc33400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc34074320 0x7efc3419e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc3a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3410dec0 0x7efc341a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc3a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc34113c60 con 0x7efc34075900 2024-03-26T19:10:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc3a000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc34113ae0 con 0x7efc34074320 2024-03-26T19:10:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc3a000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc34113de0 con 0x7efc3410dec0 2024-03-26T19:10:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc33400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc34074320 0x7efc3419e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:50962/0 (socket says 172.21.15.97:50962) 2024-03-26T19:10:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc33400640 1 -- 172.21.15.97:0/2918642923 learned_addr learned my addr 172.21.15.97:0/2918642923 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc33e00640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3410dec0 0x7efc341a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc33e00640 1 -- 172.21.15.97:0/2918642923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc34074320 msgr2=0x7efc3419e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc33e00640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc34074320 0x7efc3419e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc33e00640 1 -- 172.21.15.97:0/2918642923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc34075900 msgr2=0x7efc3419e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc33e00640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc34075900 0x7efc3419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.868+0000 7efc33e00640 1 -- 172.21.15.97:0/2918642923 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc341a5e20 con 0x7efc3410dec0 2024-03-26T19:10:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.869+0000 7efc33e00640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3410dec0 0x7efc341a57b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00b410 tx=0x7efc2c00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.869+0000 7efc2be00640 1 -- 172.21.15.97:0/2918642923 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c015020 con 0x7efc3410dec0 2024-03-26T19:10:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.869+0000 7efc2be00640 1 -- 172.21.15.97:0/2918642923 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc2c011d10 con 0x7efc3410dec0 2024-03-26T19:10:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.870+0000 7efc2be00640 1 -- 172.21.15.97:0/2918642923 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00f2e0 con 0x7efc3410dec0 2024-03-26T19:10:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.870+0000 7efc3a000640 1 -- 172.21.15.97:0/2918642923 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc341a6110 con 0x7efc3410dec0 2024-03-26T19:10:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.870+0000 7efc3a000640 1 -- 172.21.15.97:0/2918642923 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc341a66e0 con 0x7efc3410dec0 2024-03-26T19:10:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.876+0000 7efc2be00640 1 -- 172.21.15.97:0/2918642923 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7efc2c018020 con 0x7efc3410dec0 2024-03-26T19:10:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.877+0000 7efc3cde5740 1 Processor -- start 2024-03-26T19:10:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.877+0000 7efc3cde5740 1 -- start start 2024-03-26T19:10:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.878+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 0x55ccf00a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.878+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a7790 0x55ccf00ac290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.878+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.878+0000 7efc3cde5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ccf000be80 con 0x55ccf00a7790 2024-03-26T19:10:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.878+0000 7efc3cde5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cceffb4660 con 0x55ccf00a6da0 2024-03-26T19:10:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.878+0000 7efc3cde5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ccf0007e30 con 0x55ccf00ac900 2024-03-26T19:10:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.878+0000 7efc32a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 0x55ccf00a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.878+0000 7efc32a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 0x55ccf00a71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:50968/0 (socket says 172.21.15.97:50968) 2024-03-26T19:10:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.878+0000 7efc32a00640 1 -- 172.21.15.97:0/2591246909 learned_addr learned my addr 172.21.15.97:0/2591246909 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.878+0000 7efc32a00640 1 --2- 172.21.15.97:0/2591246909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.879+0000 7efc32a00640 1 -- 172.21.15.97:0/2591246909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 msgr2=0x55ccf00aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.879+0000 7efc32a00640 1 --2- 172.21.15.97:0/2591246909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.879+0000 7efc32a00640 1 -- 172.21.15.97:0/2591246909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a7790 msgr2=0x55ccf00ac290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:10:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.879+0000 7efc32a00640 1 --2- 172.21.15.97:0/2591246909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a7790 0x55ccf00ac290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.879+0000 7efc32a00640 1 -- 172.21.15.97:0/2591246909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccefd20210 con 0x55ccf00a6da0 2024-03-26T19:10:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.879+0000 7efc32a00640 1 --2- 172.21.15.97:0/2591246909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 0x55ccf00a71c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efc200153a0 tx=0x7efc20009db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff93b511765e23da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.880+0000 7efc2be00640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efc180a3040 0x7efc180a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.880+0000 7efc1fe00640 1 -- 172.21.15.97:0/2591246909 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20012040 con 0x55ccf00a6da0 2024-03-26T19:10:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.880+0000 7efc1fe00640 1 -- 172.21.15.97:0/2591246909 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc20008de0 con 0x55ccf00a6da0 2024-03-26T19:10:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.880+0000 7efc1fe00640 1 -- 172.21.15.97:0/2591246909 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20004c80 con 0x55ccf00a6da0 2024-03-26T19:10:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.880+0000 7efc2be00640 1 -- 172.21.15.97:0/2918642923 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efc2c0ce880 con 0x7efc3410dec0 2024-03-26T19:10:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.881+0000 7efc33400640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efc180a3040 0x7efc180a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.882+0000 7efc3cde5740 1 -- 172.21.15.97:0/2591246909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 msgr2=0x55ccf00a71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.882+0000 7efc3cde5740 1 --2- 172.21.15.97:0/2591246909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 0x55ccf00a71c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efc200153a0 tx=0x7efc20009db0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.882+0000 7efc3cde5740 1 -- 172.21.15.97:0/2591246909 shutdown_connections 2024-03-26T19:10:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.882+0000 7efc3cde5740 1 --2- 172.21.15.97:0/2591246909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.882+0000 7efc3cde5740 1 --2- 172.21.15.97:0/2591246909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a7790 0x55ccf00ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.882+0000 7efc3cde5740 1 --2- 172.21.15.97:0/2591246909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 0x55ccf00a71c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.882+0000 7efc3cde5740 1 -- 172.21.15.97:0/2591246909 >> 172.21.15.97:0/2591246909 conn(0x55ccf00a2310 msgr2=0x55ccf00a2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.883+0000 7efc33400640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efc180a3040 0x7efc180a5500 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc240098d0 tx=0x7efc24032380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.883+0000 7efc3cde5740 1 -- 172.21.15.97:0/2591246909 shutdown_connections 2024-03-26T19:10:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.883+0000 7efc3cde5740 1 -- 172.21.15.97:0/2591246909 wait complete. 2024-03-26T19:10:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.883+0000 7efc3cde5740 1 -- start start 2024-03-26T19:10:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.885+0000 7efc3cde5740 1 Processor -- start 2024-03-26T19:10:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.886+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 0x55ccf00b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.886+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a7790 0x55ccf00b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.886+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00b8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.886+0000 7efc3cde5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ccf00af8b0 con 0x55ccf00a6da0 2024-03-26T19:10:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.886+0000 7efc3cde5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cceffc7ac0 con 0x55ccf00a7790 2024-03-26T19:10:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.886+0000 7efc3cde5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ccf00afa30 con 0x55ccf00ac900 2024-03-26T19:10:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.886+0000 7efc3cde5740 10 client.0 _ll_register_callbacks cb 0x55cceffc7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:10:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.887+0000 7efc32a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a7790 0x55ccf00b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.887+0000 7efc32a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a7790 0x55ccf00b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:50978/0 (socket says 172.21.15.97:50978) 2024-03-26T19:10:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.887+0000 7efc32a00640 1 -- 172.21.15.97:0/1507557849 learned_addr learned my addr 172.21.15.97:0/1507557849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.888+0000 7efc33e00640 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.888+0000 7efc33400640 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 0x55ccf00b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.889+0000 7efc32a00640 1 -- 172.21.15.97:0/1507557849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 msgr2=0x55ccf00b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.889+0000 7efc32a00640 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.889+0000 7efc32a00640 1 -- 172.21.15.97:0/1507557849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 msgr2=0x55ccf00b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.889+0000 7efc32a00640 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 0x55ccf00b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.889+0000 7efc32a00640 1 -- 172.21.15.97:0/1507557849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccefdec620 con 0x55ccf00a7790 2024-03-26T19:10:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.889+0000 7efc32a00640 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a7790 0x55ccf00b1340 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc20015cf0 tx=0x7efc200017b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1b0f51acace86e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.889+0000 7efc1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:10:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.890+0000 7efc1cc00640 1 -- 172.21.15.97:0/1507557849 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20008900 con 0x55ccf00a7790 2024-03-26T19:10:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.890+0000 7efc3cde5740 1 -- 172.21.15.97:0/1507557849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc20008de0 con 0x55ccf00a7790 2024-03-26T19:10:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.890+0000 7efc1cc00640 1 -- 172.21.15.97:0/1507557849 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efc20008aa0 con 0x55ccf00a7790 2024-03-26T19:10:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.890+0000 7efc1cc00640 1 -- 172.21.15.97:0/1507557849 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc2002b050 con 0x55ccf00a7790 2024-03-26T19:10:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.890+0000 7efc3cde5740 10 client.26129 fetch_fsmap learned FSMap version 27 2024-03-26T19:10:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.890+0000 7efc3cde5740 1 -- 172.21.15.97:0/1507557849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ccf00bb6f0 con 0x55ccf00a7790 2024-03-26T19:10:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.891+0000 7efc1cc00640 1 -- 172.21.15.97:0/1507557849 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc20008de0 con 0x55ccf00a7790 2024-03-26T19:10:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.892+0000 7efc1cc00640 1 -- 172.21.15.97:0/1507557849 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efc200070f0 con 0x55ccf00a7790 2024-03-26T19:10:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.892+0000 7efc3cde5740 10 client.26129 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:10:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.892+0000 7efc3cde5740 10 client.26129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:10:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.892+0000 7efc3cde5740 20 client.26129 populate_metadata read hostname 'smithi097' 2024-03-26T19:10:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.892+0000 7efc3cde5740 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ccf00bcb40 0x55ccf00bef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.893+0000 7efc3cde5740 4 client.26129 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:10:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.893+0000 7efc3cde5740 1 -- 172.21.15.97:0/1507557849 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccf00193c0 con 0x55ccf00bcb40 2024-03-26T19:10:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.893+0000 7efc33400640 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ccf00bcb40 0x55ccf00bef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.893+0000 7efc33400640 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ccf00bcb40 0x55ccf00bef30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9940d71c7ad42c7b server_cookie=bee84b268d9d7f4a in_seq=0 out_seq=0 2024-03-26T19:10:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.893+0000 7efc1cc00640 10 client.26129 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.894+0000 7efc1cc00640 1 -- 172.21.15.97:0/1507557849 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ccf00193c0 con 0x55ccf00bcb40 2024-03-26T19:10:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.894+0000 7efc1cc00640 10 client.26129 handle_command_reply: tid=0 2024-03-26T19:10:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.894+0000 7efc3cde5740 1 client.26129 shutdown 2024-03-26T19:10:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.895+0000 7efc3cde5740 1 -- 172.21.15.97:0/1507557849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a7790 msgr2=0x55ccf00b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.895+0000 7efc3cde5740 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a7790 0x55ccf00b1340 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc20015cf0 tx=0x7efc200017b0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.895+0000 7efc3cde5740 1 -- 172.21.15.97:0/1507557849 shutdown_connections 2024-03-26T19:10:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.895+0000 7efc3cde5740 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ccf00bcb40 0x55ccf00bef30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.895+0000 7efc3cde5740 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.895+0000 7efc3cde5740 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a7790 0x55ccf00b1340 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.895+0000 7efc3cde5740 1 --2- 172.21.15.97:0/1507557849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 0x55ccf00b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.895+0000 7efc3cde5740 1 -- 172.21.15.97:0/1507557849 >> 172.21.15.97:0/1507557849 conn(0x55ccf00a2310 msgr2=0x55ccf00a1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.895+0000 7efc1cc00640 0 client.26129 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.895+0000 7efc3cde5740 1 -- 172.21.15.97:0/1507557849 shutdown_connections 2024-03-26T19:10:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.896+0000 7efc3cde5740 1 -- 172.21.15.97:0/1507557849 wait complete. 2024-03-26T19:10:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.896+0000 7efc3cde5740 20 client.26129 trim_cache size 0 max 16384 2024-03-26T19:10:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.929+0000 7efc3cde5740 1 Processor -- start 2024-03-26T19:10:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.931+0000 7efc3cde5740 1 -- start start 2024-03-26T19:10:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.931+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 0x55ccf00b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.931+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a7790 0x55ccf00b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.931+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00a5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.931+0000 7efc3cde5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc341af370 con 0x55ccf00a7790 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.931+0000 7efc3cde5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc341b0eb0 con 0x55ccf00a6da0 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.931+0000 7efc3cde5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc341afa40 con 0x55ccf00ac900 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.932+0000 7efc33e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.932+0000 7efc33e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:45284/0 (socket says 172.21.15.97:45284) 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.932+0000 7efc33e00640 1 -- 172.21.15.97:0/1524897233 learned_addr learned my addr 172.21.15.97:0/1524897233 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.932+0000 7efc33e00640 1 -- 172.21.15.97:0/1524897233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 msgr2=0x55ccf00b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.932+0000 7efc33e00640 1 --2- 172.21.15.97:0/1524897233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 0x55ccf00b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.932+0000 7efc33e00640 1 -- 172.21.15.97:0/1524897233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a7790 msgr2=0x55ccf00b8340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.934+0000 7efc32a00640 1 --2- 172.21.15.97:0/1524897233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a7790 0x55ccf00b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.935+0000 7efc33e00640 1 --2- 172.21.15.97:0/1524897233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a7790 0x55ccf00b8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.935+0000 7efc33e00640 1 -- 172.21.15.97:0/1524897233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf00abab0 con 0x55ccf00ac900 2024-03-26T19:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.937+0000 7efc32a00640 1 --2- 172.21.15.97:0/1524897233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a7790 0x55ccf00b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.937+0000 7efc33e00640 1 --2- 172.21.15.97:0/1524897233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00a5570 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efc2c011040 tx=0x7efc2c010780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f41dae51c0cc263e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.937+0000 7efc1f400640 1 -- 172.21.15.97:0/1524897233 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0a39f0 con 0x55ccf00ac900 2024-03-26T19:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.937+0000 7efc1f400640 1 -- 172.21.15.97:0/1524897233 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc2c010510 con 0x55ccf00ac900 2024-03-26T19:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.937+0000 7efc1f400640 1 -- 172.21.15.97:0/1524897233 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c095b50 con 0x55ccf00ac900 2024-03-26T19:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.938+0000 7efc3cde5740 1 -- 172.21.15.97:0/1524897233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 msgr2=0x55ccf00a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.938+0000 7efc3cde5740 1 --2- 172.21.15.97:0/1524897233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00a5570 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efc2c011040 tx=0x7efc2c010780 comp rx=0 tx=0).stop 2024-03-26T19:10:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.941+0000 7efc3cde5740 1 -- 172.21.15.97:0/1524897233 shutdown_connections 2024-03-26T19:10:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.941+0000 7efc3cde5740 1 --2- 172.21.15.97:0/1524897233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00ac900 0x55ccf00a5570 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.941+0000 7efc3cde5740 1 --2- 172.21.15.97:0/1524897233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a7790 0x55ccf00b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.941+0000 7efc3cde5740 1 --2- 172.21.15.97:0/1524897233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00a6da0 0x55ccf00b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.941+0000 7efc3cde5740 1 -- 172.21.15.97:0/1524897233 >> 172.21.15.97:0/1524897233 conn(0x55ccf00a2310 msgr2=0x55ccf00b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.941+0000 7efc3cde5740 1 -- 172.21.15.97:0/1524897233 shutdown_connections 2024-03-26T19:10:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.941+0000 7efc3cde5740 1 -- 172.21.15.97:0/1524897233 wait complete. 2024-03-26T19:10:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.942+0000 7efc3cde5740 1 -- start start 2024-03-26T19:10:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.943+0000 7efc3cde5740 1 Processor -- start 2024-03-26T19:10:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.943+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 0x55ccf00b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.943+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00a7790 0x55ccf00b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.943+0000 7efc3cde5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00ac900 0x55ccf00b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.943+0000 7efc3cde5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc341b07e0 con 0x55ccf00a6da0 2024-03-26T19:10:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.943+0000 7efc3cde5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc341b0110 con 0x55ccf00ac900 2024-03-26T19:10:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.943+0000 7efc3cde5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc341ad350 con 0x55ccf00a7790 2024-03-26T19:10:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.943+0000 7efc3cde5740 10 client.0 _ll_register_callbacks cb 0x7efc20007650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:10:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.944+0000 7efc32a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00a7790 0x55ccf00b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.944+0000 7efc32a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00a7790 0x55ccf00b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:45290/0 (socket says 172.21.15.97:45290) 2024-03-26T19:10:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.944+0000 7efc32a00640 1 -- 172.21.15.97:0/3267713614 learned_addr learned my addr 172.21.15.97:0/3267713614 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.944+0000 7efc33400640 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 0x55ccf00b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.944+0000 7efc33e00640 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00ac900 0x55ccf00b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.945+0000 7efc33400640 1 -- 172.21.15.97:0/3267713614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00a7790 msgr2=0x55ccf00b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.945+0000 7efc33400640 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00a7790 0x55ccf00b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.945+0000 7efc33400640 1 -- 172.21.15.97:0/3267713614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00ac900 msgr2=0x55ccf00b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.945+0000 7efc33400640 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00ac900 0x55ccf00b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.945+0000 7efc33400640 1 -- 172.21.15.97:0/3267713614 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf00b71a0 con 0x55ccf00a6da0 2024-03-26T19:10:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.945+0000 7efc33400640 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 0x55ccf00b56b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc24032660 tx=0x7efc24032af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de27b504091f152b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.945+0000 7efc1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T19:10:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.947+0000 7efc1cc00640 1 -- 172.21.15.97:0/3267713614 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24039040 con 0x55ccf00a6da0 2024-03-26T19:10:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.947+0000 7efc1cc00640 1 -- 172.21.15.97:0/3267713614 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efc24051460 con 0x55ccf00a6da0 2024-03-26T19:10:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.948+0000 7efc3cde5740 1 -- 172.21.15.97:0/3267713614 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc2c010510 con 0x55ccf00a6da0 2024-03-26T19:10:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.950+0000 7efc1cc00640 1 -- 172.21.15.97:0/3267713614 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc24051660 con 0x55ccf00a6da0 2024-03-26T19:10:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.950+0000 7efc3cde5740 10 client.15723 fetch_fsmap learned FSMap version 27 2024-03-26T19:10:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.950+0000 7efc3cde5740 1 -- 172.21.15.97:0/3267713614 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ccf00b8490 con 0x55ccf00a6da0 2024-03-26T19:10:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.951+0000 7efc1cc00640 1 -- 172.21.15.97:0/3267713614 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc2c010510 con 0x55ccf00a6da0 2024-03-26T19:10:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.952+0000 7efc1cc00640 1 -- 172.21.15.97:0/3267713614 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efc2402dce0 con 0x55ccf00a6da0 2024-03-26T19:10:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.952+0000 7efc3cde5740 10 client.15723 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:10:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.952+0000 7efc3cde5740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.952+0000 7efc3cde5740 20 client.15723 populate_metadata read hostname 'smithi097' 2024-03-26T19:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.952+0000 7efc3cde5740 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ccf01343a0 0x55ccf01367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.952+0000 7efc3cde5740 4 client.15723 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.952+0000 7efc3cde5740 1 -- 172.21.15.97:0/3267713614 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccf0013710 con 0x55ccf01343a0 2024-03-26T19:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.953+0000 7efc32a00640 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ccf01343a0 0x55ccf01367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.957+0000 7efc32a00640 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ccf01343a0 0x55ccf01367c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce596c72e85b9665 server_cookie=f4a2775cda0ee4f4 in_seq=0 out_seq=0 2024-03-26T19:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:42.957+0000 7efc1cc00640 10 client.15723 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc1cc00640 1 -- 172.21.15.97:0/3267713614 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ccf0013710 con 0x55ccf01343a0 2024-03-26T19:10:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc1cc00640 10 client.15723 handle_command_reply: tid=0 2024-03-26T19:10:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc3cde5740 1 client.15723 shutdown 2024-03-26T19:10:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc3cde5740 1 -- 172.21.15.97:0/3267713614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 msgr2=0x55ccf00b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc3cde5740 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 0x55ccf00b56b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc24032660 tx=0x7efc24032af0 comp rx=0 tx=0).stop 2024-03-26T19:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc3cde5740 1 -- 172.21.15.97:0/3267713614 shutdown_connections 2024-03-26T19:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc3cde5740 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ccf01343a0 0x55ccf01367c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc3cde5740 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccf00ac900 0x55ccf00b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc3cde5740 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccf00a7790 0x55ccf00b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc3cde5740 1 --2- 172.21.15.97:0/3267713614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf00a6da0 0x55ccf00b56b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc3cde5740 1 -- 172.21.15.97:0/3267713614 >> 172.21.15.97:0/3267713614 conn(0x55ccf00a2310 msgr2=0x55ccf00a5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.047+0000 7efc1cc00640 0 client.15723 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.048+0000 7efc3cde5740 1 -- 172.21.15.97:0/3267713614 shutdown_connections 2024-03-26T19:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.048+0000 7efc3cde5740 1 -- 172.21.15.97:0/3267713614 wait complete. 2024-03-26T19:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.048+0000 7efc3cde5740 20 client.15723 trim_cache size 0 max 16384 2024-03-26T19:10:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:10:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:10:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:10:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.049+0000 7efc3a000640 1 -- 172.21.15.97:0/2918642923 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efc180a3040 msgr2=0x7efc180a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.049+0000 7efc3a000640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efc180a3040 0x7efc180a5500 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc240098d0 tx=0x7efc24032380 comp rx=0 tx=0).stop 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.049+0000 7efc3a000640 1 -- 172.21.15.97:0/2918642923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3410dec0 msgr2=0x7efc341a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.049+0000 7efc3a000640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3410dec0 0x7efc341a57b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00b410 tx=0x7efc2c00b8d0 comp rx=0 tx=0).stop 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.049+0000 7efc3a000640 1 -- 172.21.15.97:0/2918642923 shutdown_connections 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.049+0000 7efc3a000640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efc180a3040 0x7efc180a5500 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.049+0000 7efc3a000640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3410dec0 0x7efc341a57b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.049+0000 7efc3a000640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc34075900 0x7efc3419e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.049+0000 7efc3a000640 1 --2- 172.21.15.97:0/2918642923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc34074320 0x7efc3419e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.049+0000 7efc3a000640 1 -- 172.21.15.97:0/2918642923 >> 172.21.15.97:0/2918642923 conn(0x7efc340fce80 msgr2=0x7efc34101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.050+0000 7efc3a000640 1 -- 172.21.15.97:0/2918642923 shutdown_connections 2024-03-26T19:10:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:43.050+0000 7efc3a000640 1 -- 172.21.15.97:0/2918642923 wait complete. 2024-03-26T19:10:44.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:44 smithi105 ceph-mon[36270]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.5 MiB/s wr, 595 op/s 2024-03-26T19:10:44.733 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:44 smithi097 ceph-mon[31955]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.5 MiB/s wr, 595 op/s 2024-03-26T19:10:44.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:44 smithi144 ceph-mon[36053]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.5 MiB/s wr, 595 op/s 2024-03-26T19:10:45.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 680.0 s, 1267.5 tps, lat 79.023 ms stddev 96.770 2024-03-26T19:10:46.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:46 smithi105 ceph-mon[36270]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 657 op/s 2024-03-26T19:10:46.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:46 smithi097 ceph-mon[31955]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 657 op/s 2024-03-26T19:10:46.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:46 smithi144 ceph-mon[36053]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 657 op/s 2024-03-26T19:10:48.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:48 smithi105 ceph-mon[36270]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 MiB/s wr, 573 op/s 2024-03-26T19:10:48.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:48 smithi097 ceph-mon[31955]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 MiB/s wr, 573 op/s 2024-03-26T19:10:48.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:48 smithi144 ceph-mon[36053]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 MiB/s wr, 573 op/s 2024-03-26T19:10:50.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 685.0 s, 1373.4 tps, lat 73.226 ms stddev 85.861 2024-03-26T19:10:50.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:50 smithi105 ceph-mon[36270]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 612 op/s 2024-03-26T19:10:50.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:50 smithi097 ceph-mon[31955]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 612 op/s 2024-03-26T19:10:50.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:50 smithi144 ceph-mon[36053]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 612 op/s 2024-03-26T19:10:52.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:52 smithi105 ceph-mon[36270]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 677 op/s 2024-03-26T19:10:52.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:52 smithi097 ceph-mon[31955]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 677 op/s 2024-03-26T19:10:52.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:52 smithi144 ceph-mon[36053]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 677 op/s 2024-03-26T19:10:54.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:54 smithi105 ceph-mon[36270]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.9 MiB/s wr, 550 op/s 2024-03-26T19:10:54.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:54 smithi097 ceph-mon[31955]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.9 MiB/s wr, 550 op/s 2024-03-26T19:10:54.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:54 smithi144 ceph-mon[36053]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.9 MiB/s wr, 550 op/s 2024-03-26T19:10:55.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 690.0 s, 1172.2 tps, lat 83.913 ms stddev 95.033 2024-03-26T19:10:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.242+0000 7eff11a00640 1 -- 172.21.15.97:0/2272993951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff0c102ae0 msgr2=0x7eff0c102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.242+0000 7eff11a00640 1 --2- 172.21.15.97:0/2272993951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff0c102ae0 0x7eff0c102f60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efefc009900 tx=0x7efefc02bc30 comp rx=0 tx=0).stop 2024-03-26T19:10:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.245+0000 7eff11a00640 1 -- 172.21.15.97:0/2272993951 shutdown_connections 2024-03-26T19:10:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.245+0000 7eff11a00640 1 --2- 172.21.15.97:0/2272993951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff0c1034a0 0x7eff0c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.245+0000 7eff11a00640 1 --2- 172.21.15.97:0/2272993951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff0c102ae0 0x7eff0c102f60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.245+0000 7eff11a00640 1 --2- 172.21.15.97:0/2272993951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0c1018f0 0x7eff0c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.245+0000 7eff11a00640 1 -- 172.21.15.97:0/2272993951 >> 172.21.15.97:0/2272993951 conn(0x7eff0c0fce80 msgr2=0x7eff0c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.245+0000 7eff11a00640 1 -- 172.21.15.97:0/2272993951 shutdown_connections 2024-03-26T19:10:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.245+0000 7eff11a00640 1 -- 172.21.15.97:0/2272993951 wait complete. 2024-03-26T19:10:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.247+0000 7eff11a00640 1 Processor -- start 2024-03-26T19:10:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.247+0000 7eff11a00640 1 -- start start 2024-03-26T19:10:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff11a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0c1018f0 0x7eff0c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff11a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff0c102ae0 0x7eff0c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff11a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff0c1034a0 0x7eff0c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff11a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c10d640 con 0x7eff0c102ae0 2024-03-26T19:10:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff11a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c10d4c0 con 0x7eff0c1034a0 2024-03-26T19:10:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff11a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c10d7c0 con 0x7eff0c1018f0 2024-03-26T19:10:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff0aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0c1018f0 0x7eff0c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff0b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff0c1034a0 0x7eff0c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff0aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0c1018f0 0x7eff0c199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:43494/0 (socket says 172.21.15.97:43494) 2024-03-26T19:10:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff0aa00640 1 -- 172.21.15.97:0/293482302 learned_addr learned my addr 172.21.15.97:0/293482302 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.248+0000 7eff0a000640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff0c102ae0 0x7eff0c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.249+0000 7eff0aa00640 1 -- 172.21.15.97:0/293482302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff0c1034a0 msgr2=0x7eff0c1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.249+0000 7eff0aa00640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff0c1034a0 0x7eff0c1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.249+0000 7eff0aa00640 1 -- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff0c102ae0 msgr2=0x7eff0c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.249+0000 7eff0aa00640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff0c102ae0 0x7eff0c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.249+0000 7eff0aa00640 1 -- 172.21.15.97:0/293482302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c1a19d0 con 0x7eff0c1018f0 2024-03-26T19:10:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.249+0000 7eff0b400640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff0c1034a0 0x7eff0c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:10:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.249+0000 7eff0a000640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff0c102ae0 0x7eff0c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:10:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.249+0000 7eff0aa00640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0c1018f0 0x7eff0c199dc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efef800e9c0 tx=0x7efef800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.250+0000 7eff03400640 1 -- 172.21.15.97:0/293482302 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef800cd80 con 0x7eff0c1018f0 2024-03-26T19:10:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.250+0000 7eff03400640 1 -- 172.21.15.97:0/293482302 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef80035f0 con 0x7eff0c1018f0 2024-03-26T19:10:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.250+0000 7eff11a00640 1 -- 172.21.15.97:0/293482302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c1a1cc0 con 0x7eff0c1018f0 2024-03-26T19:10:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.250+0000 7eff03400640 1 -- 172.21.15.97:0/293482302 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8010690 con 0x7eff0c1018f0 2024-03-26T19:10:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.251+0000 7eff11a00640 1 -- 172.21.15.97:0/293482302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c1a21c0 con 0x7eff0c1018f0 2024-03-26T19:10:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.255+0000 7eff1482a740 1 Processor -- start 2024-03-26T19:10:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.261+0000 7eff1482a740 1 -- start start 2024-03-26T19:10:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.261+0000 7eff1482a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.263+0000 7eff1482a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede3790 0x55c71ede8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.263+0000 7eff0b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.263+0000 7eff1482a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede8900 0x55c71edead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.263+0000 7eff0b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:43510/0 (socket says 172.21.15.97:43510) 2024-03-26T19:10:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.263+0000 7eff1482a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c71ed47e80 con 0x55c71ede8900 2024-03-26T19:10:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.263+0000 7eff1482a740 1 -- 172.21.15.97:0/502167427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c71ecf0660 con 0x55c71ede3790 2024-03-26T19:10:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.263+0000 7eff0b400640 1 -- 172.21.15.97:0/502167427 learned_addr learned my addr 172.21.15.97:0/502167427 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.263+0000 7eff1482a740 1 -- 172.21.15.97:0/502167427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c71ed43e30 con 0x55c71ede2da0 2024-03-26T19:10:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.264+0000 7eff0b400640 1 -- 172.21.15.97:0/502167427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede3790 msgr2=0x55c71ede8290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:10:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.264+0000 7eff0b400640 1 --2- 172.21.15.97:0/502167427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede3790 0x55c71ede8290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.264+0000 7eff0b400640 1 -- 172.21.15.97:0/502167427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede8900 msgr2=0x55c71edead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.264+0000 7eff0b400640 1 --2- 172.21.15.97:0/502167427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede8900 0x55c71edead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.264+0000 7eff0b400640 1 -- 172.21.15.97:0/502167427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c71ea5c210 con 0x55c71ede2da0 2024-03-26T19:10:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.264+0000 7eff0b400640 1 --2- 172.21.15.97:0/502167427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede31c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7eff0400a870 tx=0x7eff04009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=730009c2430317ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.266+0000 7efef7e00640 1 -- 172.21.15.97:0/502167427 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04018020 con 0x55c71ede2da0 2024-03-26T19:10:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.266+0000 7eff03400640 1 -- 172.21.15.97:0/293482302 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7efef8010830 con 0x7eff0c1018f0 2024-03-26T19:10:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.266+0000 7efef7e00640 1 -- 172.21.15.97:0/502167427 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff04014400 con 0x55c71ede2da0 2024-03-26T19:10:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.266+0000 7efef7e00640 1 -- 172.21.15.97:0/502167427 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff040135c0 con 0x55c71ede2da0 2024-03-26T19:10:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.267+0000 7eff1482a740 1 -- 172.21.15.97:0/502167427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 msgr2=0x55c71ede31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.267+0000 7eff1482a740 1 --2- 172.21.15.97:0/502167427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede31c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7eff0400a870 tx=0x7eff04009650 comp rx=0 tx=0).stop 2024-03-26T19:10:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.268+0000 7eff03400640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efef00a3040 0x7efef00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.268+0000 7eff03400640 1 -- 172.21.15.97:0/293482302 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efef80cf850 con 0x7eff0c1018f0 2024-03-26T19:10:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.269+0000 7eff0b400640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efef00a3040 0x7efef00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.270+0000 7eff0b400640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efef00a3040 0x7efef00a5500 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7eff04000f80 tx=0x7eff0400ef90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.270+0000 7eff1482a740 1 -- 172.21.15.97:0/502167427 shutdown_connections 2024-03-26T19:10:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.270+0000 7eff1482a740 1 --2- 172.21.15.97:0/502167427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede8900 0x55c71edead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.270+0000 7eff1482a740 1 --2- 172.21.15.97:0/502167427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede3790 0x55c71ede8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.270+0000 7eff1482a740 1 --2- 172.21.15.97:0/502167427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede31c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.270+0000 7eff1482a740 1 -- 172.21.15.97:0/502167427 >> 172.21.15.97:0/502167427 conn(0x55c71edde310 msgr2=0x55c71edde710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.272+0000 7eff1482a740 1 -- 172.21.15.97:0/502167427 shutdown_connections 2024-03-26T19:10:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.272+0000 7eff1482a740 1 -- 172.21.15.97:0/502167427 wait complete. 2024-03-26T19:10:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.273+0000 7eff1482a740 1 -- start start 2024-03-26T19:10:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.273+0000 7eff1482a740 1 Processor -- start 2024-03-26T19:10:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff1482a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede2da0 0x55c71edece00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff1482a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede3790 0x55c71eded340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff1482a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede8900 0x55c71edfcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff1482a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c71edeb8b0 con 0x55c71ede3790 2024-03-26T19:10:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff1482a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c71ed03ac0 con 0x55c71ede2da0 2024-03-26T19:10:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff1482a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c71edeba30 con 0x55c71ede8900 2024-03-26T19:10:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff1482a740 10 client.0 _ll_register_callbacks cb 0x55c71ed03490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:10:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff0aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede2da0 0x55c71edece00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff0aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede2da0 0x55c71edece00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57740/0 (socket says 172.21.15.97:57740) 2024-03-26T19:10:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff0aa00640 1 -- 172.21.15.97:0/1280680399 learned_addr learned my addr 172.21.15.97:0/1280680399 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.275+0000 7eff0b400640 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede8900 0x55c71edfcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.274+0000 7eff0a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede3790 0x55c71eded340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.275+0000 7eff0b400640 1 -- 172.21.15.97:0/1280680399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede2da0 msgr2=0x55c71edece00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.275+0000 7eff0b400640 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede2da0 0x55c71edece00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.275+0000 7eff0b400640 1 -- 172.21.15.97:0/1280680399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede3790 msgr2=0x55c71eded340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.275+0000 7eff0b400640 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede3790 0x55c71eded340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.275+0000 7eff0b400640 1 -- 172.21.15.97:0/1280680399 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c71eb28620 con 0x55c71ede8900 2024-03-26T19:10:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.276+0000 7eff0b400640 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede8900 0x55c71edfcb80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7eff0400a010 tx=0x7eff04009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80258e9742d4cba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.277+0000 7efef4c00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:10:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.278+0000 7efef4c00640 1 -- 172.21.15.97:0/1280680399 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04018040 con 0x55c71ede8900 2024-03-26T19:10:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.278+0000 7efef4c00640 1 -- 172.21.15.97:0/1280680399 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7eff0400ec40 con 0x55c71ede8900 2024-03-26T19:10:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.278+0000 7eff1482a740 1 -- 172.21.15.97:0/1280680399 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff04014400 con 0x55c71ede8900 2024-03-26T19:10:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.278+0000 7efef4c00640 1 -- 172.21.15.97:0/1280680399 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff040145b0 con 0x55c71ede8900 2024-03-26T19:10:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.279+0000 7eff1482a740 10 client.26122 fetch_fsmap learned FSMap version 27 2024-03-26T19:10:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.279+0000 7eff1482a740 1 -- 172.21.15.97:0/1280680399 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c71edf1180 con 0x55c71ede8900 2024-03-26T19:10:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.279+0000 7efef4c00640 1 -- 172.21.15.97:0/1280680399 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff04014400 con 0x55c71ede8900 2024-03-26T19:10:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.279+0000 7efef4c00640 1 -- 172.21.15.97:0/1280680399 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7eff040338f0 con 0x55c71ede8900 2024-03-26T19:10:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.280+0000 7eff1482a740 10 client.26122 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:10:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.280+0000 7eff1482a740 10 client.26122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:10:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.280+0000 7eff1482a740 20 client.26122 populate_metadata read hostname 'smithi097' 2024-03-26T19:10:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.280+0000 7eff1482a740 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c71edf25e0 0x55c71edf4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.280+0000 7eff1482a740 4 client.26122 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:10:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.280+0000 7eff1482a740 1 -- 172.21.15.97:0/1280680399 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c71ed553c0 con 0x55c71edf25e0 2024-03-26T19:10:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.280+0000 7eff0aa00640 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c71edf25e0 0x55c71edf4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.281+0000 7eff0aa00640 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c71edf25e0 0x55c71edf4a00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca4a15ca62ab1ec server_cookie=c62fc4a33b719be5 in_seq=0 out_seq=0 2024-03-26T19:10:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.281+0000 7efef4c00640 10 client.26122 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.283+0000 7efef4c00640 1 -- 172.21.15.97:0/1280680399 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c71ed553c0 con 0x55c71edf25e0 2024-03-26T19:10:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.283+0000 7efef4c00640 10 client.26122 handle_command_reply: tid=0 2024-03-26T19:10:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.283+0000 7eff1482a740 1 client.26122 shutdown 2024-03-26T19:10:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.285+0000 7eff1482a740 1 -- 172.21.15.97:0/1280680399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede8900 msgr2=0x55c71edfcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.286+0000 7eff1482a740 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede8900 0x55c71edfcb80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7eff0400a010 tx=0x7eff04009650 comp rx=0 tx=0).stop 2024-03-26T19:10:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.287+0000 7eff1482a740 1 -- 172.21.15.97:0/1280680399 shutdown_connections 2024-03-26T19:10:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.287+0000 7eff1482a740 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c71edf25e0 0x55c71edf4a00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.287+0000 7eff1482a740 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede8900 0x55c71edfcb80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.287+0000 7efef4c00640 0 client.26122 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.288+0000 7eff1482a740 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede3790 0x55c71eded340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.288+0000 7eff1482a740 1 --2- 172.21.15.97:0/1280680399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede2da0 0x55c71edece00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.288+0000 7eff1482a740 1 -- 172.21.15.97:0/1280680399 >> 172.21.15.97:0/1280680399 conn(0x55c71edde310 msgr2=0x55c71eddd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.288+0000 7eff1482a740 1 -- 172.21.15.97:0/1280680399 shutdown_connections 2024-03-26T19:10:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.289+0000 7eff1482a740 1 -- 172.21.15.97:0/1280680399 wait complete. 2024-03-26T19:10:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.289+0000 7eff1482a740 20 client.26122 trim_cache size 0 max 16384 2024-03-26T19:10:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.319+0000 7eff1482a740 1 Processor -- start 2024-03-26T19:10:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.320+0000 7eff1482a740 1 -- start start 2024-03-26T19:10:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.320+0000 7eff1482a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.320+0000 7eff1482a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede3790 0x55c71edec480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.320+0000 7eff1482a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede8900 0x55c71edec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.320+0000 7eff0aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.321+0000 7eff1482a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1a6b10 con 0x55c71ede8900 2024-03-26T19:10:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.321+0000 7eff0a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede3790 0x55c71edec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.321+0000 7eff1482a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1a8650 con 0x55c71ede3790 2024-03-26T19:10:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.321+0000 7eff1482a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1a71e0 con 0x55c71ede2da0 2024-03-26T19:10:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.321+0000 7eff0a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede3790 0x55c71edec480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57752/0 (socket says 172.21.15.97:57752) 2024-03-26T19:10:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.321+0000 7eff0a000640 1 -- 172.21.15.97:0/4255511099 learned_addr learned my addr 172.21.15.97:0/4255511099 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.321+0000 7eff0b400640 1 --2- 172.21.15.97:0/4255511099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede8900 0x55c71edec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.321+0000 7eff0aa00640 1 -- 172.21.15.97:0/4255511099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede3790 msgr2=0x55c71edec480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.321+0000 7eff0aa00640 1 --2- 172.21.15.97:0/4255511099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede3790 0x55c71edec480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.321+0000 7eff0aa00640 1 -- 172.21.15.97:0/4255511099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede8900 msgr2=0x55c71edec9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.322+0000 7eff0aa00640 1 --2- 172.21.15.97:0/4255511099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede8900 0x55c71edec9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.322+0000 7eff0aa00640 1 -- 172.21.15.97:0/4255511099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c71ede29d0 con 0x55c71ede2da0 2024-03-26T19:10:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.322+0000 7eff0b400640 1 --2- 172.21.15.97:0/4255511099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede8900 0x55c71edec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.322+0000 7eff0aa00640 1 --2- 172.21.15.97:0/4255511099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede1570 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efef8004630 tx=0x7efef80bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c076ad085f8924 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.323+0000 7efef7400640 1 -- 172.21.15.97:0/4255511099 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef809bf50 con 0x55c71ede2da0 2024-03-26T19:10:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.324+0000 7efef7400640 1 -- 172.21.15.97:0/4255511099 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef80b4070 con 0x55c71ede2da0 2024-03-26T19:10:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.324+0000 7efef7400640 1 -- 172.21.15.97:0/4255511099 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef80037a0 con 0x55c71ede2da0 2024-03-26T19:10:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.325+0000 7eff1482a740 1 -- 172.21.15.97:0/4255511099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 msgr2=0x55c71ede1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.325+0000 7eff1482a740 1 --2- 172.21.15.97:0/4255511099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede1570 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efef8004630 tx=0x7efef80bc040 comp rx=0 tx=0).stop 2024-03-26T19:10:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.325+0000 7eff1482a740 1 -- 172.21.15.97:0/4255511099 shutdown_connections 2024-03-26T19:10:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.325+0000 7eff1482a740 1 --2- 172.21.15.97:0/4255511099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede8900 0x55c71edec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.325+0000 7eff1482a740 1 --2- 172.21.15.97:0/4255511099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede3790 0x55c71edec480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.325+0000 7eff1482a740 1 --2- 172.21.15.97:0/4255511099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede2da0 0x55c71ede1570 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.325+0000 7eff1482a740 1 -- 172.21.15.97:0/4255511099 >> 172.21.15.97:0/4255511099 conn(0x55c71edde310 msgr2=0x55c71ede4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.325+0000 7eff1482a740 1 -- 172.21.15.97:0/4255511099 shutdown_connections 2024-03-26T19:10:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.325+0000 7eff1482a740 1 -- 172.21.15.97:0/4255511099 wait complete. 2024-03-26T19:10:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.327+0000 7eff1482a740 1 -- start start 2024-03-26T19:10:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.328+0000 7eff1482a740 1 Processor -- start 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.329+0000 7eff1482a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede2da0 0x55c71edf85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.329+0000 7eff1482a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede3790 0x55c71edf8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.329+0000 7eff1482a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede8900 0x55c71edf9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.329+0000 7eff1482a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1a7f80 con 0x55c71ede2da0 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.329+0000 7eff1482a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1a78b0 con 0x55c71ede8900 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.329+0000 7eff1482a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1a4b30 con 0x55c71ede3790 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.329+0000 7eff1482a740 10 client.0 _ll_register_callbacks cb 0x7eff04013450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.329+0000 7eff0b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede8900 0x55c71edf9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.329+0000 7eff0b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede8900 0x55c71edf9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57754/0 (socket says 172.21.15.97:57754) 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.329+0000 7eff0b400640 1 -- 172.21.15.97:0/1712873718 learned_addr learned my addr 172.21.15.97:0/1712873718 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.330+0000 7eff0a000640 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede3790 0x55c71edf8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.330+0000 7eff0b400640 1 -- 172.21.15.97:0/1712873718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede3790 msgr2=0x55c71edf8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.330+0000 7eff0aa00640 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede2da0 0x55c71edf85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.330+0000 7eff0b400640 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede3790 0x55c71edf8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.331+0000 7eff0b400640 1 -- 172.21.15.97:0/1712873718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede2da0 msgr2=0x55c71edf85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:10:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.331+0000 7eff0b400640 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede2da0 0x55c71edf85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.331+0000 7eff0b400640 1 -- 172.21.15.97:0/1712873718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c71edfa0c0 con 0x55c71ede8900 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.331+0000 7eff0aa00640 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede2da0 0x55c71edf85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.331+0000 7eff0b400640 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede8900 0x55c71edf9b00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7eff04033cc0 tx=0x7eff04007070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32750daf7bd77a49 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.331+0000 7efef4c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.331+0000 7efef4c00640 1 -- 172.21.15.97:0/1712873718 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04018050 con 0x55c71ede8900 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.331+0000 7efef4c00640 1 -- 172.21.15.97:0/1712873718 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7eff04033890 con 0x55c71ede8900 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.332+0000 7eff1482a740 1 -- 172.21.15.97:0/1712873718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef80b4070 con 0x55c71ede8900 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.332+0000 7efef4c00640 1 -- 172.21.15.97:0/1712873718 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff04014400 con 0x55c71ede8900 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.332+0000 7eff1482a740 10 client.26147 fetch_fsmap learned FSMap version 27 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.332+0000 7eff1482a740 1 -- 172.21.15.97:0/1712873718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c71ee81020 con 0x55c71ede8900 2024-03-26T19:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.332+0000 7efef4c00640 1 -- 172.21.15.97:0/1712873718 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff04014400 con 0x55c71ede8900 2024-03-26T19:10:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.334+0000 7efef4c00640 1 -- 172.21.15.97:0/1712873718 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7eff04014750 con 0x55c71ede8900 2024-03-26T19:10:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.334+0000 7eff1482a740 10 client.26147 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:10:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.334+0000 7eff1482a740 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:10:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.334+0000 7eff1482a740 20 client.26147 populate_metadata read hostname 'smithi097' 2024-03-26T19:10:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.334+0000 7eff1482a740 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c71ee81980 0x55c71ee83da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:10:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.334+0000 7eff1482a740 4 client.26147 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:10:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.334+0000 7eff1482a740 1 -- 172.21.15.97:0/1712873718 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c71ed4f710 con 0x55c71ee81980 2024-03-26T19:10:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.334+0000 7eff0aa00640 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c71ee81980 0x55c71ee83da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:10:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.335+0000 7eff0aa00640 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c71ee81980 0x55c71ee83da0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8646bf7304aacf8 server_cookie=8076c9ce6b9d0edc in_seq=0 out_seq=0 2024-03-26T19:10:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.335+0000 7efef4c00640 10 client.26147 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.355+0000 7efef4c00640 1 -- 172.21.15.97:0/1712873718 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c71ed4f710 con 0x55c71ee81980 2024-03-26T19:10:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.355+0000 7efef4c00640 10 client.26147 handle_command_reply: tid=0 2024-03-26T19:10:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.355+0000 7eff1482a740 1 client.26147 shutdown 2024-03-26T19:10:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.355+0000 7eff1482a740 1 -- 172.21.15.97:0/1712873718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede8900 msgr2=0x55c71edf9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.355+0000 7eff1482a740 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede8900 0x55c71edf9b00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7eff04033cc0 tx=0x7eff04007070 comp rx=0 tx=0).stop 2024-03-26T19:10:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.356+0000 7eff1482a740 1 -- 172.21.15.97:0/1712873718 shutdown_connections 2024-03-26T19:10:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.356+0000 7eff1482a740 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c71ee81980 0x55c71ee83da0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.356+0000 7eff1482a740 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c71ede8900 0x55c71edf9b00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.356+0000 7eff1482a740 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c71ede3790 0x55c71edf8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.356+0000 7eff1482a740 1 --2- 172.21.15.97:0/1712873718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c71ede2da0 0x55c71edf85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.356+0000 7eff1482a740 1 -- 172.21.15.97:0/1712873718 >> 172.21.15.97:0/1712873718 conn(0x55c71edde310 msgr2=0x55c71eddccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.356+0000 7efef4c00640 0 client.26147 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:10:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.356+0000 7eff1482a740 1 -- 172.21.15.97:0/1712873718 shutdown_connections 2024-03-26T19:10:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.356+0000 7eff1482a740 1 -- 172.21.15.97:0/1712873718 wait complete. 2024-03-26T19:10:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:10:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:10:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:10:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.356+0000 7eff1482a740 20 client.26147 trim_cache size 0 max 16384 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 -- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efef00a3040 msgr2=0x7efef00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efef00a3040 0x7efef00a5500 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7eff04000f80 tx=0x7eff0400ef90 comp rx=0 tx=0).stop 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 -- 172.21.15.97:0/293482302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0c1018f0 msgr2=0x7eff0c199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0c1018f0 0x7eff0c199dc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efef800e9c0 tx=0x7efef800ee80 comp rx=0 tx=0).stop 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 -- 172.21.15.97:0/293482302 shutdown_connections 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efef00a3040 0x7efef00a5500 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff0c1034a0 0x7eff0c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff0c102ae0 0x7eff0c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 --2- 172.21.15.97:0/293482302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0c1018f0 0x7eff0c199dc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 -- 172.21.15.97:0/293482302 >> 172.21.15.97:0/293482302 conn(0x7eff0c0fce80 msgr2=0x7eff0c073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 -- 172.21.15.97:0/293482302 shutdown_connections 2024-03-26T19:10:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:10:56.357+0000 7eff11a00640 1 -- 172.21.15.97:0/293482302 wait complete. 2024-03-26T19:10:56.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:56 smithi105 ceph-mon[36270]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 590 op/s 2024-03-26T19:10:56.858 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:56 smithi097 ceph-mon[31955]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 590 op/s 2024-03-26T19:10:56.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:56 smithi144 ceph-mon[36053]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 590 op/s 2024-03-26T19:10:57.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:10:57 smithi097 ceph-mon[31955]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.1 MiB/s wr, 576 op/s 2024-03-26T19:10:57.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:10:57 smithi144 ceph-mon[36053]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.1 MiB/s wr, 576 op/s 2024-03-26T19:10:57.966 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:10:57 smithi105 ceph-mon[36270]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.1 MiB/s wr, 576 op/s 2024-03-26T19:11:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 695.0 s, 1295.8 tps, lat 78.576 ms stddev 102.874 2024-03-26T19:11:00.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:00 smithi144 ceph-mon[36053]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-03-26T19:11:00.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:00 smithi105 ceph-mon[36270]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-03-26T19:11:00.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:00 smithi097 ceph-mon[31955]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-03-26T19:11:02.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:02 smithi144 ceph-mon[36053]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 672 op/s 2024-03-26T19:11:02.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:02 smithi097 ceph-mon[31955]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 672 op/s 2024-03-26T19:11:02.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:02 smithi105 ceph-mon[36270]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 672 op/s 2024-03-26T19:11:04.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:04 smithi105 ceph-mon[36270]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 579 op/s 2024-03-26T19:11:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:04 smithi097 ceph-mon[31955]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 579 op/s 2024-03-26T19:11:04.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:04 smithi144 ceph-mon[36053]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 579 op/s 2024-03-26T19:11:05.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 700.0 s, 1738.8 tps, lat 57.092 ms stddev 70.998 2024-03-26T19:11:06.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:06 smithi105 ceph-mon[36270]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-03-26T19:11:06.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:06 smithi097 ceph-mon[31955]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-03-26T19:11:06.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:06 smithi144 ceph-mon[36053]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-03-26T19:11:08.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:08 smithi105 ceph-mon[36270]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 608 op/s 2024-03-26T19:11:08.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:08 smithi097 ceph-mon[31955]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 608 op/s 2024-03-26T19:11:08.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:08 smithi144 ceph-mon[36053]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 608 op/s 2024-03-26T19:11:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.556+0000 7efdbd000640 1 -- 172.21.15.97:0/4037996132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8075900 msgr2=0x7efdb8073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.556+0000 7efdbd000640 1 --2- 172.21.15.97:0/4037996132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8075900 0x7efdb8073d50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efda8009cd0 tx=0x7efda802c0e0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.556+0000 7efdbd000640 1 -- 172.21.15.97:0/4037996132 shutdown_connections 2024-03-26T19:11:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.556+0000 7efdbd000640 1 --2- 172.21.15.97:0/4037996132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdb810dd00 0x7efdb81100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.556+0000 7efdbd000640 1 --2- 172.21.15.97:0/4037996132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdb8074320 0x7efdb80747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.556+0000 7efdbd000640 1 --2- 172.21.15.97:0/4037996132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8075900 0x7efdb8073d50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.556+0000 7efdbd000640 1 -- 172.21.15.97:0/4037996132 >> 172.21.15.97:0/4037996132 conn(0x7efdb80fcca0 msgr2=0x7efdb80ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.556+0000 7efdbd000640 1 -- 172.21.15.97:0/4037996132 shutdown_connections 2024-03-26T19:11:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.556+0000 7efdbd000640 1 -- 172.21.15.97:0/4037996132 wait complete. 2024-03-26T19:11:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.558+0000 7efdbd000640 1 Processor -- start 2024-03-26T19:11:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.558+0000 7efdbd000640 1 -- start start 2024-03-26T19:11:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.558+0000 7efdbd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdb8074320 0x7efdb819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.558+0000 7efdbd000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8075900 0x7efdb819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.558+0000 7efdbd000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdb810dd00 0x7efdb81a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.558+0000 7efdbd000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efdb81139f0 con 0x7efdb8074320 2024-03-26T19:11:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.558+0000 7efdbd000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8113870 con 0x7efdb8075900 2024-03-26T19:11:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.558+0000 7efdbd000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8113b70 con 0x7efdb810dd00 2024-03-26T19:11:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8075900 0x7efdb819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdb7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdb810dd00 0x7efdb81a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8075900 0x7efdb819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57520/0 (socket says 172.21.15.97:57520) 2024-03-26T19:11:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdb7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdb810dd00 0x7efdb81a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50892/0 (socket says 172.21.15.97:50892) 2024-03-26T19:11:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdafe00640 1 -- 172.21.15.97:0/2212980570 learned_addr learned my addr 172.21.15.97:0/2212980570 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdb7400640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdb8074320 0x7efdb819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdb7e00640 1 -- 172.21.15.97:0/2212980570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8075900 msgr2=0x7efdb819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdb7e00640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8075900 0x7efdb819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdb7e00640 1 -- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdb8074320 msgr2=0x7efdb819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdb7e00640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdb8074320 0x7efdb819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdb7e00640 1 -- 172.21.15.97:0/2212980570 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb81a5cd0 con 0x7efdb810dd00 2024-03-26T19:11:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdb7400640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdb8074320 0x7efdb819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:11:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.559+0000 7efdb7e00640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdb810dd00 0x7efdb81a5660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efdb000e970 tx=0x7efdb000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.560+0000 7efdb4c00640 1 -- 172.21.15.97:0/2212980570 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb000ccc0 con 0x7efdb810dd00 2024-03-26T19:11:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.560+0000 7efdb4c00640 1 -- 172.21.15.97:0/2212980570 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb0005020 con 0x7efdb810dd00 2024-03-26T19:11:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.560+0000 7efdb4c00640 1 -- 172.21.15.97:0/2212980570 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0004a00 con 0x7efdb810dd00 2024-03-26T19:11:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.560+0000 7efdbd000640 1 -- 172.21.15.97:0/2212980570 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb81a5fc0 con 0x7efdb810dd00 2024-03-26T19:11:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.560+0000 7efdbd000640 1 -- 172.21.15.97:0/2212980570 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb81a6430 con 0x7efdb810dd00 2024-03-26T19:11:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.564+0000 7efdb4c00640 1 -- 172.21.15.97:0/2212980570 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7efdb00051d0 con 0x7efdb810dd00 2024-03-26T19:11:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.565+0000 7efdb4c00640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efd9c0a2fd0 0x7efd9c0a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.565+0000 7efdb4c00640 1 -- 172.21.15.97:0/2212980570 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efdb00cf300 con 0x7efdb810dd00 2024-03-26T19:11:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.566+0000 7efdb7400640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efd9c0a2fd0 0x7efd9c0a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.566+0000 7efdc13c9740 1 Processor -- start 2024-03-26T19:11:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.567+0000 7efdb7400640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efd9c0a2fd0 0x7efd9c0a5490 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efda8009ca0 tx=0x7efda8009980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.567+0000 7efdc13c9740 1 -- start start 2024-03-26T19:11:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.567+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb29f60 0x55acccb2a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.567+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2a950 0x55acccb2f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.567+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2fac0 0x55acccb31f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.567+0000 7efdc13c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55accca8ee80 con 0x55acccb29f60 2024-03-26T19:11:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.567+0000 7efdc13c9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55accca37660 con 0x55acccb2a950 2024-03-26T19:11:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.567+0000 7efdc13c9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55accca8ae30 con 0x55acccb2fac0 2024-03-26T19:11:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.567+0000 7efdafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2a950 0x55acccb2f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2a950 0x55acccb2f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57528/0 (socket says 172.21.15.97:57528) 2024-03-26T19:11:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdafe00640 1 -- 172.21.15.97:0/125345499 learned_addr learned my addr 172.21.15.97:0/125345499 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdb7e00640 1 --2- 172.21.15.97:0/125345499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2fac0 0x55acccb31f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdb7400640 1 --2- 172.21.15.97:0/125345499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb29f60 0x55acccb2a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdafe00640 1 -- 172.21.15.97:0/125345499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2fac0 msgr2=0x55acccb31f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdafe00640 1 --2- 172.21.15.97:0/125345499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2fac0 0x55acccb31f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdafe00640 1 -- 172.21.15.97:0/125345499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb29f60 msgr2=0x55acccb2a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdafe00640 1 --2- 172.21.15.97:0/125345499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb29f60 0x55acccb2a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdafe00640 1 -- 172.21.15.97:0/125345499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55accc7a3210 con 0x55acccb2a950 2024-03-26T19:11:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdafe00640 1 --2- 172.21.15.97:0/125345499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2a950 0x55acccb2f450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efda000bd60 tx=0x7efda00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dbc760f88e67c52 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.568+0000 7efdacc00640 1 -- 172.21.15.97:0/125345499 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0011640 con 0x55acccb2a950 2024-03-26T19:11:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.569+0000 7efdacc00640 1 -- 172.21.15.97:0/125345499 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda0011cc0 con 0x55acccb2a950 2024-03-26T19:11:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.569+0000 7efdacc00640 1 -- 172.21.15.97:0/125345499 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda001bd10 con 0x55acccb2a950 2024-03-26T19:11:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.570+0000 7efdc13c9740 1 -- 172.21.15.97:0/125345499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2a950 msgr2=0x55acccb2f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.570+0000 7efdc13c9740 1 --2- 172.21.15.97:0/125345499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2a950 0x55acccb2f450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efda000bd60 tx=0x7efda00107a0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.571+0000 7efdc13c9740 1 -- 172.21.15.97:0/125345499 shutdown_connections 2024-03-26T19:11:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.571+0000 7efdc13c9740 1 --2- 172.21.15.97:0/125345499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2fac0 0x55acccb31f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.571+0000 7efdc13c9740 1 --2- 172.21.15.97:0/125345499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2a950 0x55acccb2f450 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.571+0000 7efdc13c9740 1 --2- 172.21.15.97:0/125345499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb29f60 0x55acccb2a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.571+0000 7efdc13c9740 1 -- 172.21.15.97:0/125345499 >> 172.21.15.97:0/125345499 conn(0x55acccb254d0 msgr2=0x55acccb258d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.572+0000 7efdc13c9740 1 -- 172.21.15.97:0/125345499 shutdown_connections 2024-03-26T19:11:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.572+0000 7efdc13c9740 1 -- 172.21.15.97:0/125345499 wait complete. 2024-03-26T19:11:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.573+0000 7efdc13c9740 1 -- start start 2024-03-26T19:11:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.573+0000 7efdc13c9740 1 Processor -- start 2024-03-26T19:11:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.574+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb29f60 0x55acccb33f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.574+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.574+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb2fac0 0x55acccb3b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdafe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdafe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb344d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50900/0 (socket says 172.21.15.97:50900) 2024-03-26T19:11:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdc13c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55acccb32a70 con 0x55acccb2fac0 2024-03-26T19:11:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdc13c9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55accca4aac0 con 0x55acccb29f60 2024-03-26T19:11:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdc13c9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55acccb32bf0 con 0x55acccb2a950 2024-03-26T19:11:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdafe00640 1 -- 172.21.15.97:0/766961582 learned_addr learned my addr 172.21.15.97:0/766961582 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdb7400640 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb29f60 0x55acccb33f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdc13c9740 10 client.0 _ll_register_callbacks cb 0x55accca4a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:11:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdafe00640 1 -- 172.21.15.97:0/766961582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb29f60 msgr2=0x55acccb33f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdafe00640 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb29f60 0x55acccb33f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdafe00640 1 -- 172.21.15.97:0/766961582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb2fac0 msgr2=0x55acccb3b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.575+0000 7efdafe00640 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb2fac0 0x55acccb3b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.576+0000 7efdafe00640 1 -- 172.21.15.97:0/766961582 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55accc86f620 con 0x55acccb2a950 2024-03-26T19:11:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.576+0000 7efdafe00640 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb344d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efda0010ca0 tx=0x7efda000bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c4191c05894d77 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.576+0000 7efda5600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:11:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.577+0000 7efda5600640 1 -- 172.21.15.97:0/766961582 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda001bc60 con 0x55acccb2a950 2024-03-26T19:11:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.577+0000 7efda5600640 1 -- 172.21.15.97:0/766961582 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efda001be00 con 0x55acccb2a950 2024-03-26T19:11:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.577+0000 7efdc13c9740 1 -- 172.21.15.97:0/766961582 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda0011cc0 con 0x55acccb2a950 2024-03-26T19:11:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.577+0000 7efda5600640 1 -- 172.21.15.97:0/766961582 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efda0004020 con 0x55acccb2a950 2024-03-26T19:11:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.577+0000 7efdc13c9740 10 client.26140 fetch_fsmap learned FSMap version 27 2024-03-26T19:11:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.577+0000 7efdc13c9740 1 -- 172.21.15.97:0/766961582 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acccb3e860 con 0x55acccb2a950 2024-03-26T19:11:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.578+0000 7efda5600640 1 -- 172.21.15.97:0/766961582 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda0011cc0 con 0x55acccb2a950 2024-03-26T19:11:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.578+0000 7efda5600640 1 -- 172.21.15.97:0/766961582 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efda0011550 con 0x55acccb2a950 2024-03-26T19:11:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.578+0000 7efdc13c9740 10 client.26140 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:11:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.578+0000 7efdc13c9740 10 client.26140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:11:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.578+0000 7efdc13c9740 20 client.26140 populate_metadata read hostname 'smithi097' 2024-03-26T19:11:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.578+0000 7efdc13c9740 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55acccb3fc60 0x55acccb42050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.578+0000 7efdc13c9740 4 client.26140 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:11:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.578+0000 7efdc13c9740 1 -- 172.21.15.97:0/766961582 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55accca9c3c0 con 0x55acccb3fc60 2024-03-26T19:11:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.579+0000 7efdb7400640 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55acccb3fc60 0x55acccb42050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.579+0000 7efda5600640 10 client.26140 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.579+0000 7efdb7400640 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55acccb3fc60 0x55acccb42050 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d156dc3ff5db833 server_cookie=3221bfbec2e0e0ef in_seq=0 out_seq=0 2024-03-26T19:11:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.580+0000 7efda5600640 1 -- 172.21.15.97:0/766961582 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55accca9c3c0 con 0x55acccb3fc60 2024-03-26T19:11:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.581+0000 7efda5600640 10 client.26140 handle_command_reply: tid=0 2024-03-26T19:11:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.581+0000 7efdc13c9740 1 client.26140 shutdown 2024-03-26T19:11:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.582+0000 7efdc13c9740 1 -- 172.21.15.97:0/766961582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 msgr2=0x55acccb344d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.582+0000 7efdc13c9740 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb344d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efda0010ca0 tx=0x7efda000bf80 comp rx=0 tx=0).stop 2024-03-26T19:11:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.582+0000 7efdc13c9740 1 -- 172.21.15.97:0/766961582 shutdown_connections 2024-03-26T19:11:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.582+0000 7efdc13c9740 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55acccb3fc60 0x55acccb42050 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.582+0000 7efda5600640 0 client.26140 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.582+0000 7efdc13c9740 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb2fac0 0x55acccb3b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.582+0000 7efdc13c9740 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb344d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.582+0000 7efdc13c9740 1 --2- 172.21.15.97:0/766961582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb29f60 0x55acccb33f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.582+0000 7efdc13c9740 1 -- 172.21.15.97:0/766961582 >> 172.21.15.97:0/766961582 conn(0x55acccb254d0 msgr2=0x55acccb246f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.586+0000 7efdc13c9740 1 -- 172.21.15.97:0/766961582 shutdown_connections 2024-03-26T19:11:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.586+0000 7efdc13c9740 1 -- 172.21.15.97:0/766961582 wait complete. 2024-03-26T19:11:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.586+0000 7efdc13c9740 20 client.26140 trim_cache size 0 max 16384 2024-03-26T19:11:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.631+0000 7efdc13c9740 1 Processor -- start 2024-03-26T19:11:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.631+0000 7efdc13c9740 1 -- start start 2024-03-26T19:11:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.632+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb29f60 0x55acccb2efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.632+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb3b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.632+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 0x55acccb2b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.632+0000 7efdc13c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efdb81af390 con 0x55acccb29f60 2024-03-26T19:11:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.632+0000 7efdc13c9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efdb81b0ed0 con 0x55acccb2fac0 2024-03-26T19:11:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.632+0000 7efdc13c9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efdb81afa60 con 0x55acccb2a950 2024-03-26T19:11:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.632+0000 7efdb7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 0x55acccb2b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.632+0000 7efdafe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb3b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.632+0000 7efdafe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb3b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50906/0 (socket says 172.21.15.97:50906) 2024-03-26T19:11:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.633+0000 7efdafe00640 1 -- 172.21.15.97:0/2236007335 learned_addr learned my addr 172.21.15.97:0/2236007335 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.633+0000 7efdb7e00640 1 -- 172.21.15.97:0/2236007335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 msgr2=0x55acccb3b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.633+0000 7efdb7e00640 1 --2- 172.21.15.97:0/2236007335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb3b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.633+0000 7efdb7e00640 1 -- 172.21.15.97:0/2236007335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb29f60 msgr2=0x55acccb2efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.633+0000 7efdb7e00640 1 --2- 172.21.15.97:0/2236007335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb29f60 0x55acccb2efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.633+0000 7efdb7e00640 1 -- 172.21.15.97:0/2236007335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acccb20ae0 con 0x55acccb2fac0 2024-03-26T19:11:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.634+0000 7efdb7e00640 1 --2- 172.21.15.97:0/2236007335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 0x55acccb2b2d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efdb00cce80 tx=0x7efdb009c160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f333ed73e0ee04 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.634+0000 7efda7e00640 1 -- 172.21.15.97:0/2236007335 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb00b9070 con 0x55acccb2fac0 2024-03-26T19:11:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.634+0000 7efda7e00640 1 -- 172.21.15.97:0/2236007335 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb009fbb0 con 0x55acccb2fac0 2024-03-26T19:11:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.634+0000 7efda7e00640 1 -- 172.21.15.97:0/2236007335 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb001c040 con 0x55acccb2fac0 2024-03-26T19:11:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.635+0000 7efdc13c9740 1 -- 172.21.15.97:0/2236007335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 msgr2=0x55acccb2b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.635+0000 7efdc13c9740 1 --2- 172.21.15.97:0/2236007335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 0x55acccb2b2d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efdb00cce80 tx=0x7efdb009c160 comp rx=0 tx=0).stop 2024-03-26T19:11:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.636+0000 7efdc13c9740 1 -- 172.21.15.97:0/2236007335 shutdown_connections 2024-03-26T19:11:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.636+0000 7efdc13c9740 1 --2- 172.21.15.97:0/2236007335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 0x55acccb2b2d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.636+0000 7efdc13c9740 1 --2- 172.21.15.97:0/2236007335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb2a950 0x55acccb3b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.636+0000 7efdc13c9740 1 --2- 172.21.15.97:0/2236007335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb29f60 0x55acccb2efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.636+0000 7efdc13c9740 1 -- 172.21.15.97:0/2236007335 >> 172.21.15.97:0/2236007335 conn(0x55acccb254d0 msgr2=0x55acccb28fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.636+0000 7efdc13c9740 1 -- 172.21.15.97:0/2236007335 shutdown_connections 2024-03-26T19:11:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.636+0000 7efdc13c9740 1 -- 172.21.15.97:0/2236007335 wait complete. 2024-03-26T19:11:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.637+0000 7efdc13c9740 1 -- start start 2024-03-26T19:11:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.637+0000 7efdc13c9740 1 Processor -- start 2024-03-26T19:11:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.638+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb29f60 0x55acccb38de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.638+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb2a950 0x55acccb39320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.638+0000 7efdc13c9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 0x55acccb3a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.638+0000 7efdb7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb29f60 0x55acccb38de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.638+0000 7efdb7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb29f60 0x55acccb38de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50922/0 (socket says 172.21.15.97:50922) 2024-03-26T19:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.638+0000 7efdb7400640 1 -- 172.21.15.97:0/3181441307 learned_addr learned my addr 172.21.15.97:0/3181441307 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.638+0000 7efdb7e00640 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 0x55acccb3a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.639+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efdb81b0800 con 0x55acccb2a950 2024-03-26T19:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.639+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efdb81b0130 con 0x55acccb2fac0 2024-03-26T19:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.639+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efdb81ad370 con 0x55acccb29f60 2024-03-26T19:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.639+0000 7efdc13c9740 10 client.0 _ll_register_callbacks cb 0x7efda0005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.640+0000 7efdafe00640 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb2a950 0x55acccb39320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.640+0000 7efdb7400640 1 -- 172.21.15.97:0/3181441307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 msgr2=0x55acccb3a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.641+0000 7efdb7400640 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 0x55acccb3a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.641+0000 7efdb7400640 1 -- 172.21.15.97:0/3181441307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb2a950 msgr2=0x55acccb39320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.641+0000 7efdb7400640 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb2a950 0x55acccb39320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.642+0000 7efdb7400640 1 -- 172.21.15.97:0/3181441307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acccb3a920 con 0x55acccb29f60 2024-03-26T19:11:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.642+0000 7efdafe00640 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb2a950 0x55acccb39320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:11:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.642+0000 7efdb7400640 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb29f60 0x55acccb38de0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efda8006010 tx=0x7efda80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0b49df6610d3d24 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.642+0000 7efda5600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:11:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.643+0000 7efda5600640 1 -- 172.21.15.97:0/3181441307 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8051070 con 0x55acccb29f60 2024-03-26T19:11:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.643+0000 7efda5600640 1 -- 172.21.15.97:0/3181441307 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efda8051210 con 0x55acccb29f60 2024-03-26T19:11:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.643+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdb009fbb0 con 0x55acccb29f60 2024-03-26T19:11:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.644+0000 7efda5600640 1 -- 172.21.15.97:0/3181441307 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efda8039040 con 0x55acccb29f60 2024-03-26T19:11:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.644+0000 7efdc13c9740 10 client.26146 fetch_fsmap learned FSMap version 27 2024-03-26T19:11:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.644+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acccba6160 con 0x55acccb29f60 2024-03-26T19:11:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.644+0000 7efda5600640 1 -- 172.21.15.97:0/3181441307 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb009fbb0 con 0x55acccb29f60 2024-03-26T19:11:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.646+0000 7efda5600640 1 -- 172.21.15.97:0/3181441307 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efda804c040 con 0x55acccb29f60 2024-03-26T19:11:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.646+0000 7efdc13c9740 10 client.26146 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:11:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.646+0000 7efdc13c9740 10 client.26146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:11:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.646+0000 7efdc13c9740 20 client.26146 populate_metadata read hostname 'smithi097' 2024-03-26T19:11:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.646+0000 7efdc13c9740 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55acccba6b10 0x55acccba8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.646+0000 7efdc13c9740 4 client.26146 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:11:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.646+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55accca96710 con 0x55acccba6b10 2024-03-26T19:11:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.648+0000 7efdafe00640 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55acccba6b10 0x55acccba8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.649+0000 7efdafe00640 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55acccba6b10 0x55acccba8f30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7461ee1941279f7f server_cookie=e391c97be2422bc3 in_seq=0 out_seq=0 2024-03-26T19:11:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.649+0000 7efda5600640 10 client.26146 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.673+0000 7efda5600640 1 -- 172.21.15.97:0/3181441307 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55accca96710 con 0x55acccba6b10 2024-03-26T19:11:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.673+0000 7efda5600640 10 client.26146 handle_command_reply: tid=0 2024-03-26T19:11:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.673+0000 7efdc13c9740 1 client.26146 shutdown 2024-03-26T19:11:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.673+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb29f60 msgr2=0x55acccb38de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.673+0000 7efdc13c9740 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb29f60 0x55acccb38de0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efda8006010 tx=0x7efda80095c0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.674+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 shutdown_connections 2024-03-26T19:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.674+0000 7efdc13c9740 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55acccba6b10 0x55acccba8f30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.674+0000 7efdc13c9740 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acccb2fac0 0x55acccb3a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.674+0000 7efdc13c9740 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55acccb2a950 0x55acccb39320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.674+0000 7efdc13c9740 1 --2- 172.21.15.97:0/3181441307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acccb29f60 0x55acccb38de0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.674+0000 7efda5600640 0 client.26146 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.674+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 >> 172.21.15.97:0/3181441307 conn(0x55acccb254d0 msgr2=0x55acccb28fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.675+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 shutdown_connections 2024-03-26T19:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.675+0000 7efdc13c9740 1 -- 172.21.15.97:0/3181441307 wait complete. 2024-03-26T19:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.675+0000 7efdc13c9740 20 client.26146 trim_cache size 0 max 16384 2024-03-26T19:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 -- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efd9c0a2fd0 msgr2=0x7efd9c0a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efd9c0a2fd0 0x7efd9c0a5490 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efda8009ca0 tx=0x7efda8009980 comp rx=0 tx=0).stop 2024-03-26T19:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 -- 172.21.15.97:0/2212980570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdb810dd00 msgr2=0x7efdb81a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdb810dd00 0x7efdb81a5660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efdb000e970 tx=0x7efdb000ee30 comp rx=0 tx=0).stop 2024-03-26T19:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 -- 172.21.15.97:0/2212980570 shutdown_connections 2024-03-26T19:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7efd9c0a2fd0 0x7efd9c0a5490 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdb810dd00 0x7efdb81a5660 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8075900 0x7efdb819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 --2- 172.21.15.97:0/2212980570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdb8074320 0x7efdb819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 -- 172.21.15.97:0/2212980570 >> 172.21.15.97:0/2212980570 conn(0x7efdb80fcca0 msgr2=0x7efdb81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 -- 172.21.15.97:0/2212980570 shutdown_connections 2024-03-26T19:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:09.676+0000 7efdbd000640 1 -- 172.21.15.97:0/2212980570 wait complete. 2024-03-26T19:11:10.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 705.0 s, 924.0 tps, lat 107.017 ms stddev 109.895 2024-03-26T19:11:10.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:10 smithi105 ceph-mon[36270]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 635 op/s 2024-03-26T19:11:10.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:10 smithi097 ceph-mon[31955]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 635 op/s 2024-03-26T19:11:10.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:10 smithi144 ceph-mon[36053]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 635 op/s 2024-03-26T19:11:12.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:12 smithi105 ceph-mon[36270]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 679 op/s 2024-03-26T19:11:12.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:12 smithi097 ceph-mon[31955]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 679 op/s 2024-03-26T19:11:12.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:12 smithi144 ceph-mon[36053]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 679 op/s 2024-03-26T19:11:14.684 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:14 smithi144 ceph-mon[36053]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 566 op/s 2024-03-26T19:11:14.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:14 smithi105 ceph-mon[36270]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 566 op/s 2024-03-26T19:11:14.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:14 smithi097 ceph-mon[31955]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 566 op/s 2024-03-26T19:11:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 710.0 s, 894.6 tps, lat 109.247 ms stddev 126.014 2024-03-26T19:11:16.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:16 smithi105 ceph-mon[36270]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 637 op/s 2024-03-26T19:11:16.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:16 smithi097 ceph-mon[31955]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 637 op/s 2024-03-26T19:11:16.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:16 smithi144 ceph-mon[36053]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 637 op/s 2024-03-26T19:11:18.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:18 smithi105 ceph-mon[36270]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 589 op/s 2024-03-26T19:11:18.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:18 smithi097 ceph-mon[31955]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 589 op/s 2024-03-26T19:11:18.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:18 smithi144 ceph-mon[36053]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 589 op/s 2024-03-26T19:11:19.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:19 smithi097 ceph-mon[31955]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 634 op/s 2024-03-26T19:11:19.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:19 smithi144 ceph-mon[36053]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 634 op/s 2024-03-26T19:11:19.966 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:19 smithi105 ceph-mon[36270]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 634 op/s 2024-03-26T19:11:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 715.0 s, 916.0 tps, lat 112.433 ms stddev 126.748 2024-03-26T19:11:22.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:22 smithi105 ceph-mon[36270]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 644 op/s 2024-03-26T19:11:22.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:22 smithi097 ceph-mon[31955]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 644 op/s 2024-03-26T19:11:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.860+0000 7f330d400640 1 -- 172.21.15.97:0/3678366472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3308074320 msgr2=0x7f33080747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.860+0000 7f330d400640 1 --2- 172.21.15.97:0/3678366472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3308074320 0x7f33080747a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009930 tx=0x7f32f402bc30 comp rx=0 tx=0).stop 2024-03-26T19:11:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.860+0000 7f330d400640 1 -- 172.21.15.97:0/3678366472 shutdown_connections 2024-03-26T19:11:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.860+0000 7f330d400640 1 --2- 172.21.15.97:0/3678366472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f330810dee0 0x7f33081102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.860+0000 7f330d400640 1 --2- 172.21.15.97:0/3678366472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3308074320 0x7f33080747a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.860+0000 7f330d400640 1 --2- 172.21.15.97:0/3678366472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3308075900 0x7f3308073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.860+0000 7f330d400640 1 -- 172.21.15.97:0/3678366472 >> 172.21.15.97:0/3678366472 conn(0x7f33080fce80 msgr2=0x7f33080ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.860+0000 7f330d400640 1 -- 172.21.15.97:0/3678366472 shutdown_connections 2024-03-26T19:11:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.860+0000 7f330d400640 1 -- 172.21.15.97:0/3678366472 wait complete. 2024-03-26T19:11:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.861+0000 7f330d400640 1 Processor -- start 2024-03-26T19:11:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.861+0000 7f330d400640 1 -- start start 2024-03-26T19:11:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f330d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3308074320 0x7f330819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f330d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3308075900 0x7f330819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f330d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f330810dee0 0x7f33081a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f330d400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3308113b60 con 0x7f3308074320 2024-03-26T19:11:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f330d400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f33081139e0 con 0x7f330810dee0 2024-03-26T19:11:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f330d400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3308113ce0 con 0x7f3308075900 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f3307200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f330810dee0 0x7f33081a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f3307200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f330810dee0 0x7f33081a56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:43914/0 (socket says 172.21.15.97:43914) 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f3307200640 1 -- 172.21.15.97:0/3526513929 learned_addr learned my addr 172.21.15.97:0/3526513929 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f3306800640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3308074320 0x7f330819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f3305e00640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3308075900 0x7f330819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f3307200640 1 -- 172.21.15.97:0/3526513929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3308075900 msgr2=0x7f330819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.862+0000 7f3307200640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3308075900 0x7f330819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.863+0000 7f3307200640 1 -- 172.21.15.97:0/3526513929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3308074320 msgr2=0x7f330819e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.863+0000 7f3307200640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3308074320 0x7f330819e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.863+0000 7f3307200640 1 -- 172.21.15.97:0/3526513929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33081a5d50 con 0x7f330810dee0 2024-03-26T19:11:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.863+0000 7f3307200640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f330810dee0 0x7f33081a56e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f330000eb10 tx=0x7f330000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.863+0000 7f32ff400640 1 -- 172.21.15.97:0/3526513929 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300004b20 con 0x7f330810dee0 2024-03-26T19:11:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.863+0000 7f32ff400640 1 -- 172.21.15.97:0/3526513929 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3300004cc0 con 0x7f330810dee0 2024-03-26T19:11:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.863+0000 7f32ff400640 1 -- 172.21.15.97:0/3526513929 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300010760 con 0x7f330810dee0 2024-03-26T19:11:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.864+0000 7f330d400640 1 -- 172.21.15.97:0/3526513929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33081a5fe0 con 0x7f330810dee0 2024-03-26T19:11:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.864+0000 7f330d400640 1 -- 172.21.15.97:0/3526513929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33081a64b0 con 0x7f330810dee0 2024-03-26T19:11:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.868+0000 7f32ff400640 1 -- 172.21.15.97:0/3526513929 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f3300010900 con 0x7f330810dee0 2024-03-26T19:11:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.869+0000 7f3310051740 1 Processor -- start 2024-03-26T19:11:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.869+0000 7f3310051740 1 -- start start 2024-03-26T19:11:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.870+0000 7f3310051740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa9971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.870+0000 7f3310051740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa99c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.870+0000 7f3310051740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa99ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.870+0000 7f3310051740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5572fa8fbe80 con 0x5572fa996da0 2024-03-26T19:11:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.870+0000 7f3310051740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5572fa8a4660 con 0x5572fa997790 2024-03-26T19:11:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.870+0000 7f3310051740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5572fa8f7e30 con 0x5572fa99c900 2024-03-26T19:11:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.871+0000 7f3305e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa99ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.871+0000 7f3305e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa99ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:33578/0 (socket says 172.21.15.97:33578) 2024-03-26T19:11:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.871+0000 7f3305e00640 1 -- 172.21.15.97:0/1656790579 learned_addr learned my addr 172.21.15.97:0/1656790579 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.871+0000 7f3305e00640 1 --2- 172.21.15.97:0/1656790579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa9971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.872+0000 7f3306800640 1 --2- 172.21.15.97:0/1656790579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa99c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.872+0000 7f32ff400640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f32ec0a30c0 0x7f32ec0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.872+0000 7f3305e00640 1 -- 172.21.15.97:0/1656790579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 msgr2=0x5572fa99c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.872+0000 7f3305e00640 1 --2- 172.21.15.97:0/1656790579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa99c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.872+0000 7f3305e00640 1 -- 172.21.15.97:0/1656790579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 msgr2=0x5572fa9971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.872+0000 7f3305e00640 1 --2- 172.21.15.97:0/1656790579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa9971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.872+0000 7f3305e00640 1 -- 172.21.15.97:0/1656790579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572fa610210 con 0x5572fa99c900 2024-03-26T19:11:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.872+0000 7f3305e00640 1 --2- 172.21.15.97:0/1656790579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa99ed90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5572fa997ff0 tx=0x7f32f4005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ce0ab9350c07f99 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.873+0000 7f32f3400640 1 -- 172.21.15.97:0/1656790579 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f402cc80 con 0x5572fa99c900 2024-03-26T19:11:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.873+0000 7f32f3400640 1 -- 172.21.15.97:0/1656790579 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f40042f0 con 0x5572fa99c900 2024-03-26T19:11:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.873+0000 7f32f3400640 1 -- 172.21.15.97:0/1656790579 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40346a0 con 0x5572fa99c900 2024-03-26T19:11:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.874+0000 7f3310051740 1 -- 172.21.15.97:0/1656790579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 msgr2=0x5572fa99ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.874+0000 7f3310051740 1 --2- 172.21.15.97:0/1656790579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa99ed90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5572fa997ff0 tx=0x7f32f4005b50 comp rx=0 tx=0).stop 2024-03-26T19:11:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.874+0000 7f32ff400640 1 -- 172.21.15.97:0/3526513929 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f33000cf970 con 0x7f330810dee0 2024-03-26T19:11:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.875+0000 7f3307200640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f32ec0a30c0 0x7f32ec0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.875+0000 7f3310051740 1 -- 172.21.15.97:0/1656790579 shutdown_connections 2024-03-26T19:11:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.875+0000 7f3310051740 1 --2- 172.21.15.97:0/1656790579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa99ed90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.875+0000 7f3310051740 1 --2- 172.21.15.97:0/1656790579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa99c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.875+0000 7f3310051740 1 --2- 172.21.15.97:0/1656790579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa9971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.875+0000 7f3310051740 1 -- 172.21.15.97:0/1656790579 >> 172.21.15.97:0/1656790579 conn(0x5572fa992310 msgr2=0x5572fa992710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.876+0000 7f3310051740 1 -- 172.21.15.97:0/1656790579 shutdown_connections 2024-03-26T19:11:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.876+0000 7f3307200640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f32ec0a30c0 0x7f32ec0a5580 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f330009c000 tx=0x7f3300005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.876+0000 7f3310051740 1 -- 172.21.15.97:0/1656790579 wait complete. 2024-03-26T19:11:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.876+0000 7f3310051740 1 -- start start 2024-03-26T19:11:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.877+0000 7f3310051740 1 Processor -- start 2024-03-26T19:11:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.877+0000 7f3310051740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa9a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.877+0000 7f3310051740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.877+0000 7f3310051740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa9a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.877+0000 7f3310051740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5572fa99f850 con 0x5572fa996da0 2024-03-26T19:11:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.878+0000 7f3310051740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5572fa8b7ac0 con 0x5572fa997790 2024-03-26T19:11:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.878+0000 7f3310051740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5572fa99f9d0 con 0x5572fa99c900 2024-03-26T19:11:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.878+0000 7f3310051740 10 client.0 _ll_register_callbacks cb 0x5572fa8b7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:11:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.878+0000 7f3306800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.878+0000 7f3306800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:43932/0 (socket says 172.21.15.97:43932) 2024-03-26T19:11:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.878+0000 7f3306800640 1 -- 172.21.15.97:0/3075457433 learned_addr learned my addr 172.21.15.97:0/3075457433 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.879+0000 7f3305e00640 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa9a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.879+0000 7f3305e00640 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa9a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.879+0000 7f3306800640 1 -- 172.21.15.97:0/3075457433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 msgr2=0x5572fa9a8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.879+0000 7f3306800640 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa9a8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.879+0000 7f3306800640 1 -- 172.21.15.97:0/3075457433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 msgr2=0x5572fa9a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.879+0000 7f3306800640 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa9a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.879+0000 7f3306800640 1 -- 172.21.15.97:0/3075457433 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572fa6dc620 con 0x5572fa997790 2024-03-26T19:11:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.879+0000 7f3305e00640 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa9a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:11:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.880+0000 7f3306800640 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a1310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f32f8012700 tx=0x7f32f8012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f6fbec9a4f0a16b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.880+0000 7f32e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:11:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.880+0000 7f32e7e00640 1 -- 172.21.15.97:0/3075457433 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8004050 con 0x5572fa997790 2024-03-26T19:11:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.880+0000 7f32e7e00640 1 -- 172.21.15.97:0/3075457433 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f32f80041f0 con 0x5572fa997790 2024-03-26T19:11:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.880+0000 7f3310051740 1 -- 172.21.15.97:0/3075457433 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f40042f0 con 0x5572fa997790 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.881+0000 7f32e7e00640 1 -- 172.21.15.97:0/3075457433 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32f8023070 con 0x5572fa997790 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.881+0000 7f3310051740 10 client.26177 fetch_fsmap learned FSMap version 27 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.881+0000 7f3310051740 1 -- 172.21.15.97:0/3075457433 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5572fa9ab730 con 0x5572fa997790 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.881+0000 7f32e7e00640 1 -- 172.21.15.97:0/3075457433 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f40042f0 con 0x5572fa997790 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.881+0000 7f32e7e00640 1 -- 172.21.15.97:0/3075457433 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f32f8003ae0 con 0x5572fa997790 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.881+0000 7f3310051740 10 client.26177 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.881+0000 7f3310051740 10 client.26177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.881+0000 7f3310051740 20 client.26177 populate_metadata read hostname 'smithi097' 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.881+0000 7f3310051740 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5572fa9ace20 0x5572fa9af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.882+0000 7f3310051740 4 client.26177 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.882+0000 7f3310051740 1 -- 172.21.15.97:0/3075457433 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572fa9093c0 con 0x5572fa9ace20 2024-03-26T19:11:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.882+0000 7f3305e00640 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5572fa9ace20 0x5572fa9af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:22 smithi144 ceph-mon[36053]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 644 op/s 2024-03-26T19:11:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.882+0000 7f3305e00640 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5572fa9ace20 0x5572fa9af280 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88302c9a0501e5 server_cookie=b1215f31475e9bb1 in_seq=0 out_seq=0 2024-03-26T19:11:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.882+0000 7f32e7e00640 10 client.26177 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.884+0000 7f32e7e00640 1 -- 172.21.15.97:0/3075457433 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5572fa9093c0 con 0x5572fa9ace20 2024-03-26T19:11:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.884+0000 7f32e7e00640 10 client.26177 handle_command_reply: tid=0 2024-03-26T19:11:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.884+0000 7f3310051740 1 client.26177 shutdown 2024-03-26T19:11:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.886+0000 7f3310051740 1 -- 172.21.15.97:0/3075457433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 msgr2=0x5572fa9a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.886+0000 7f3310051740 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a1310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f32f8012700 tx=0x7f32f8012bc0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.887+0000 7f3310051740 1 -- 172.21.15.97:0/3075457433 shutdown_connections 2024-03-26T19:11:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.887+0000 7f3310051740 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5572fa9ace20 0x5572fa9af280 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.887+0000 7f3310051740 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa9a8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.887+0000 7f3310051740 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a1310 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.887+0000 7f3310051740 1 --2- 172.21.15.97:0/3075457433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa9a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.887+0000 7f3310051740 1 -- 172.21.15.97:0/3075457433 >> 172.21.15.97:0/3075457433 conn(0x5572fa992310 msgr2=0x5572fa991540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.887+0000 7f32e7e00640 0 client.26177 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.887+0000 7f3310051740 1 -- 172.21.15.97:0/3075457433 shutdown_connections 2024-03-26T19:11:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.888+0000 7f3310051740 1 -- 172.21.15.97:0/3075457433 wait complete. 2024-03-26T19:11:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.888+0000 7f3310051740 20 client.26177 trim_cache size 0 max 16384 2024-03-26T19:11:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.918+0000 7f3310051740 1 Processor -- start 2024-03-26T19:11:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.918+0000 7f3310051740 1 -- start start 2024-03-26T19:11:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.918+0000 7f3310051740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa995570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.918+0000 7f3310051740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.918+0000 7f3310051740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa995ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.918+0000 7f3310051740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f33081af280 con 0x5572fa996da0 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.918+0000 7f3310051740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b0dc0 con 0x5572fa997790 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.918+0000 7f3310051740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33081af950 con 0x5572fa99c900 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3306800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3306800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:43940/0 (socket says 172.21.15.97:43940) 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3306800640 1 -- 172.21.15.97:0/2366490268 learned_addr learned my addr 172.21.15.97:0/2366490268 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3305e00640 1 --2- 172.21.15.97:0/2366490268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa995570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3305e00640 1 --2- 172.21.15.97:0/2366490268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa995ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3306800640 1 -- 172.21.15.97:0/2366490268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 msgr2=0x5572fa995ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3306800640 1 --2- 172.21.15.97:0/2366490268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa995ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3306800640 1 -- 172.21.15.97:0/2366490268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 msgr2=0x5572fa995570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3306800640 1 --2- 172.21.15.97:0/2366490268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa995570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3306800640 1 -- 172.21.15.97:0/2366490268 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572fa98d730 con 0x5572fa997790 2024-03-26T19:11:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3305e00640 1 --2- 172.21.15.97:0/2366490268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa995570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:11:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.919+0000 7f3306800640 1 --2- 172.21.15.97:0/2366490268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a8310 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32f80058c0 tx=0x7f32f80107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b5541b327454d15 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.920+0000 7f32f2a00640 1 -- 172.21.15.97:0/2366490268 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f800f920 con 0x5572fa997790 2024-03-26T19:11:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.920+0000 7f32f2a00640 1 -- 172.21.15.97:0/2366490268 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f802c070 con 0x5572fa997790 2024-03-26T19:11:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.920+0000 7f32f2a00640 1 -- 172.21.15.97:0/2366490268 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8005360 con 0x5572fa997790 2024-03-26T19:11:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.921+0000 7f3310051740 1 -- 172.21.15.97:0/2366490268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 msgr2=0x5572fa9a8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.921+0000 7f3310051740 1 --2- 172.21.15.97:0/2366490268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a8310 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32f80058c0 tx=0x7f32f80107e0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.921+0000 7f3310051740 1 -- 172.21.15.97:0/2366490268 shutdown_connections 2024-03-26T19:11:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.921+0000 7f3310051740 1 --2- 172.21.15.97:0/2366490268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa99c900 0x5572fa995ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.921+0000 7f3310051740 1 --2- 172.21.15.97:0/2366490268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa997790 0x5572fa9a8310 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.921+0000 7f3310051740 1 --2- 172.21.15.97:0/2366490268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa996da0 0x5572fa995570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.921+0000 7f3310051740 1 -- 172.21.15.97:0/2366490268 >> 172.21.15.97:0/2366490268 conn(0x5572fa992310 msgr2=0x5572fa99fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.921+0000 7f3310051740 1 -- 172.21.15.97:0/2366490268 shutdown_connections 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.921+0000 7f3310051740 1 -- 172.21.15.97:0/2366490268 wait complete. 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.921+0000 7f3310051740 1 -- start start 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.922+0000 7f3310051740 1 Processor -- start 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.922+0000 7f3310051740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa996da0 0x5572fa9a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.922+0000 7f3310051740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa997790 0x5572fa9a5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.922+0000 7f3310051740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa99c900 0x5572faa11160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.922+0000 7f3310051740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b06f0 con 0x5572fa99c900 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.922+0000 7f3310051740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b0020 con 0x5572fa996da0 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.922+0000 7f3310051740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33081ad260 con 0x5572fa997790 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.922+0000 7f3310051740 10 client.0 _ll_register_callbacks cb 0x7f32f8009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:11:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.922+0000 7f3306800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa997790 0x5572fa9a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.922+0000 7f3306800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa997790 0x5572fa9a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:33606/0 (socket says 172.21.15.97:33606) 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.923+0000 7f3306800640 1 -- 172.21.15.97:0/240286211 learned_addr learned my addr 172.21.15.97:0/240286211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.923+0000 7f3305e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa996da0 0x5572fa9a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.923+0000 7f3305e00640 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa99c900 0x5572faa11160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.924+0000 7f3306800640 1 -- 172.21.15.97:0/240286211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa996da0 msgr2=0x5572fa9a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.924+0000 7f3306800640 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa996da0 0x5572fa9a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.924+0000 7f3306800640 1 -- 172.21.15.97:0/240286211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa99c900 msgr2=0x5572faa11160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.924+0000 7f3306800640 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa99c900 0x5572faa11160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.924+0000 7f3306800640 1 -- 172.21.15.97:0/240286211 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572fa9a5ca0 con 0x5572fa997790 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.924+0000 7f3305e00640 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa996da0 0x5572fa9a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.925+0000 7f3306800640 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa997790 0x5572fa9a5630 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f32f8003e50 tx=0x7f32f8010ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c71f6a9da2a6c6c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.925+0000 7f32e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.925+0000 7f32e7e00640 1 -- 172.21.15.97:0/240286211 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f80104d0 con 0x5572fa997790 2024-03-26T19:11:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.925+0000 7f32e7e00640 1 -- 172.21.15.97:0/240286211 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f32f80052f0 con 0x5572fa997790 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.925+0000 7f3310051740 1 -- 172.21.15.97:0/240286211 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f802c070 con 0x5572fa997790 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f32e7e00640 1 -- 172.21.15.97:0/240286211 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32f8040050 con 0x5572fa997790 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f3310051740 10 client.26164 fetch_fsmap learned FSMap version 27 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f3310051740 1 -- 172.21.15.97:0/240286211 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5572faa237c0 con 0x5572fa997790 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f32e7e00640 1 -- 172.21.15.97:0/240286211 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f802c070 con 0x5572fa997790 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f32e7e00640 1 -- 172.21.15.97:0/240286211 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f32f8003a30 con 0x5572fa997790 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f3310051740 10 client.26164 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f3310051740 10 client.26164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f3310051740 20 client.26164 populate_metadata read hostname 'smithi097' 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f3310051740 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5572faa24130 0x5572faa26550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f3310051740 4 client.26164 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.926+0000 7f3310051740 1 -- 172.21.15.97:0/240286211 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572fa903710 con 0x5572faa24130 2024-03-26T19:11:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.927+0000 7f3305e00640 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5572faa24130 0x5572faa26550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.927+0000 7f3305e00640 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5572faa24130 0x5572faa26550 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db70792e7c894485 server_cookie=76575aa668a52943 in_seq=0 out_seq=0 2024-03-26T19:11:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.927+0000 7f32e7e00640 10 client.26164 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.969+0000 7f32e7e00640 1 -- 172.21.15.97:0/240286211 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5572fa903710 con 0x5572faa24130 2024-03-26T19:11:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.969+0000 7f32e7e00640 10 client.26164 handle_command_reply: tid=0 2024-03-26T19:11:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.969+0000 7f3310051740 1 client.26164 shutdown 2024-03-26T19:11:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.969+0000 7f3310051740 1 -- 172.21.15.97:0/240286211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa997790 msgr2=0x5572fa9a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.969+0000 7f3310051740 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa997790 0x5572fa9a5630 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f32f8003e50 tx=0x7f32f8010ae0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.970+0000 7f3310051740 1 -- 172.21.15.97:0/240286211 shutdown_connections 2024-03-26T19:11:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.970+0000 7f3310051740 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5572faa24130 0x5572faa26550 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.970+0000 7f3310051740 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572fa99c900 0x5572faa11160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.970+0000 7f32e7e00640 0 client.26164 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.970+0000 7f3310051740 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572fa997790 0x5572fa9a5630 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.970+0000 7f3310051740 1 --2- 172.21.15.97:0/240286211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572fa996da0 0x5572fa9a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.970+0000 7f3310051740 1 -- 172.21.15.97:0/240286211 >> 172.21.15.97:0/240286211 conn(0x5572fa992310 msgr2=0x5572fa995570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.970+0000 7f3310051740 1 -- 172.21.15.97:0/240286211 shutdown_connections 2024-03-26T19:11:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.970+0000 7f3310051740 1 -- 172.21.15.97:0/240286211 wait complete. 2024-03-26T19:11:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.970+0000 7f3310051740 20 client.26164 trim_cache size 0 max 16384 2024-03-26T19:11:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:11:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:11:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:11:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:11:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 -- 172.21.15.97:0/3526513929 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f32ec0a30c0 msgr2=0x7f32ec0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f32ec0a30c0 0x7f32ec0a5580 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f330009c000 tx=0x7f3300005b40 comp rx=0 tx=0).stop 2024-03-26T19:11:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 -- 172.21.15.97:0/3526513929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f330810dee0 msgr2=0x7f33081a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f330810dee0 0x7f33081a56e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f330000eb10 tx=0x7f330000c2d0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 -- 172.21.15.97:0/3526513929 shutdown_connections 2024-03-26T19:11:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f32ec0a30c0 0x7f32ec0a5580 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f330810dee0 0x7f33081a56e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3308075900 0x7f330819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 --2- 172.21.15.97:0/3526513929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3308074320 0x7f330819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 -- 172.21.15.97:0/3526513929 >> 172.21.15.97:0/3526513929 conn(0x7f33080fce80 msgr2=0x7f3308101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 -- 172.21.15.97:0/3526513929 shutdown_connections 2024-03-26T19:11:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:22.971+0000 7f330d400640 1 -- 172.21.15.97:0/3526513929 wait complete. 2024-03-26T19:11:24.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:24 smithi105 ceph-mon[36270]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 524 op/s 2024-03-26T19:11:24.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:24 smithi097 ceph-mon[31955]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 524 op/s 2024-03-26T19:11:24.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:24 smithi144 ceph-mon[36053]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 524 op/s 2024-03-26T19:11:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 720.0 s, 1069.4 tps, lat 92.512 ms stddev 118.269 2024-03-26T19:11:26.466 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:26 smithi105 ceph-mon[36270]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.2 MiB/s wr, 610 op/s 2024-03-26T19:11:26.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:26 smithi097 ceph-mon[31955]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.2 MiB/s wr, 610 op/s 2024-03-26T19:11:26.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:26 smithi144 ceph-mon[36053]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.2 MiB/s wr, 610 op/s 2024-03-26T19:11:28.632 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:28 smithi144 ceph-mon[36053]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-03-26T19:11:28.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:28 smithi105 ceph-mon[36270]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-03-26T19:11:28.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:28 smithi097 ceph-mon[31955]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-03-26T19:11:30.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 725.0 s, 1261.0 tps, lat 80.137 ms stddev 107.315 2024-03-26T19:11:30.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:30 smithi105 ceph-mon[36270]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 579 op/s 2024-03-26T19:11:30.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:30 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:11:30.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:30 smithi097 ceph-mon[31955]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 579 op/s 2024-03-26T19:11:30.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:30 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:11:30.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:30 smithi144 ceph-mon[36053]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 579 op/s 2024-03-26T19:11:30.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:30 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:11:32.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:32 smithi097 ceph-mon[31955]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.3 MiB/s wr, 624 op/s 2024-03-26T19:11:32.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:32 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:11:32.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:32 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:11:32.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:32 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:11:32.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:32 smithi105 ceph-mon[36270]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.3 MiB/s wr, 624 op/s 2024-03-26T19:11:32.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:32 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:11:32.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:32 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:11:32.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:32 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:11:32.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:32 smithi144 ceph-mon[36053]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.3 MiB/s wr, 624 op/s 2024-03-26T19:11:32.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:32 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:11:32.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:32 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:11:32.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:32 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:11:34.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:34 smithi105 ceph-mon[36270]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-03-26T19:11:34.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:34 smithi097 ceph-mon[31955]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-03-26T19:11:34.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:34 smithi144 ceph-mon[36053]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-03-26T19:11:35.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 730.0 s, 1265.2 tps, lat 78.828 ms stddev 90.815 2024-03-26T19:11:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.160+0000 7f0fcfc00640 1 -- 172.21.15.97:0/3266018570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc8102ac0 msgr2=0x7f0fc8104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.160+0000 7f0fcfc00640 1 --2- 172.21.15.97:0/3266018570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc8102ac0 0x7f0fc8104f30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009900 tx=0x7f0fb402bc30 comp rx=0 tx=0).stop 2024-03-26T19:11:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.161+0000 7f0fcfc00640 1 -- 172.21.15.97:0/3266018570 shutdown_connections 2024-03-26T19:11:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.161+0000 7f0fcfc00640 1 --2- 172.21.15.97:0/3266018570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fc8105470 0x7f0fc81058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.161+0000 7f0fcfc00640 1 --2- 172.21.15.97:0/3266018570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc8102ac0 0x7f0fc8104f30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.161+0000 7f0fcfc00640 1 --2- 172.21.15.97:0/3266018570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc8100150 0x7f0fc8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.161+0000 7f0fcfc00640 1 -- 172.21.15.97:0/3266018570 >> 172.21.15.97:0/3266018570 conn(0x7f0fc80fbd50 msgr2=0x7f0fc80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.161+0000 7f0fcfc00640 1 -- 172.21.15.97:0/3266018570 shutdown_connections 2024-03-26T19:11:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.161+0000 7f0fcfc00640 1 -- 172.21.15.97:0/3266018570 wait complete. 2024-03-26T19:11:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.163+0000 7f0fcfc00640 1 Processor -- start 2024-03-26T19:11:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.163+0000 7f0fcfc00640 1 -- start start 2024-03-26T19:11:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.163+0000 7f0fcfc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fc8100150 0x7f0fc8072d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.163+0000 7f0fcfc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc8102ac0 0x7f0fc8073280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.163+0000 7f0fcfc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc8105470 0x7f0fc8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.163+0000 7f0fcfc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc806bd30 con 0x7f0fc8102ac0 2024-03-26T19:11:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.163+0000 7f0fcfc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc806bbb0 con 0x7f0fc8100150 2024-03-26T19:11:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.163+0000 7f0fcfc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc806beb0 con 0x7f0fc8105470 2024-03-26T19:11:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcd200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fc8100150 0x7f0fc8072d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcdc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc8105470 0x7f0fc8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc8102ac0 0x7f0fc8073280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcd200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fc8100150 0x7f0fc8072d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60176/0 (socket says 172.21.15.97:60176) 2024-03-26T19:11:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcd200640 1 -- 172.21.15.97:0/2939048358 learned_addr learned my addr 172.21.15.97:0/2939048358 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcd200640 1 -- 172.21.15.97:0/2939048358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc8105470 msgr2=0x7f0fc8071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcd200640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc8105470 0x7f0fc8071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcd200640 1 -- 172.21.15.97:0/2939048358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc8102ac0 msgr2=0x7f0fc8073280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcd200640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc8102ac0 0x7f0fc8073280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcd200640 1 -- 172.21.15.97:0/2939048358 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc8071ab0 con 0x7f0fc8100150 2024-03-26T19:11:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcdc00640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc8105470 0x7f0fc8071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:11:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.164+0000 7f0fcd200640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fc8100150 0x7f0fc8072d40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00d950 tx=0x7f0fbc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.165+0000 7f0fc5600640 1 -- 172.21.15.97:0/2939048358 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc017070 con 0x7f0fc8100150 2024-03-26T19:11:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.165+0000 7f0fc5600640 1 -- 172.21.15.97:0/2939048358 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fbc009c30 con 0x7f0fc8100150 2024-03-26T19:11:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.165+0000 7f0fc5600640 1 -- 172.21.15.97:0/2939048358 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00f6b0 con 0x7f0fc8100150 2024-03-26T19:11:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.165+0000 7f0fcfc00640 1 -- 172.21.15.97:0/2939048358 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc8071da0 con 0x7f0fc8100150 2024-03-26T19:11:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.165+0000 7f0fcfc00640 1 -- 172.21.15.97:0/2939048358 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc81aee80 con 0x7f0fc8100150 2024-03-26T19:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.170+0000 7f0fd2800740 1 Processor -- start 2024-03-26T19:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.171+0000 7f0fd2800740 1 -- start start 2024-03-26T19:11:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.171+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 0x561e22a9b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.171+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9b790 0x561e22aa0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.171+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aa2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.171+0000 7f0fd2800740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561e229ffe80 con 0x561e22a9ada0 2024-03-26T19:11:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.171+0000 7f0fd2800740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e229a8660 con 0x561e22a9b790 2024-03-26T19:11:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.171+0000 7f0fd2800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e229fbe30 con 0x561e22aa0900 2024-03-26T19:11:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.171+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aa2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.171+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aa2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:42938/0 (socket says 172.21.15.97:42938) 2024-03-26T19:11:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.171+0000 7f0fc7e00640 1 -- 172.21.15.97:0/359538944 learned_addr learned my addr 172.21.15.97:0/359538944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.172+0000 7f0fcdc00640 1 --2- 172.21.15.97:0/359538944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 0x561e22a9b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.172+0000 7f0fc7e00640 1 -- 172.21.15.97:0/359538944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9b790 msgr2=0x561e22aa0290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:11:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.172+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/359538944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9b790 0x561e22aa0290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.172+0000 7f0fc7e00640 1 -- 172.21.15.97:0/359538944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 msgr2=0x561e22a9b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.172+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/359538944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 0x561e22a9b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.172+0000 7f0fc7e00640 1 -- 172.21.15.97:0/359538944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e22714210 con 0x561e22aa0900 2024-03-26T19:11:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.173+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/359538944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aa2d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4000c00 tx=0x7f0fb4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14707f5379acdf41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.173+0000 7f0fba000640 1 -- 172.21.15.97:0/359538944 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4045020 con 0x561e22aa0900 2024-03-26T19:11:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.174+0000 7f0fba000640 1 -- 172.21.15.97:0/359538944 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb4004040 con 0x561e22aa0900 2024-03-26T19:11:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.174+0000 7f0fba000640 1 -- 172.21.15.97:0/359538944 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40046c0 con 0x561e22aa0900 2024-03-26T19:11:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.175+0000 7f0fc5600640 1 -- 172.21.15.97:0/2939048358 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f0fbc0109d0 con 0x7f0fc8100150 2024-03-26T19:11:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.175+0000 7f0fd2800740 1 -- 172.21.15.97:0/359538944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 msgr2=0x561e22aa2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.175+0000 7f0fd2800740 1 --2- 172.21.15.97:0/359538944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aa2d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4000c00 tx=0x7f0fb4005c00 comp rx=0 tx=0).stop 2024-03-26T19:11:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.176+0000 7f0fd2800740 1 -- 172.21.15.97:0/359538944 shutdown_connections 2024-03-26T19:11:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.176+0000 7f0fd2800740 1 --2- 172.21.15.97:0/359538944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aa2d90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.176+0000 7f0fd2800740 1 --2- 172.21.15.97:0/359538944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9b790 0x561e22aa0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.176+0000 7f0fd2800740 1 --2- 172.21.15.97:0/359538944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 0x561e22a9b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.176+0000 7f0fd2800740 1 -- 172.21.15.97:0/359538944 >> 172.21.15.97:0/359538944 conn(0x561e22a96310 msgr2=0x561e22a96710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.176+0000 7f0fd2800740 1 -- 172.21.15.97:0/359538944 shutdown_connections 2024-03-26T19:11:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.176+0000 7f0fd2800740 1 -- 172.21.15.97:0/359538944 wait complete. 2024-03-26T19:11:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.177+0000 7f0fd2800740 1 -- start start 2024-03-26T19:11:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.177+0000 7f0fd2800740 1 Processor -- start 2024-03-26T19:11:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22aa4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9b790 0x561e22aa52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fd2800740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561e22aa3850 con 0x561e22a9b790 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fd2800740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e229bbac0 con 0x561e22a9ada0 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fd2800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e22aa39d0 con 0x561e22aa0900 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fd2800740 10 client.0 _ll_register_callbacks cb 0x561e229bb490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fcdc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22aa4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fcdc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22aa4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60178/0 (socket says 172.21.15.97:60178) 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fcdc00640 1 -- 172.21.15.97:0/239597694 learned_addr learned my addr 172.21.15.97:0/239597694 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.178+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.179+0000 7f0fcd200640 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9b790 0x561e22aa52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.179+0000 7f0fc7e00640 1 -- 172.21.15.97:0/239597694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 msgr2=0x561e22aa4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.179+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22aa4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.179+0000 7f0fc7e00640 1 -- 172.21.15.97:0/239597694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9b790 msgr2=0x561e22aa52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.179+0000 7f0fc5600640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb00a3020 0x7f0fb00a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.179+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9b790 0x561e22aa52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.179+0000 7f0fc7e00640 1 -- 172.21.15.97:0/239597694 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e227e0620 con 0x561e22aa0900 2024-03-26T19:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.179+0000 7f0fc5600640 1 -- 172.21.15.97:0/2939048358 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0fbc0ceee0 con 0x7f0fc8100150 2024-03-26T19:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.180+0000 7f0fcdc00640 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22aa4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.180+0000 7f0fcd200640 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9b790 0x561e22aa52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.180+0000 7f0fcdc00640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb00a3020 0x7f0fb00a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.180+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aac330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0fb402c7b0 tx=0x7f0fb4004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34c2b5ae6341895c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.181+0000 7f0faea00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.181+0000 7f0faea00640 1 -- 172.21.15.97:0/239597694 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4045030 con 0x561e22aa0900 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.181+0000 7f0faea00640 1 -- 172.21.15.97:0/239597694 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0fb40035f0 con 0x561e22aa0900 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0fd2800740 1 -- 172.21.15.97:0/239597694 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb4004040 con 0x561e22aa0900 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0fcdc00640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb00a3020 0x7f0fb00a54e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0023e10 tx=0x7f0fc0025040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0faea00640 1 -- 172.21.15.97:0/239597694 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0fb405b050 con 0x561e22aa0900 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0fd2800740 10 client.26176 fetch_fsmap learned FSMap version 27 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0fd2800740 1 -- 172.21.15.97:0/239597694 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e22aa3850 con 0x561e22aa0900 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0faea00640 1 -- 172.21.15.97:0/239597694 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb4004040 con 0x561e22aa0900 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0faea00640 1 -- 172.21.15.97:0/239597694 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0fb4004bb0 con 0x561e22aa0900 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0fd2800740 10 client.26176 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0fd2800740 10 client.26176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0fd2800740 20 client.26176 populate_metadata read hostname 'smithi097' 2024-03-26T19:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0fd2800740 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561e22ab0930 0x561e22ab2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0fd2800740 4 client.26176 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.182+0000 7f0fd2800740 1 -- 172.21.15.97:0/239597694 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e22a0d3c0 con 0x561e22ab0930 2024-03-26T19:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.183+0000 7f0fcd200640 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561e22ab0930 0x561e22ab2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.183+0000 7f0faea00640 10 client.26176 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.183+0000 7f0fcd200640 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561e22ab0930 0x561e22ab2d20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9b5cbb835f614 server_cookie=892fb9ed93f29f6a in_seq=0 out_seq=0 2024-03-26T19:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0faea00640 1 -- 172.21.15.97:0/239597694 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561e22a0d3c0 con 0x561e22ab0930 2024-03-26T19:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0faea00640 10 client.26176 handle_command_reply: tid=0 2024-03-26T19:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0fd2800740 1 client.26176 shutdown 2024-03-26T19:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0fd2800740 1 -- 172.21.15.97:0/239597694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 msgr2=0x561e22aac330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0fd2800740 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aac330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0fb402c7b0 tx=0x7f0fb4004420 comp rx=0 tx=0).stop 2024-03-26T19:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0fd2800740 1 -- 172.21.15.97:0/239597694 shutdown_connections 2024-03-26T19:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0fd2800740 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561e22ab0930 0x561e22ab2d20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0fd2800740 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aac330 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0faea00640 0 client.26176 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0fd2800740 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9b790 0x561e22aa52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0fd2800740 1 --2- 172.21.15.97:0/239597694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22aa4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.184+0000 7f0fd2800740 1 -- 172.21.15.97:0/239597694 >> 172.21.15.97:0/239597694 conn(0x561e22a96310 msgr2=0x561e22a95540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.185+0000 7f0fd2800740 1 -- 172.21.15.97:0/239597694 shutdown_connections 2024-03-26T19:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.185+0000 7f0fd2800740 1 -- 172.21.15.97:0/239597694 wait complete. 2024-03-26T19:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.185+0000 7f0fd2800740 20 client.26176 trim_cache size 0 max 16384 2024-03-26T19:11:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.216+0000 7f0fd2800740 1 Processor -- start 2024-03-26T19:11:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.216+0000 7f0fd2800740 1 -- start start 2024-03-26T19:11:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.216+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22a9c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.217+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22a9b790 0x561e22aac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.217+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22aa0900 0x561e22ab7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.217+0000 7f0fd2800740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc81af1c0 con 0x561e22aa0900 2024-03-26T19:11:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.217+0000 7f0fd2800740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc81b0d00 con 0x561e22a9ada0 2024-03-26T19:11:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.217+0000 7f0fd2800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc81af890 con 0x561e22a9b790 2024-03-26T19:11:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.218+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22a9b790 0x561e22aac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.218+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22a9b790 0x561e22aac2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:42956/0 (socket says 172.21.15.97:42956) 2024-03-26T19:11:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.218+0000 7f0fc7e00640 1 -- 172.21.15.97:0/415715187 learned_addr learned my addr 172.21.15.97:0/415715187 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.218+0000 7f0fcdc00640 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22aa0900 0x561e22ab7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.218+0000 7f0fcd200640 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22a9c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.219+0000 7f0fc7e00640 1 -- 172.21.15.97:0/415715187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 msgr2=0x561e22a9c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.219+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22a9c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.219+0000 7f0fc7e00640 1 -- 172.21.15.97:0/415715187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22aa0900 msgr2=0x561e22ab7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.219+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22aa0900 0x561e22ab7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.219+0000 7f0fc7e00640 1 -- 172.21.15.97:0/415715187 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e22ab3350 con 0x561e22a9b790 2024-03-26T19:11:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.219+0000 7f0fcdc00640 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22aa0900 0x561e22ab7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:11:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.219+0000 7f0fcd200640 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22a9c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:11:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.220+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22a9b790 0x561e22aac2c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4005ab0 tx=0x7f0fb402c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0622a23f7f400a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.220+0000 7f0fb9600640 1 -- 172.21.15.97:0/415715187 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4058070 con 0x561e22a9b790 2024-03-26T19:11:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.220+0000 7f0fb9600640 1 -- 172.21.15.97:0/415715187 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb4041dd0 con 0x561e22a9b790 2024-03-26T19:11:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.220+0000 7f0fb9600640 1 -- 172.21.15.97:0/415715187 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4045040 con 0x561e22a9b790 2024-03-26T19:11:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.225+0000 7f0fd2800740 1 -- 172.21.15.97:0/415715187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22a9b790 msgr2=0x561e22aac2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.225+0000 7f0fd2800740 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22a9b790 0x561e22aac2c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4005ab0 tx=0x7f0fb402c880 comp rx=0 tx=0).stop 2024-03-26T19:11:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.225+0000 7f0fd2800740 1 -- 172.21.15.97:0/415715187 shutdown_connections 2024-03-26T19:11:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.225+0000 7f0fd2800740 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22aa0900 0x561e22ab7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.225+0000 7f0fd2800740 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22a9b790 0x561e22aac2c0 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4005ab0 tx=0x7f0fb402c880 comp rx=0 tx=0).stop 2024-03-26T19:11:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.225+0000 7f0fd2800740 1 --2- 172.21.15.97:0/415715187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22a9ada0 0x561e22a9c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.225+0000 7f0fd2800740 1 -- 172.21.15.97:0/415715187 >> 172.21.15.97:0/415715187 conn(0x561e22a96310 msgr2=0x561e22a9c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.227+0000 7f0fd2800740 1 -- 172.21.15.97:0/415715187 shutdown_connections 2024-03-26T19:11:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.227+0000 7f0fd2800740 1 -- 172.21.15.97:0/415715187 wait complete. 2024-03-26T19:11:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.227+0000 7f0fd2800740 1 -- start start 2024-03-26T19:11:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.228+0000 7f0fd2800740 1 Processor -- start 2024-03-26T19:11:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.230+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 0x561e22aa9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.230+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aaa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.231+0000 7f0fd2800740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22aab120 0x561e22aab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.231+0000 7f0fd2800740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc81b0630 con 0x561e22a9ada0 2024-03-26T19:11:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.231+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aaa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.232+0000 7f0fcdc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22aab120 0x561e22aab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.234+0000 7f0fd2800740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc81aff60 con 0x561e22aab120 2024-03-26T19:11:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.234+0000 7f0fd2800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc810d0c0 con 0x561e22aa0900 2024-03-26T19:11:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.234+0000 7f0fcdc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22aab120 0x561e22aab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:60206/0 (socket says 172.21.15.97:60206) 2024-03-26T19:11:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.234+0000 7f0fcdc00640 1 -- 172.21.15.97:0/126482855 learned_addr learned my addr 172.21.15.97:0/126482855 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.234+0000 7f0fcd200640 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 0x561e22aa9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.234+0000 7f0fd2800740 10 client.0 _ll_register_callbacks cb 0x7f0fb4005740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:11:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.234+0000 7f0fc7e00640 1 -- 172.21.15.97:0/126482855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22aab120 msgr2=0x561e22aab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.234+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22aab120 0x561e22aab5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.234+0000 7f0fc7e00640 1 -- 172.21.15.97:0/126482855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 msgr2=0x561e22aa9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.235+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 0x561e22aa9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.235+0000 7f0fc7e00640 1 -- 172.21.15.97:0/126482855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e22a95cc0 con 0x561e22aa0900 2024-03-26T19:11:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.235+0000 7f0fc7e00640 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aaa110 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x561e22a9c1d0 tx=0x7f0fb4005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2e7c0384e7fdd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.235+0000 7f0faea00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:11:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.235+0000 7f0faea00640 1 -- 172.21.15.97:0/126482855 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4058040 con 0x561e22aa0900 2024-03-26T19:11:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.235+0000 7f0faea00640 1 -- 172.21.15.97:0/126482855 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0fb40046c0 con 0x561e22aa0900 2024-03-26T19:11:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.236+0000 7f0fd2800740 1 -- 172.21.15.97:0/126482855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb4041dd0 con 0x561e22aa0900 2024-03-26T19:11:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.236+0000 7f0fcd200640 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 0x561e22aa9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:11:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.236+0000 7f0faea00640 1 -- 172.21.15.97:0/126482855 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0fb4045040 con 0x561e22aa0900 2024-03-26T19:11:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.236+0000 7f0fd2800740 10 client.26182 fetch_fsmap learned FSMap version 27 2024-03-26T19:11:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.236+0000 7f0fd2800740 1 -- 172.21.15.97:0/126482855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e22a97080 con 0x561e22aa0900 2024-03-26T19:11:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.237+0000 7f0fcdc00640 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22aab120 0x561e22aab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:11:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.237+0000 7f0faea00640 1 -- 172.21.15.97:0/126482855 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb4041dd0 con 0x561e22aa0900 2024-03-26T19:11:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.237+0000 7f0faea00640 1 -- 172.21.15.97:0/126482855 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0fb4060680 con 0x561e22aa0900 2024-03-26T19:11:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.237+0000 7f0fd2800740 10 client.26182 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:11:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.237+0000 7f0fd2800740 10 client.26182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:11:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.237+0000 7f0fd2800740 20 client.26182 populate_metadata read hostname 'smithi097' 2024-03-26T19:11:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.237+0000 7f0fd2800740 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561e22b176c0 0x561e22b19ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.237+0000 7f0fd2800740 4 client.26182 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:11:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.237+0000 7f0fd2800740 1 -- 172.21.15.97:0/126482855 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e22a07710 con 0x561e22b176c0 2024-03-26T19:11:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.239+0000 7f0fcd200640 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561e22b176c0 0x561e22b19ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.244+0000 7f0fcd200640 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561e22b176c0 0x561e22b19ae0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e962c41100f64ff server_cookie=d52303a68702fba in_seq=0 out_seq=0 2024-03-26T19:11:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.246+0000 7f0faea00640 10 client.26182 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.336+0000 7f0faea00640 1 -- 172.21.15.97:0/126482855 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e22a07710 con 0x561e22b176c0 2024-03-26T19:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.336+0000 7f0faea00640 10 client.26182 handle_command_reply: tid=0 2024-03-26T19:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.336+0000 7f0fd2800740 1 client.26182 shutdown 2024-03-26T19:11:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.337+0000 7f0fd2800740 1 -- 172.21.15.97:0/126482855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 msgr2=0x561e22aaa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.337+0000 7f0fd2800740 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aaa110 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x561e22a9c1d0 tx=0x7f0fb4005970 comp rx=0 tx=0).stop 2024-03-26T19:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.338+0000 7f0fd2800740 1 -- 172.21.15.97:0/126482855 shutdown_connections 2024-03-26T19:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.338+0000 7f0fd2800740 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561e22b176c0 0x561e22b19ae0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.338+0000 7f0fd2800740 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e22aab120 0x561e22aab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.338+0000 7f0fd2800740 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e22aa0900 0x561e22aaa110 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.338+0000 7f0fd2800740 1 --2- 172.21.15.97:0/126482855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e22a9ada0 0x561e22aa9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.338+0000 7f0fd2800740 1 -- 172.21.15.97:0/126482855 >> 172.21.15.97:0/126482855 conn(0x561e22a96310 msgr2=0x561e22a9c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.338+0000 7f0faea00640 0 client.26182 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.338+0000 7f0fd2800740 1 -- 172.21.15.97:0/126482855 shutdown_connections 2024-03-26T19:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.338+0000 7f0fd2800740 1 -- 172.21.15.97:0/126482855 wait complete. 2024-03-26T19:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.338+0000 7f0fd2800740 20 client.26182 trim_cache size 0 max 16384 2024-03-26T19:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 -- 172.21.15.97:0/2939048358 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb00a3020 msgr2=0x7f0fb00a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb00a3020 0x7f0fb00a54e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0023e10 tx=0x7f0fc0025040 comp rx=0 tx=0).stop 2024-03-26T19:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 -- 172.21.15.97:0/2939048358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fc8100150 msgr2=0x7f0fc8072d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fc8100150 0x7f0fc8072d40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00d950 tx=0x7f0fbc00de10 comp rx=0 tx=0).stop 2024-03-26T19:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 -- 172.21.15.97:0/2939048358 shutdown_connections 2024-03-26T19:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0fb00a3020 0x7f0fb00a54e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc8105470 0x7f0fc8071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc8102ac0 0x7f0fc8073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 --2- 172.21.15.97:0/2939048358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fc8100150 0x7f0fc8072d40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 -- 172.21.15.97:0/2939048358 >> 172.21.15.97:0/2939048358 conn(0x7f0fc80fbd50 msgr2=0x7f0fc8100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 -- 172.21.15.97:0/2939048358 shutdown_connections 2024-03-26T19:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:36.340+0000 7f0fcfc00640 1 -- 172.21.15.97:0/2939048358 wait complete. 2024-03-26T19:11:36.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:36 smithi105 ceph-mon[36270]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 MiB/s wr, 670 op/s 2024-03-26T19:11:36.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:36 smithi097 ceph-mon[31955]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 MiB/s wr, 670 op/s 2024-03-26T19:11:36.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:36 smithi144 ceph-mon[36053]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 MiB/s wr, 670 op/s 2024-03-26T19:11:38.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:38 smithi105 ceph-mon[36270]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 605 op/s 2024-03-26T19:11:38.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:38 smithi097 ceph-mon[31955]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 605 op/s 2024-03-26T19:11:38.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:38 smithi144 ceph-mon[36053]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 605 op/s 2024-03-26T19:11:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 735.0 s, 1291.6 tps, lat 76.874 ms stddev 90.227 2024-03-26T19:11:40.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:40 smithi105 ceph-mon[36270]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-03-26T19:11:40.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:40 smithi097 ceph-mon[31955]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-03-26T19:11:40.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:40 smithi144 ceph-mon[36053]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-03-26T19:11:42.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:42 smithi105 ceph-mon[36270]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 MiB/s wr, 699 op/s 2024-03-26T19:11:42.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:42 smithi097 ceph-mon[31955]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 MiB/s wr, 699 op/s 2024-03-26T19:11:42.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:42 smithi144 ceph-mon[36053]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 MiB/s wr, 699 op/s 2024-03-26T19:11:44.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:44 smithi105 ceph-mon[36270]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-03-26T19:11:44.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:44 smithi097 ceph-mon[31955]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-03-26T19:11:44.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:44 smithi144 ceph-mon[36053]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-03-26T19:11:45.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 740.0 s, 1367.0 tps, lat 74.551 ms stddev 91.501 2024-03-26T19:11:46.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:46 smithi105 ceph-mon[36270]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.0 MiB/s wr, 717 op/s 2024-03-26T19:11:46.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:46 smithi097 ceph-mon[31955]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.0 MiB/s wr, 717 op/s 2024-03-26T19:11:46.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:46 smithi144 ceph-mon[36053]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.0 MiB/s wr, 717 op/s 2024-03-26T19:11:48.719 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:48 smithi105 ceph-mon[36270]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 MiB/s wr, 597 op/s 2024-03-26T19:11:48.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:48 smithi097 ceph-mon[31955]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 MiB/s wr, 597 op/s 2024-03-26T19:11:48.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:48 smithi144 ceph-mon[36053]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 MiB/s wr, 597 op/s 2024-03-26T19:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.547+0000 7f0103200640 1 -- 172.21.15.97:0/314166568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00fc1056d0 msgr2=0x7f00fc10bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.547+0000 7f0103200640 1 --2- 172.21.15.97:0/314166568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00fc1056d0 0x7f00fc10bf60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009ca0 tx=0x7f00f402c100 comp rx=0 tx=0).stop 2024-03-26T19:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.547+0000 7f0103200640 1 -- 172.21.15.97:0/314166568 shutdown_connections 2024-03-26T19:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.547+0000 7f0103200640 1 --2- 172.21.15.97:0/314166568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00fc1056d0 0x7f00fc10bf60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.547+0000 7f0103200640 1 --2- 172.21.15.97:0/314166568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00fc104d10 0x7f00fc105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.547+0000 7f0103200640 1 --2- 172.21.15.97:0/314166568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00fc103b20 0x7f00fc103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.547+0000 7f0103200640 1 -- 172.21.15.97:0/314166568 >> 172.21.15.97:0/314166568 conn(0x7f00fc0ff0b0 msgr2=0x7f00fc1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.547+0000 7f0103200640 1 -- 172.21.15.97:0/314166568 shutdown_connections 2024-03-26T19:11:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.547+0000 7f0103200640 1 -- 172.21.15.97:0/314166568 wait complete. 2024-03-26T19:11:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.548+0000 7f0103200640 1 Processor -- start 2024-03-26T19:11:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.548+0000 7f0103200640 1 -- start start 2024-03-26T19:11:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.548+0000 7f0103200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00fc103b20 0x7f00fc199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.548+0000 7f0103200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00fc104d10 0x7f00fc19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f0103200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00fc1056d0 0x7f00fc1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f0103200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc10f810 con 0x7f00fc103b20 2024-03-26T19:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f0103200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc10f690 con 0x7f00fc1056d0 2024-03-26T19:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f0103200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc10f990 con 0x7f00fc104d10 2024-03-26T19:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00fc103b20 0x7f00fc199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f00fb400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00fc104d10 0x7f00fc19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f00fb400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00fc104d10 0x7f00fc19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:43526/0 (socket says 172.21.15.97:43526) 2024-03-26T19:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f00fb400640 1 -- 172.21.15.97:0/2749841215 learned_addr learned my addr 172.21.15.97:0/2749841215 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f0101200640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00fc1056d0 0x7f00fc1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f00fb400640 1 -- 172.21.15.97:0/2749841215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00fc1056d0 msgr2=0x7f00fc1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f00fb400640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00fc1056d0 0x7f00fc1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f00fb400640 1 -- 172.21.15.97:0/2749841215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00fc103b20 msgr2=0x7f00fc199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f00fb400640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00fc103b20 0x7f00fc199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.549+0000 7f00fb400640 1 -- 172.21.15.97:0/2749841215 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc1a19d0 con 0x7f00fc104d10 2024-03-26T19:11:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.550+0000 7f00fb400640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00fc104d10 0x7f00fc19a300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f00e800e9f0 tx=0x7f00e800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.550+0000 7f00f8c00640 1 -- 172.21.15.97:0/2749841215 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e800ccc0 con 0x7f00fc104d10 2024-03-26T19:11:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.551+0000 7f00f8c00640 1 -- 172.21.15.97:0/2749841215 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e8010910 con 0x7f00fc104d10 2024-03-26T19:11:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.551+0000 7f00f8c00640 1 -- 172.21.15.97:0/2749841215 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e80188a0 con 0x7f00fc104d10 2024-03-26T19:11:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.551+0000 7f0103200640 1 -- 172.21.15.97:0/2749841215 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc1a1cc0 con 0x7f00fc104d10 2024-03-26T19:11:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.551+0000 7f0103200640 1 -- 172.21.15.97:0/2749841215 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc1a21c0 con 0x7f00fc104d10 2024-03-26T19:11:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.555+0000 7f0105f7e740 1 Processor -- start 2024-03-26T19:11:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.555+0000 7f00f8c00640 1 -- 172.21.15.97:0/2749841215 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f00e8010ac0 con 0x7f00fc104d10 2024-03-26T19:11:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.555+0000 7f0105f7e740 1 -- start start 2024-03-26T19:11:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.555+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c4436051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.555+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c443605790 0x55c44360a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.555+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 0x55c44360cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.555+0000 7f0105f7e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c443569e80 con 0x55c44360a900 2024-03-26T19:11:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.555+0000 7f0105f7e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c443512660 con 0x55c443605790 2024-03-26T19:11:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.555+0000 7f0105f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c443565e30 con 0x55c443604da0 2024-03-26T19:11:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.556+0000 7f0101200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c4436051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.556+0000 7f0101200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c4436051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:43540/0 (socket says 172.21.15.97:43540) 2024-03-26T19:11:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.556+0000 7f0101200640 1 -- 172.21.15.97:0/528245789 learned_addr learned my addr 172.21.15.97:0/528245789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.556+0000 7f00fbe00640 1 --2- 172.21.15.97:0/528245789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c443605790 0x55c44360a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.556+0000 7f00fb400640 1 --2- 172.21.15.97:0/528245789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 0x55c44360cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.556+0000 7f0101200640 1 -- 172.21.15.97:0/528245789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c443605790 msgr2=0x55c44360a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.556+0000 7f0101200640 1 --2- 172.21.15.97:0/528245789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c443605790 0x55c44360a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.556+0000 7f0101200640 1 -- 172.21.15.97:0/528245789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 msgr2=0x55c44360cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.556+0000 7f0101200640 1 --2- 172.21.15.97:0/528245789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 0x55c44360cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.556+0000 7f0101200640 1 -- 172.21.15.97:0/528245789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44327e210 con 0x55c443604da0 2024-03-26T19:11:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.557+0000 7f0101200640 1 --2- 172.21.15.97:0/528245789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c4436051c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f00f402c710 tx=0x7f00f40323e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9d1ec1a82828a8b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.557+0000 7f00f8c00640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f00e00a3040 0x7f00e00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.557+0000 7f00f8c00640 1 -- 172.21.15.97:0/2749841215 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f00e80cf860 con 0x7f00fc104d10 2024-03-26T19:11:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.557+0000 7f00f1600640 1 -- 172.21.15.97:0/528245789 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4043070 con 0x55c443604da0 2024-03-26T19:11:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.557+0000 7f00f1600640 1 -- 172.21.15.97:0/528245789 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f4003970 con 0x55c443604da0 2024-03-26T19:11:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.557+0000 7f00f1600640 1 -- 172.21.15.97:0/528245789 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4039040 con 0x55c443604da0 2024-03-26T19:11:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.557+0000 7f00fbe00640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f00e00a3040 0x7f00e00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.558+0000 7f0105f7e740 1 -- 172.21.15.97:0/528245789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 msgr2=0x55c4436051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.558+0000 7f0105f7e740 1 --2- 172.21.15.97:0/528245789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c4436051c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f00f402c710 tx=0x7f00f40323e0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.558+0000 7f00fbe00640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f00e00a3040 0x7f00e00a5500 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55c443605b90 tx=0x7f00ec009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.558+0000 7f0105f7e740 1 -- 172.21.15.97:0/528245789 shutdown_connections 2024-03-26T19:11:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.558+0000 7f0105f7e740 1 --2- 172.21.15.97:0/528245789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 0x55c44360cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.558+0000 7f0105f7e740 1 --2- 172.21.15.97:0/528245789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c443605790 0x55c44360a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.558+0000 7f0105f7e740 1 --2- 172.21.15.97:0/528245789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c4436051c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.558+0000 7f0105f7e740 1 -- 172.21.15.97:0/528245789 >> 172.21.15.97:0/528245789 conn(0x55c443600310 msgr2=0x55c443600710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.559+0000 7f0105f7e740 1 -- 172.21.15.97:0/528245789 shutdown_connections 2024-03-26T19:11:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.559+0000 7f0105f7e740 1 -- 172.21.15.97:0/528245789 wait complete. 2024-03-26T19:11:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.560+0000 7f0105f7e740 1 -- start start 2024-03-26T19:11:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.560+0000 7f0105f7e740 1 Processor -- start 2024-03-26T19:11:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.560+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443604da0 0x55c443617610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.560+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 0x55c443617b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.560+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.560+0000 7f0105f7e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c44360d850 con 0x55c443604da0 2024-03-26T19:11:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.560+0000 7f0105f7e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c443525ac0 con 0x55c44360a900 2024-03-26T19:11:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.560+0000 7f0105f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c44360d9d0 con 0x55c443605790 2024-03-26T19:11:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f0105f7e740 10 client.0 _ll_register_callbacks cb 0x55c443525490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:11:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f00fb400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 0x55c443617b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f0101200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f00fb400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 0x55c443617b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:43542/0 (socket says 172.21.15.97:43542) 2024-03-26T19:11:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f0101200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:39094/0 (socket says 172.21.15.97:39094) 2024-03-26T19:11:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443604da0 0x55c443617610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f00fb400640 1 -- 172.21.15.97:0/756922011 learned_addr learned my addr 172.21.15.97:0/756922011 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f0101200640 1 -- 172.21.15.97:0/756922011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 msgr2=0x55c443617b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f0101200640 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 0x55c443617b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f0101200640 1 -- 172.21.15.97:0/756922011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443604da0 msgr2=0x55c443617610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f0101200640 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443604da0 0x55c443617610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f0101200640 1 -- 172.21.15.97:0/756922011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44334a620 con 0x55c44360a900 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.561+0000 7f00fbe00640 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443604da0 0x55c443617610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.562+0000 7f0101200640 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361eb90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f00f402c640 tx=0x7f00f4032ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2f8748d5f79e4c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.562+0000 7f00e6000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.562+0000 7f00e6000640 1 -- 172.21.15.97:0/756922011 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f404c070 con 0x55c44360a900 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.562+0000 7f00e6000640 1 -- 172.21.15.97:0/756922011 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f00f4004120 con 0x55c44360a900 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f0105f7e740 1 -- 172.21.15.97:0/756922011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f4003970 con 0x55c44360a900 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f00e6000640 1 -- 172.21.15.97:0/756922011 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f00f4058050 con 0x55c44360a900 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f0105f7e740 10 client.26210 fetch_fsmap learned FSMap version 27 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f0105f7e740 1 -- 172.21.15.97:0/756922011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c443621f00 con 0x55c44360a900 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f00e6000640 1 -- 172.21.15.97:0/756922011 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f4003970 con 0x55c44360a900 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f00e6000640 1 -- 172.21.15.97:0/756922011 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f00f40489c0 con 0x55c44360a900 2024-03-26T19:11:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f0105f7e740 10 client.26210 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:11:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f0105f7e740 10 client.26210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:11:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f0105f7e740 20 client.26210 populate_metadata read hostname 'smithi097' 2024-03-26T19:11:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f0105f7e740 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c443623250 0x55c443625640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.563+0000 7f0105f7e740 4 client.26210 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:11:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.564+0000 7f0105f7e740 1 -- 172.21.15.97:0/756922011 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4435773c0 con 0x55c443623250 2024-03-26T19:11:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.564+0000 7f00fbe00640 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c443623250 0x55c443625640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.565+0000 7f00fbe00640 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c443623250 0x55c443625640 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a519449f276de980 server_cookie=e75047a5f1e26764 in_seq=0 out_seq=0 2024-03-26T19:11:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.565+0000 7f00e6000640 10 client.26210 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.565+0000 7f00e6000640 1 -- 172.21.15.97:0/756922011 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c4435773c0 con 0x55c443623250 2024-03-26T19:11:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.565+0000 7f00e6000640 10 client.26210 handle_command_reply: tid=0 2024-03-26T19:11:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.566+0000 7f0105f7e740 1 client.26210 shutdown 2024-03-26T19:11:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.566+0000 7f0105f7e740 1 -- 172.21.15.97:0/756922011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 msgr2=0x55c44361eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.566+0000 7f0105f7e740 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361eb90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f00f402c640 tx=0x7f00f4032ad0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.566+0000 7f0105f7e740 1 -- 172.21.15.97:0/756922011 shutdown_connections 2024-03-26T19:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.566+0000 7f0105f7e740 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c443623250 0x55c443625640 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.566+0000 7f0105f7e740 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361eb90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.566+0000 7f0105f7e740 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 0x55c443617b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.566+0000 7f0105f7e740 1 --2- 172.21.15.97:0/756922011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443604da0 0x55c443617610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.566+0000 7f0105f7e740 1 -- 172.21.15.97:0/756922011 >> 172.21.15.97:0/756922011 conn(0x55c443600310 msgr2=0x55c4435ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.566+0000 7f00e6000640 0 client.26210 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.567+0000 7f0105f7e740 1 -- 172.21.15.97:0/756922011 shutdown_connections 2024-03-26T19:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.567+0000 7f0105f7e740 1 -- 172.21.15.97:0/756922011 wait complete. 2024-03-26T19:11:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.567+0000 7f0105f7e740 20 client.26210 trim_cache size 0 max 16384 2024-03-26T19:11:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.610+0000 7f0105f7e740 1 Processor -- start 2024-03-26T19:11:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.610+0000 7f0105f7e740 1 -- start start 2024-03-26T19:11:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.610+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c443604da0 0x55c4435fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.610+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 0x55c44361eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.610+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 0x55c4435ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.610+0000 7f0105f7e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1a6ba0 con 0x55c44360a900 2024-03-26T19:11:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.610+0000 7f0105f7e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1a86e0 con 0x55c443604da0 2024-03-26T19:11:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.610+0000 7f0105f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1a7270 con 0x55c443605790 2024-03-26T19:11:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.611+0000 7f00fb400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 0x55c44361eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.611+0000 7f0101200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 0x55c4435ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.613+0000 7f0101200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 0x55c4435ff1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60042/0 (socket says 172.21.15.97:60042) 2024-03-26T19:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.613+0000 7f0101200640 1 -- 172.21.15.97:0/2572083237 learned_addr learned my addr 172.21.15.97:0/2572083237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.611+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c443604da0 0x55c4435fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.613+0000 7f0101200640 1 -- 172.21.15.97:0/2572083237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 msgr2=0x55c44361eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.613+0000 7f0101200640 1 --2- 172.21.15.97:0/2572083237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 0x55c44361eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.613+0000 7f0101200640 1 -- 172.21.15.97:0/2572083237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c443604da0 msgr2=0x55c4435fecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.613+0000 7f0101200640 1 --2- 172.21.15.97:0/2572083237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c443604da0 0x55c4435fecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.613+0000 7f0101200640 1 -- 172.21.15.97:0/2572083237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c443600d80 con 0x55c44360a900 2024-03-26T19:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.615+0000 7f0101200640 1 --2- 172.21.15.97:0/2572083237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 0x55c4435ff1f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f00f4002230 tx=0x7f00f4032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa059d43851bbb9a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.615+0000 7f00f0c00640 1 -- 172.21.15.97:0/2572083237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f404c070 con 0x55c44360a900 2024-03-26T19:11:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.615+0000 7f00f0c00640 1 -- 172.21.15.97:0/2572083237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f4055020 con 0x55c44360a900 2024-03-26T19:11:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.615+0000 7f00f0c00640 1 -- 172.21.15.97:0/2572083237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4039070 con 0x55c44360a900 2024-03-26T19:11:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.616+0000 7f0105f7e740 1 -- 172.21.15.97:0/2572083237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 msgr2=0x55c4435ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.616+0000 7f0105f7e740 1 --2- 172.21.15.97:0/2572083237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 0x55c4435ff1f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f00f4002230 tx=0x7f00f4032590 comp rx=0 tx=0).stop 2024-03-26T19:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.617+0000 7f0105f7e740 1 -- 172.21.15.97:0/2572083237 shutdown_connections 2024-03-26T19:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.617+0000 7f0105f7e740 1 --2- 172.21.15.97:0/2572083237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c44360a900 0x55c4435ff1f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.617+0000 7f0105f7e740 1 --2- 172.21.15.97:0/2572083237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443605790 0x55c44361eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.617+0000 7f0105f7e740 1 --2- 172.21.15.97:0/2572083237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c443604da0 0x55c4435fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.617+0000 7f0105f7e740 1 -- 172.21.15.97:0/2572083237 >> 172.21.15.97:0/2572083237 conn(0x55c443600310 msgr2=0x55c443603e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.617+0000 7f0105f7e740 1 -- 172.21.15.97:0/2572083237 shutdown_connections 2024-03-26T19:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.617+0000 7f0105f7e740 1 -- 172.21.15.97:0/2572083237 wait complete. 2024-03-26T19:11:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.617+0000 7f0105f7e740 1 -- start start 2024-03-26T19:11:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.618+0000 7f0105f7e740 1 Processor -- start 2024-03-26T19:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.619+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c44361c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.619+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443605790 0x55c44361c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.619+0000 7f0105f7e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.619+0000 7f0105f7e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1a8010 con 0x55c443605790 2024-03-26T19:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.619+0000 7f0105f7e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1a7940 con 0x55c44360a900 2024-03-26T19:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.619+0000 7f0105f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1a4b80 con 0x55c443604da0 2024-03-26T19:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.619+0000 7f0105f7e740 10 client.0 _ll_register_callbacks cb 0x7f00f4009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.620+0000 7f00fb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443605790 0x55c44361c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.620+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c44361c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.620+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c44361c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:43564/0 (socket says 172.21.15.97:43564) 2024-03-26T19:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.620+0000 7f00fbe00640 1 -- 172.21.15.97:0/3590054028 learned_addr learned my addr 172.21.15.97:0/3590054028 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.621+0000 7f0101200640 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.622+0000 7f00fbe00640 1 -- 172.21.15.97:0/3590054028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 msgr2=0x55c44361d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.622+0000 7f00fbe00640 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.622+0000 7f00fbe00640 1 -- 172.21.15.97:0/3590054028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443605790 msgr2=0x55c44361c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.622+0000 7f00fbe00640 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443605790 0x55c44361c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.622+0000 7f00fbe00640 1 -- 172.21.15.97:0/3590054028 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44361dab0 con 0x55c443604da0 2024-03-26T19:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.622+0000 7f0101200640 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.626+0000 7f00fbe00640 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c44361c010 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55c443606200 tx=0x7f00ec014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=321688b4027856b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.626+0000 7f00e6000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:11:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.626+0000 7f00e6000640 1 -- 172.21.15.97:0/3590054028 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec023c70 con 0x55c443604da0 2024-03-26T19:11:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.627+0000 7f00e6000640 1 -- 172.21.15.97:0/3590054028 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f00ec034480 con 0x55c443604da0 2024-03-26T19:11:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.628+0000 7f0105f7e740 1 -- 172.21.15.97:0/3590054028 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f4055020 con 0x55c443604da0 2024-03-26T19:11:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.628+0000 7f00e6000640 1 -- 172.21.15.97:0/3590054028 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f00ec023e10 con 0x55c443604da0 2024-03-26T19:11:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.629+0000 7f0105f7e740 10 client.26200 fetch_fsmap learned FSMap version 27 2024-03-26T19:11:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.629+0000 7f0105f7e740 1 -- 172.21.15.97:0/3590054028 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c44361ee90 con 0x55c443604da0 2024-03-26T19:11:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.630+0000 7f00e6000640 1 -- 172.21.15.97:0/3590054028 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f4055020 con 0x55c443604da0 2024-03-26T19:11:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.630+0000 7f00e6000640 1 -- 172.21.15.97:0/3590054028 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f00ec03e6c0 con 0x55c443604da0 2024-03-26T19:11:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.630+0000 7f0105f7e740 10 client.26200 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:11:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.630+0000 7f0105f7e740 10 client.26200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:11:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.630+0000 7f0105f7e740 20 client.26200 populate_metadata read hostname 'smithi097' 2024-03-26T19:11:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.630+0000 7f0105f7e740 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c443689b90 0x55c44368bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:11:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.631+0000 7f0105f7e740 4 client.26200 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:11:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.631+0000 7f0105f7e740 1 -- 172.21.15.97:0/3590054028 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c443571710 con 0x55c443689b90 2024-03-26T19:11:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.631+0000 7f00fb400640 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c443689b90 0x55c44368bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:11:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.632+0000 7f00e6000640 10 client.26200 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.632+0000 7f00fb400640 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c443689b90 0x55c44368bfb0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c07b9ff6e6ad6f1 server_cookie=fd949d3c596f3f3c in_seq=0 out_seq=0 2024-03-26T19:11:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.667+0000 7f00e6000640 1 -- 172.21.15.97:0/3590054028 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c443571710 con 0x55c443689b90 2024-03-26T19:11:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.667+0000 7f00e6000640 10 client.26200 handle_command_reply: tid=0 2024-03-26T19:11:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.667+0000 7f0105f7e740 1 client.26200 shutdown 2024-03-26T19:11:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 1 -- 172.21.15.97:0/3590054028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 msgr2=0x55c44361c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c44361c010 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55c443606200 tx=0x7f00ec014000 comp rx=0 tx=0).stop 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 1 -- 172.21.15.97:0/3590054028 shutdown_connections 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c443689b90 0x55c44368bfb0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c44360a900 0x55c44361d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c443605790 0x55c44361c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 1 --2- 172.21.15.97:0/3590054028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c443604da0 0x55c44361c010 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 1 -- 172.21.15.97:0/3590054028 >> 172.21.15.97:0/3590054028 conn(0x55c443600310 msgr2=0x55c443606230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f00e6000640 0 client.26200 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 1 -- 172.21.15.97:0/3590054028 shutdown_connections 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 1 -- 172.21.15.97:0/3590054028 wait complete. 2024-03-26T19:11:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.668+0000 7f0105f7e740 20 client.26200 trim_cache size 0 max 16384 2024-03-26T19:11:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:11:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:11:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:11:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:11:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.669+0000 7f0103200640 1 -- 172.21.15.97:0/2749841215 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f00e00a3040 msgr2=0x7f00e00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.669+0000 7f0103200640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f00e00a3040 0x7f00e00a5500 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55c443605b90 tx=0x7f00ec009110 comp rx=0 tx=0).stop 2024-03-26T19:11:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.669+0000 7f0103200640 1 -- 172.21.15.97:0/2749841215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00fc104d10 msgr2=0x7f00fc19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:11:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.669+0000 7f0103200640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00fc104d10 0x7f00fc19a300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f00e800e9f0 tx=0x7f00e800edf0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.669+0000 7f0103200640 1 -- 172.21.15.97:0/2749841215 shutdown_connections 2024-03-26T19:11:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.669+0000 7f0103200640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f00e00a3040 0x7f00e00a5500 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.669+0000 7f0103200640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00fc1056d0 0x7f00fc1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.669+0000 7f0103200640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00fc104d10 0x7f00fc19a300 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.669+0000 7f0103200640 1 --2- 172.21.15.97:0/2749841215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00fc103b20 0x7f00fc199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:11:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.669+0000 7f0103200640 1 -- 172.21.15.97:0/2749841215 >> 172.21.15.97:0/2749841215 conn(0x7f00fc0ff0b0 msgr2=0x7f00fc107fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:11:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.670+0000 7f0103200640 1 -- 172.21.15.97:0/2749841215 shutdown_connections 2024-03-26T19:11:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:11:49.670+0000 7f0103200640 1 -- 172.21.15.97:0/2749841215 wait complete. 2024-03-26T19:11:50.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 745.0 s, 1412.0 tps, lat 70.188 ms stddev 82.176 2024-03-26T19:11:50.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:50 smithi105 ceph-mon[36270]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.7 MiB/s wr, 629 op/s 2024-03-26T19:11:50.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:50 smithi097 ceph-mon[31955]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.7 MiB/s wr, 629 op/s 2024-03-26T19:11:50.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:50 smithi144 ceph-mon[36053]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.7 MiB/s wr, 629 op/s 2024-03-26T19:11:52.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:52 smithi097 ceph-mon[31955]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 709 op/s 2024-03-26T19:11:52.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:52 smithi144 ceph-mon[36053]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 709 op/s 2024-03-26T19:11:52.966 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:52 smithi105 ceph-mon[36270]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 709 op/s 2024-03-26T19:11:53.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:53 smithi097 ceph-mon[31955]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-03-26T19:11:53.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:53 smithi144 ceph-mon[36053]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-03-26T19:11:53.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:53 smithi105 ceph-mon[36270]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-03-26T19:11:55.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 750.0 s, 1286.6 tps, lat 76.553 ms stddev 91.348 2024-03-26T19:11:56.669 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:56 smithi105 ceph-mon[36270]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 MiB/s wr, 673 op/s 2024-03-26T19:11:56.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:56 smithi097 ceph-mon[31955]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 MiB/s wr, 673 op/s 2024-03-26T19:11:56.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:56 smithi144 ceph-mon[36053]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 MiB/s wr, 673 op/s 2024-03-26T19:11:58.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:11:58 smithi105 ceph-mon[36270]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-03-26T19:11:58.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:11:58 smithi097 ceph-mon[31955]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-03-26T19:11:58.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:11:58 smithi144 ceph-mon[36053]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-03-26T19:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 755.0 s, 1186.2 tps, lat 85.775 ms stddev 105.449 2024-03-26T19:12:00.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:00 smithi105 ceph-mon[36270]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 596 op/s 2024-03-26T19:12:00.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:00 smithi097 ceph-mon[31955]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 596 op/s 2024-03-26T19:12:00.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:00 smithi144 ceph-mon[36053]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 596 op/s 2024-03-26T19:12:02.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:02 smithi097 ceph-mon[31955]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 644 op/s 2024-03-26T19:12:02.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:02 smithi105 ceph-mon[36270]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 644 op/s 2024-03-26T19:12:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.855+0000 7f9f3ca00640 1 -- 172.21.15.97:0/1924304508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f38109070 msgr2=0x7f9f381094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.855+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/1924304508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f38109070 0x7f9f381094f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009900 tx=0x7f9f2002bc30 comp rx=0 tx=0).stop 2024-03-26T19:12:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.857+0000 7f9f3ca00640 1 -- 172.21.15.97:0/1924304508 shutdown_connections 2024-03-26T19:12:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.857+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/1924304508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38109a30 0x7f9f38110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.857+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/1924304508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f38109070 0x7f9f381094f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.857+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/1924304508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38107f20 0x7f9f38108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.857+0000 7f9f3ca00640 1 -- 172.21.15.97:0/1924304508 >> 172.21.15.97:0/1924304508 conn(0x7f9f38076ed0 msgr2=0x7f9f380792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.857+0000 7f9f3ca00640 1 -- 172.21.15.97:0/1924304508 shutdown_connections 2024-03-26T19:12:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.857+0000 7f9f3ca00640 1 -- 172.21.15.97:0/1924304508 wait complete. 2024-03-26T19:12:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.859+0000 7f9f3ca00640 1 Processor -- start 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.859+0000 7f9f3ca00640 1 -- start start 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.859+0000 7f9f3ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38107f20 0x7f9f3819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.859+0000 7f9f3ca00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f38109070 0x7f9f3819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.859+0000 7f9f3ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38109a30 0x7f9f381a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.859+0000 7f9f3ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38113c00 con 0x7f9f38107f20 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.859+0000 7f9f3ca00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38113a80 con 0x7f9f38109070 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.859+0000 7f9f3ca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38113d80 con 0x7f9f38109a30 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.860+0000 7f9f36800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38109a30 0x7f9f381a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.860+0000 7f9f2fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f38109070 0x7f9f3819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.860+0000 7f9f36800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38109a30 0x7f9f381a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:46950/0 (socket says 172.21.15.97:46950) 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.860+0000 7f9f35e00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38107f20 0x7f9f3819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.860+0000 7f9f36800640 1 -- 172.21.15.97:0/3128961743 learned_addr learned my addr 172.21.15.97:0/3128961743 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.860+0000 7f9f2fe00640 1 -- 172.21.15.97:0/3128961743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38109a30 msgr2=0x7f9f381a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.860+0000 7f9f2fe00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38109a30 0x7f9f381a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.860+0000 7f9f2fe00640 1 -- 172.21.15.97:0/3128961743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38107f20 msgr2=0x7f9f3819e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.860+0000 7f9f2fe00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38107f20 0x7f9f3819e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.860+0000 7f9f2fe00640 1 -- 172.21.15.97:0/3128961743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f381a5d70 con 0x7f9f38109070 2024-03-26T19:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.861+0000 7f9f36800640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38109a30 0x7f9f381a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.861+0000 7f9f2fe00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f38109070 0x7f9f3819e690 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009230 tx=0x7f9f2002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.861+0000 7f9f2ea00640 1 -- 172.21.15.97:0/3128961743 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20043020 con 0x7f9f38109070 2024-03-26T19:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.861+0000 7f9f2ea00640 1 -- 172.21.15.97:0/3128961743 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f200054e0 con 0x7f9f38109070 2024-03-26T19:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.861+0000 7f9f2ea00640 1 -- 172.21.15.97:0/3128961743 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20003c60 con 0x7f9f38109070 2024-03-26T19:12:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.861+0000 7f9f3ca00640 1 -- 172.21.15.97:0/3128961743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f381a6000 con 0x7f9f38109070 2024-03-26T19:12:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.862+0000 7f9f3ca00640 1 -- 172.21.15.97:0/3128961743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f381a6570 con 0x7f9f38109070 2024-03-26T19:12:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.866+0000 7f9f2ea00640 1 -- 172.21.15.97:0/3128961743 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f9f20046020 con 0x7f9f38109070 2024-03-26T19:12:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.866+0000 7f9f3f604740 1 Processor -- start 2024-03-26T19:12:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.866+0000 7f9f3f604740 1 -- start start 2024-03-26T19:12:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.867+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de6a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.867+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de6f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.867+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 0x55b27de71d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.867+0000 7f9f3f604740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b27ddcee80 con 0x55b27de6f900 2024-03-26T19:12:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.867+0000 7f9f3f604740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b27dd77660 con 0x55b27de69da0 2024-03-26T19:12:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.867+0000 7f9f3f604740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b27ddcae30 con 0x55b27de6a790 2024-03-26T19:12:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.868+0000 7f9f35e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de6f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.868+0000 7f9f36800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de6a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.868+0000 7f9f35e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de6f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:46966/0 (socket says 172.21.15.97:46966) 2024-03-26T19:12:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.868+0000 7f9f35e00640 1 -- 172.21.15.97:0/2817799700 learned_addr learned my addr 172.21.15.97:0/2817799700 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.868+0000 7f9f2ea00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a3040 0x7f9f1c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.868+0000 7f9f2fe00640 1 --2- 172.21.15.97:0/2817799700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 0x55b27de71d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.869+0000 7f9f36800640 1 -- 172.21.15.97:0/2817799700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 msgr2=0x55b27de6f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.869+0000 7f9f36800640 1 --2- 172.21.15.97:0/2817799700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de6f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.869+0000 7f9f36800640 1 -- 172.21.15.97:0/2817799700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 msgr2=0x55b27de71d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.869+0000 7f9f36800640 1 --2- 172.21.15.97:0/2817799700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 0x55b27de71d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.869+0000 7f9f36800640 1 -- 172.21.15.97:0/2817799700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b27dae3210 con 0x55b27de69da0 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.869+0000 7f9f35e00640 1 --2- 172.21.15.97:0/2817799700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de6f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.869+0000 7f9f2fe00640 1 --2- 172.21.15.97:0/2817799700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 0x55b27de71d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.869+0000 7f9f2ea00640 1 -- 172.21.15.97:0/3128961743 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9f200f0f10 con 0x7f9f38109070 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.869+0000 7f9f36800640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a3040 0x7f9f1c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.869+0000 7f9f36800640 1 --2- 172.21.15.97:0/2817799700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de6a1c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f300114b0 tx=0x7f9f30011970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c4aafa6e58fa7c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.870+0000 7f9f27000640 1 -- 172.21.15.97:0/2817799700 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3001ca60 con 0x55b27de69da0 2024-03-26T19:12:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.870+0000 7f9f27000640 1 -- 172.21.15.97:0/2817799700 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3001cc00 con 0x55b27de69da0 2024-03-26T19:12:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.870+0000 7f9f27000640 1 -- 172.21.15.97:0/2817799700 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3001b370 con 0x55b27de69da0 2024-03-26T19:12:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.871+0000 7f9f3f604740 1 -- 172.21.15.97:0/2817799700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 msgr2=0x55b27de6a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.871+0000 7f9f36800640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a3040 0x7f9f1c0a5500 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9f30023000 tx=0x7f9f30020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.871+0000 7f9f3f604740 1 --2- 172.21.15.97:0/2817799700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de6a1c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f300114b0 tx=0x7f9f30011970 comp rx=0 tx=0).stop 2024-03-26T19:12:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.871+0000 7f9f3f604740 1 -- 172.21.15.97:0/2817799700 shutdown_connections 2024-03-26T19:12:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.871+0000 7f9f3f604740 1 --2- 172.21.15.97:0/2817799700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 0x55b27de71d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.871+0000 7f9f3f604740 1 --2- 172.21.15.97:0/2817799700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de6f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.871+0000 7f9f3f604740 1 --2- 172.21.15.97:0/2817799700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de6a1c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.871+0000 7f9f3f604740 1 -- 172.21.15.97:0/2817799700 >> 172.21.15.97:0/2817799700 conn(0x55b27de65310 msgr2=0x55b27de65710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.871+0000 7f9f3f604740 1 -- 172.21.15.97:0/2817799700 shutdown_connections 2024-03-26T19:12:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.871+0000 7f9f3f604740 1 -- 172.21.15.97:0/2817799700 wait complete. 2024-03-26T19:12:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.872+0000 7f9f3f604740 1 -- start start 2024-03-26T19:12:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.872+0000 7f9f3f604740 1 Processor -- start 2024-03-26T19:12:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.872+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de73e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.872+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6a790 0x55b27de74340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.872+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6f900 0x55b27de7b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f3f604740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b27de728b0 con 0x55b27de6a790 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f3f604740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b27dd8aac0 con 0x55b27de69da0 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f3f604740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b27de72a30 con 0x55b27de6f900 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f3f604740 10 client.0 _ll_register_callbacks cb 0x55b27dd8a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f36800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6f900 0x55b27de7b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f36800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6f900 0x55b27de7b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:46976/0 (socket says 172.21.15.97:46976) 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f35e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de73e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f36800640 1 -- 172.21.15.97:0/1430086757 learned_addr learned my addr 172.21.15.97:0/1430086757 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f2fe00640 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6a790 0x55b27de74340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f36800640 1 -- 172.21.15.97:0/1430086757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 msgr2=0x55b27de73e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f36800640 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de73e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f36800640 1 -- 172.21.15.97:0/1430086757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6a790 msgr2=0x55b27de74340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f36800640 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6a790 0x55b27de74340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f36800640 1 -- 172.21.15.97:0/1430086757 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b27dbaf620 con 0x55b27de6f900 2024-03-26T19:12:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f35e00640 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de73e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:12:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.873+0000 7f9f2fe00640 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6a790 0x55b27de74340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:12:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.874+0000 7f9f36800640 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6f900 0x55b27de7b310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9f30005b70 tx=0x7f9f30004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf337773db2ad04 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.874+0000 7f9f1b400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:12:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.875+0000 7f9f1b400640 1 -- 172.21.15.97:0/1430086757 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30009c20 con 0x55b27de6f900 2024-03-26T19:12:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.875+0000 7f9f1b400640 1 -- 172.21.15.97:0/1430086757 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9f3001c7e0 con 0x55b27de6f900 2024-03-26T19:12:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.875+0000 7f9f3f604740 1 -- 172.21.15.97:0/1430086757 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3001cc00 con 0x55b27de6f900 2024-03-26T19:12:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.876+0000 7f9f1b400640 1 -- 172.21.15.97:0/1430086757 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f3001c9e0 con 0x55b27de6f900 2024-03-26T19:12:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.877+0000 7f9f3f604740 10 client.26212 fetch_fsmap learned FSMap version 27 2024-03-26T19:12:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.877+0000 7f9f3f604740 1 -- 172.21.15.97:0/1430086757 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b27de7e680 con 0x55b27de6f900 2024-03-26T19:12:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.877+0000 7f9f1b400640 1 -- 172.21.15.97:0/1430086757 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3001cc00 con 0x55b27de6f900 2024-03-26T19:12:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.878+0000 7f9f1b400640 1 -- 172.21.15.97:0/1430086757 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9f30012b40 con 0x55b27de6f900 2024-03-26T19:12:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.878+0000 7f9f3f604740 10 client.26212 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:12:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.878+0000 7f9f3f604740 10 client.26212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:12:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.878+0000 7f9f3f604740 20 client.26212 populate_metadata read hostname 'smithi097' 2024-03-26T19:12:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.878+0000 7f9f3f604740 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b27de7fba0 0x55b27de81f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.878+0000 7f9f3f604740 4 client.26212 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:12:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.878+0000 7f9f3f604740 1 -- 172.21.15.97:0/1430086757 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b27dddc3c0 con 0x55b27de7fba0 2024-03-26T19:12:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.879+0000 7f9f35e00640 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b27de7fba0 0x55b27de81f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.880+0000 7f9f35e00640 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b27de7fba0 0x55b27de81f90 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991a0f6c176f21ce server_cookie=b66afbc5f9c5e18a in_seq=0 out_seq=0 2024-03-26T19:12:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.880+0000 7f9f1b400640 10 client.26212 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.881+0000 7f9f1b400640 1 -- 172.21.15.97:0/1430086757 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b27dddc3c0 con 0x55b27de7fba0 2024-03-26T19:12:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.881+0000 7f9f1b400640 10 client.26212 handle_command_reply: tid=0 2024-03-26T19:12:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.881+0000 7f9f3f604740 1 client.26212 shutdown 2024-03-26T19:12:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 1 -- 172.21.15.97:0/1430086757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6f900 msgr2=0x55b27de7b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6f900 0x55b27de7b310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9f30005b70 tx=0x7f9f30004a70 comp rx=0 tx=0).stop 2024-03-26T19:12:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 1 -- 172.21.15.97:0/1430086757 shutdown_connections 2024-03-26T19:12:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b27de7fba0 0x55b27de81f90 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6f900 0x55b27de7b310 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6a790 0x55b27de74340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 1 --2- 172.21.15.97:0/1430086757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de73e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 1 -- 172.21.15.97:0/1430086757 >> 172.21.15.97:0/1430086757 conn(0x55b27de65310 msgr2=0x55b27de64540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f1b400640 0 client.26212 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 1 -- 172.21.15.97:0/1430086757 shutdown_connections 2024-03-26T19:12:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 1 -- 172.21.15.97:0/1430086757 wait complete. 2024-03-26T19:12:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.882+0000 7f9f3f604740 20 client.26212 trim_cache size 0 max 16384 2024-03-26T19:12:02.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:02 smithi144 ceph-mon[36053]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 644 op/s 2024-03-26T19:12:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.913+0000 7f9f3f604740 1 Processor -- start 2024-03-26T19:12:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.913+0000 7f9f3f604740 1 -- start start 2024-03-26T19:12:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.913+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de69da0 0x55b27de6edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.913+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de7b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.913+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de6f900 0x55b27de63cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.913+0000 7f9f3f604740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381af2f0 con 0x55b27de69da0 2024-03-26T19:12:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.913+0000 7f9f3f604740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381b0e30 con 0x55b27de6f900 2024-03-26T19:12:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.913+0000 7f9f3f604740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381af9c0 con 0x55b27de6a790 2024-03-26T19:12:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.914+0000 7f9f35e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de69da0 0x55b27de6edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.914+0000 7f9f2fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de7b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.915+0000 7f9f36800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de6f900 0x55b27de63cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.915+0000 7f9f2fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de7b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:46990/0 (socket says 172.21.15.97:46990) 2024-03-26T19:12:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.915+0000 7f9f36800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de6f900 0x55b27de63cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:42198/0 (socket says 172.21.15.97:42198) 2024-03-26T19:12:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.915+0000 7f9f2fe00640 1 -- 172.21.15.97:0/502042967 learned_addr learned my addr 172.21.15.97:0/502042967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.916+0000 7f9f36800640 1 -- 172.21.15.97:0/502042967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 msgr2=0x55b27de7b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.916+0000 7f9f36800640 1 --2- 172.21.15.97:0/502042967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de7b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.916+0000 7f9f36800640 1 -- 172.21.15.97:0/502042967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de69da0 msgr2=0x55b27de6edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.916+0000 7f9f36800640 1 --2- 172.21.15.97:0/502042967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de69da0 0x55b27de6edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.916+0000 7f9f36800640 1 -- 172.21.15.97:0/502042967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b27de66080 con 0x55b27de6f900 2024-03-26T19:12:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.916+0000 7f9f36800640 1 --2- 172.21.15.97:0/502042967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de6f900 0x55b27de63cb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9f30012f10 tx=0x7f9f30011f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3a9af0b23b903a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.917+0000 7f9f26600640 1 -- 172.21.15.97:0/502042967 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f300099a0 con 0x55b27de6f900 2024-03-26T19:12:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.917+0000 7f9f26600640 1 -- 172.21.15.97:0/502042967 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3001c7e0 con 0x55b27de6f900 2024-03-26T19:12:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.917+0000 7f9f26600640 1 -- 172.21.15.97:0/502042967 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30033970 con 0x55b27de6f900 2024-03-26T19:12:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.918+0000 7f9f3f604740 1 -- 172.21.15.97:0/502042967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de6f900 msgr2=0x55b27de63cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.918+0000 7f9f3f604740 1 --2- 172.21.15.97:0/502042967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de6f900 0x55b27de63cb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9f30012f10 tx=0x7f9f30011f40 comp rx=0 tx=0).stop 2024-03-26T19:12:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.919+0000 7f9f3f604740 1 -- 172.21.15.97:0/502042967 shutdown_connections 2024-03-26T19:12:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.919+0000 7f9f3f604740 1 --2- 172.21.15.97:0/502042967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de6f900 0x55b27de63cb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.919+0000 7f9f3f604740 1 --2- 172.21.15.97:0/502042967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de7b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.919+0000 7f9f3f604740 1 --2- 172.21.15.97:0/502042967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de69da0 0x55b27de6edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.919+0000 7f9f3f604740 1 -- 172.21.15.97:0/502042967 >> 172.21.15.97:0/502042967 conn(0x55b27de65310 msgr2=0x55b27de68e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.919+0000 7f9f3f604740 1 -- 172.21.15.97:0/502042967 shutdown_connections 2024-03-26T19:12:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.919+0000 7f9f3f604740 1 -- 172.21.15.97:0/502042967 wait complete. 2024-03-26T19:12:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.920+0000 7f9f3f604740 1 -- start start 2024-03-26T19:12:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.920+0000 7f9f3f604740 1 Processor -- start 2024-03-26T19:12:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.921+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de78c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.921+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de79170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.921+0000 7f9f3f604740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 0x55b27de7a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.922+0000 7f9f2fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de79170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.922+0000 7f9f2fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de79170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:47002/0 (socket says 172.21.15.97:47002) 2024-03-26T19:12:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.922+0000 7f9f36800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 0x55b27de7a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.923+0000 7f9f35e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de78c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.923+0000 7f9f35e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de78c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:42212/0 (socket says 172.21.15.97:42212) 2024-03-26T19:12:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.924+0000 7f9f36800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 0x55b27de7a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50978/0 (socket says 172.21.15.97:50978) 2024-03-26T19:12:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.925+0000 7f9f2fe00640 1 -- 172.21.15.97:0/3224888227 learned_addr learned my addr 172.21.15.97:0/3224888227 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.925+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381b0760 con 0x55b27de6f900 2024-03-26T19:12:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.925+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381b0090 con 0x55b27de69da0 2024-03-26T19:12:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.925+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381ad2d0 con 0x55b27de6a790 2024-03-26T19:12:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.925+0000 7f9f3f604740 10 client.0 _ll_register_callbacks cb 0x7f9f30014530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:12:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.926+0000 7f9f2fe00640 1 -- 172.21.15.97:0/3224888227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 msgr2=0x55b27de78c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.926+0000 7f9f2fe00640 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de78c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.926+0000 7f9f2fe00640 1 -- 172.21.15.97:0/3224888227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 msgr2=0x55b27de7a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.926+0000 7f9f2fe00640 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 0x55b27de7a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.926+0000 7f9f2fe00640 1 -- 172.21.15.97:0/3224888227 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b27de7a700 con 0x55b27de6a790 2024-03-26T19:12:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.928+0000 7f9f35e00640 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de78c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:12:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.928+0000 7f9f2fe00640 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de79170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9f2002c7b0 tx=0x7f9f200036d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b7dcd3c7263f8e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.928+0000 7f9f1b400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:12:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.928+0000 7f9f1b400640 1 -- 172.21.15.97:0/3224888227 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f200d5020 con 0x55b27de6a790 2024-03-26T19:12:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.929+0000 7f9f1b400640 1 -- 172.21.15.97:0/3224888227 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f9f200eeaf0 con 0x55b27de6a790 2024-03-26T19:12:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.929+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3001c7e0 con 0x55b27de6a790 2024-03-26T19:12:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.929+0000 7f9f1b400640 1 -- 172.21.15.97:0/3224888227 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f20005690 con 0x55b27de6a790 2024-03-26T19:12:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.929+0000 7f9f3f604740 10 client.26218 fetch_fsmap learned FSMap version 27 2024-03-26T19:12:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.929+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b27dee6130 con 0x55b27de6a790 2024-03-26T19:12:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.930+0000 7f9f1b400640 1 -- 172.21.15.97:0/3224888227 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3001c7e0 con 0x55b27de6a790 2024-03-26T19:12:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.930+0000 7f9f1b400640 1 -- 172.21.15.97:0/3224888227 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9f200bd870 con 0x55b27de6a790 2024-03-26T19:12:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.930+0000 7f9f3f604740 10 client.26218 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:12:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.930+0000 7f9f3f604740 10 client.26218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:12:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.930+0000 7f9f3f604740 20 client.26218 populate_metadata read hostname 'smithi097' 2024-03-26T19:12:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.930+0000 7f9f3f604740 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b27dee6b30 0x55b27dee8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.930+0000 7f9f3f604740 4 client.26218 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:12:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.930+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b27ddd6710 con 0x55b27dee6b30 2024-03-26T19:12:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.932+0000 7f9f35e00640 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b27dee6b30 0x55b27dee8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.933+0000 7f9f35e00640 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b27dee6b30 0x55b27dee8f50 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e13331044affdd6 server_cookie=c9172654f951f254 in_seq=0 out_seq=0 2024-03-26T19:12:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:02.933+0000 7f9f1b400640 10 client.26218 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:04.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:04 smithi105 ceph-mon[36270]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 548 op/s 2024-03-26T19:12:04.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:04 smithi097 ceph-mon[31955]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 548 op/s 2024-03-26T19:12:04.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:04 smithi144 ceph-mon[36053]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 548 op/s 2024-03-26T19:12:05.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 760.0 s, 1098.0 tps, lat 89.748 ms stddev 109.084 2024-03-26T19:12:06.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:06 smithi105 ceph-mon[36270]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 658 op/s 2024-03-26T19:12:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:06 smithi097 ceph-mon[31955]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 658 op/s 2024-03-26T19:12:06.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:06 smithi144 ceph-mon[36053]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 658 op/s 2024-03-26T19:12:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.416+0000 7f9f1b400640 1 -- 172.21.15.97:0/3224888227 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b27ddd6710 con 0x55b27dee6b30 2024-03-26T19:12:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.416+0000 7f9f1b400640 10 client.26218 handle_command_reply: tid=0 2024-03-26T19:12:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.417+0000 7f9f3f604740 1 client.26218 shutdown 2024-03-26T19:12:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 msgr2=0x55b27de79170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de79170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9f2002c7b0 tx=0x7f9f200036d0 comp rx=0 tx=0).stop 2024-03-26T19:12:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 shutdown_connections 2024-03-26T19:12:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55b27dee6b30 0x55b27dee8f50 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b27de6f900 0x55b27de7a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f1b400640 0 client.26218 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b27de6a790 0x55b27de79170 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 1 --2- 172.21.15.97:0/3224888227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b27de69da0 0x55b27de78c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 >> 172.21.15.97:0/3224888227 conn(0x55b27de65310 msgr2=0x55b27de68e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 shutdown_connections 2024-03-26T19:12:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 1 -- 172.21.15.97:0/3224888227 wait complete. 2024-03-26T19:12:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.418+0000 7f9f3f604740 20 client.26218 trim_cache size 0 max 16384 2024-03-26T19:12:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:12:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:12:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:12:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:12:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.419+0000 7f9f3ca00640 1 -- 172.21.15.97:0/3128961743 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a3040 msgr2=0x7f9f1c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.419+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a3040 0x7f9f1c0a5500 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9f30023000 tx=0x7f9f30020000 comp rx=0 tx=0).stop 2024-03-26T19:12:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.419+0000 7f9f3ca00640 1 -- 172.21.15.97:0/3128961743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f38109070 msgr2=0x7f9f3819e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.419+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f38109070 0x7f9f3819e690 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009230 tx=0x7f9f2002c6c0 comp rx=0 tx=0).stop 2024-03-26T19:12:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.419+0000 7f9f3ca00640 1 -- 172.21.15.97:0/3128961743 shutdown_connections 2024-03-26T19:12:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.419+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f9f1c0a3040 0x7f9f1c0a5500 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.419+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f38109a30 0x7f9f381a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.419+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f38109070 0x7f9f3819e690 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.419+0000 7f9f3ca00640 1 --2- 172.21.15.97:0/3128961743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f38107f20 0x7f9f3819e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.419+0000 7f9f3ca00640 1 -- 172.21.15.97:0/3128961743 >> 172.21.15.97:0/3128961743 conn(0x7f9f38076ed0 msgr2=0x7f9f3810c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.420+0000 7f9f3ca00640 1 -- 172.21.15.97:0/3128961743 shutdown_connections 2024-03-26T19:12:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:07.420+0000 7f9f3ca00640 1 -- 172.21.15.97:0/3128961743 wait complete. 2024-03-26T19:12:08.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:08 smithi105 ceph-mon[36270]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-03-26T19:12:08.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:08 smithi097 ceph-mon[31955]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-03-26T19:12:08.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:08 smithi144 ceph-mon[36053]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-03-26T19:12:10.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 765.0 s, 1067.2 tps, lat 94.472 ms stddev 96.933 2024-03-26T19:12:10.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:10 smithi105 ceph-mon[36270]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-03-26T19:12:10.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:10 smithi097 ceph-mon[31955]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-03-26T19:12:10.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:10 smithi144 ceph-mon[36053]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-03-26T19:12:12.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:12 smithi105 ceph-mon[36270]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 650 op/s 2024-03-26T19:12:12.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:12 smithi097 ceph-mon[31955]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 650 op/s 2024-03-26T19:12:12.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:12 smithi144 ceph-mon[36053]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 650 op/s 2024-03-26T19:12:14.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:14 smithi097 ceph-mon[31955]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 MiB/s wr, 528 op/s 2024-03-26T19:12:14.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:14 smithi144 ceph-mon[36053]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 MiB/s wr, 528 op/s 2024-03-26T19:12:14.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:14 smithi105 ceph-mon[36270]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 MiB/s wr, 528 op/s 2024-03-26T19:12:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 770.0 s, 1370.4 tps, lat 72.882 ms stddev 97.266 2024-03-26T19:12:16.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:16 smithi097 ceph-mon[31955]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-03-26T19:12:16.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:16 smithi144 ceph-mon[36053]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-03-26T19:12:16.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:16 smithi105 ceph-mon[36270]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-03-26T19:12:18.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:18 smithi097 ceph-mon[31955]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.5 MiB/s wr, 596 op/s 2024-03-26T19:12:18.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:18 smithi144 ceph-mon[36053]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.5 MiB/s wr, 596 op/s 2024-03-26T19:12:18.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:18 smithi105 ceph-mon[36270]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.5 MiB/s wr, 596 op/s 2024-03-26T19:12:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 775.0 s, 1685.2 tps, lat 60.131 ms stddev 64.845 2024-03-26T19:12:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.619+0000 7f3d51c00640 1 -- 172.21.15.97:0/2289649707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c1056c0 msgr2=0x7f3d4c10bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.619+0000 7f3d51c00640 1 --2- 172.21.15.97:0/2289649707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c1056c0 0x7f3d4c10bf50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009ca0 tx=0x7f3d4402c100 comp rx=0 tx=0).stop 2024-03-26T19:12:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.619+0000 7f3d51c00640 1 -- 172.21.15.97:0/2289649707 shutdown_connections 2024-03-26T19:12:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.619+0000 7f3d51c00640 1 --2- 172.21.15.97:0/2289649707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c1056c0 0x7f3d4c10bf50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.619+0000 7f3d51c00640 1 --2- 172.21.15.97:0/2289649707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d4c104d00 0x7f3d4c105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.619+0000 7f3d51c00640 1 --2- 172.21.15.97:0/2289649707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c103b10 0x7f3d4c103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.619+0000 7f3d51c00640 1 -- 172.21.15.97:0/2289649707 >> 172.21.15.97:0/2289649707 conn(0x7f3d4c0ff080 msgr2=0x7f3d4c1014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.620+0000 7f3d51c00640 1 -- 172.21.15.97:0/2289649707 shutdown_connections 2024-03-26T19:12:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.620+0000 7f3d51c00640 1 -- 172.21.15.97:0/2289649707 wait complete. 2024-03-26T19:12:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.622+0000 7f3d51c00640 1 Processor -- start 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.622+0000 7f3d51c00640 1 -- start start 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.623+0000 7f3d51c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c103b10 0x7f3d4c19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.623+0000 7f3d51c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d4c104d00 0x7f3d4c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.623+0000 7f3d51c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c1056c0 0x7f3d4c1a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.623+0000 7f3d51c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c10f7f0 con 0x7f3d4c104d00 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.623+0000 7f3d51c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c10f670 con 0x7f3d4c1056c0 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.623+0000 7f3d51c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c10f970 con 0x7f3d4c103b10 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.623+0000 7f3d4b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c1056c0 0x7f3d4c1a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.623+0000 7f3d4aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c103b10 0x7f3d4c19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.623+0000 7f3d4aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c103b10 0x7f3d4c19e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:42710/0 (socket says 172.21.15.97:42710) 2024-03-26T19:12:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.623+0000 7f3d4aa00640 1 -- 172.21.15.97:0/654300601 learned_addr learned my addr 172.21.15.97:0/654300601 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.624+0000 7f3d4aa00640 1 -- 172.21.15.97:0/654300601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c1056c0 msgr2=0x7f3d4c1a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.624+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c1056c0 0x7f3d4c1a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.624+0000 7f3d4aa00640 1 -- 172.21.15.97:0/654300601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d4c104d00 msgr2=0x7f3d4c19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.624+0000 7f3d4a000640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d4c104d00 0x7f3d4c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.624+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d4c104d00 0x7f3d4c19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.624+0000 7f3d4aa00640 1 -- 172.21.15.97:0/654300601 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4c1a5da0 con 0x7f3d4c103b10 2024-03-26T19:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.624+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c103b10 0x7f3d4c19e180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00d770 tx=0x7f3d3c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.626+0000 7f3d43400640 1 -- 172.21.15.97:0/654300601 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c017070 con 0x7f3d4c103b10 2024-03-26T19:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.626+0000 7f3d43400640 1 -- 172.21.15.97:0/654300601 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d3c009c30 con 0x7f3d4c103b10 2024-03-26T19:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.626+0000 7f3d43400640 1 -- 172.21.15.97:0/654300601 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c003dd0 con 0x7f3d4c103b10 2024-03-26T19:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.626+0000 7f3d51c00640 1 -- 172.21.15.97:0/654300601 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4c1a6090 con 0x7f3d4c103b10 2024-03-26T19:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.626+0000 7f3d51c00640 1 -- 172.21.15.97:0/654300601 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d4c1a6610 con 0x7f3d4c103b10 2024-03-26T19:12:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.632+0000 7f3d43400640 1 -- 172.21.15.97:0/654300601 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f3d3c003f70 con 0x7f3d4c103b10 2024-03-26T19:12:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.632+0000 7f3d54a0b740 1 Processor -- start 2024-03-26T19:12:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.633+0000 7f3d54a0b740 1 -- start start 2024-03-26T19:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.636+0000 7f3d43400640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d300a3130 0x7f3d300a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.636+0000 7f3d43400640 1 -- 172.21.15.97:0/654300601 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3d3c0cec80 con 0x7f3d4c103b10 2024-03-26T19:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.636+0000 7f3d4b400640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d300a3130 0x7f3d300a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.638+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fe4da0 0x55e573fe51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.638+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 0x55e573fea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.638+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fea900 0x55e573fecd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.638+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e573f49e80 con 0x55e573fe4da0 2024-03-26T19:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.638+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e573ef2660 con 0x55e573fea900 2024-03-26T19:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.638+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e573f45e30 con 0x55e573fe5790 2024-03-26T19:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.638+0000 7f3d4b400640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d300a3130 0x7f3d300a55f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3d4402c640 tx=0x7f3d44032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.639+0000 7f3d4a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 0x55e573fea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.639+0000 7f3d4b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fea900 0x55e573fecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.639+0000 7f3d4b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fea900 0x55e573fecd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54714/0 (socket says 172.21.15.97:54714) 2024-03-26T19:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.639+0000 7f3d4b400640 1 -- 172.21.15.97:0/1834882271 learned_addr learned my addr 172.21.15.97:0/1834882271 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.639+0000 7f3d4a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 0x55e573fea290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:42722/0 (socket says 172.21.15.97:42722) 2024-03-26T19:12:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.640+0000 7f3d4a000640 1 -- 172.21.15.97:0/1834882271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fea900 msgr2=0x55e573fecd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.640+0000 7f3d4a000640 1 --2- 172.21.15.97:0/1834882271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fea900 0x55e573fecd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.640+0000 7f3d4a000640 1 -- 172.21.15.97:0/1834882271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fe4da0 msgr2=0x55e573fe51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.640+0000 7f3d4a000640 1 --2- 172.21.15.97:0/1834882271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fe4da0 0x55e573fe51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.640+0000 7f3d4a000640 1 -- 172.21.15.97:0/1834882271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e573c5e210 con 0x55e573fe5790 2024-03-26T19:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.640+0000 7f3d4a000640 1 --2- 172.21.15.97:0/1834882271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 0x55e573fea290 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c19f690 tx=0x7f3d3800acc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6367e75ee8d252 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.641+0000 7f3d37e00640 1 -- 172.21.15.97:0/1834882271 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38015070 con 0x55e573fe5790 2024-03-26T19:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.641+0000 7f3d37e00640 1 -- 172.21.15.97:0/1834882271 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d3800cb40 con 0x55e573fe5790 2024-03-26T19:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.641+0000 7f3d37e00640 1 -- 172.21.15.97:0/1834882271 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d380185c0 con 0x55e573fe5790 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.642+0000 7f3d54a0b740 1 -- 172.21.15.97:0/1834882271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 msgr2=0x55e573fea290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.642+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/1834882271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 0x55e573fea290 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c19f690 tx=0x7f3d3800acc0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.642+0000 7f3d54a0b740 1 -- 172.21.15.97:0/1834882271 shutdown_connections 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.642+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/1834882271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fea900 0x55e573fecd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.642+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/1834882271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 0x55e573fea290 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.642+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/1834882271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fe4da0 0x55e573fe51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.642+0000 7f3d54a0b740 1 -- 172.21.15.97:0/1834882271 >> 172.21.15.97:0/1834882271 conn(0x55e573fe0310 msgr2=0x55e573fe0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.643+0000 7f3d54a0b740 1 -- 172.21.15.97:0/1834882271 shutdown_connections 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.643+0000 7f3d54a0b740 1 -- 172.21.15.97:0/1834882271 wait complete. 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.643+0000 7f3d54a0b740 1 -- start start 2024-03-26T19:12:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.644+0000 7f3d54a0b740 1 Processor -- start 2024-03-26T19:12:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.644+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe4da0 0x55e573feee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.644+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe5790 0x55e573fef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.644+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 0x55e573ff63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.644+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e573fed8b0 con 0x55e573fea900 2024-03-26T19:12:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.644+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e573f05ac0 con 0x55e573fe5790 2024-03-26T19:12:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.644+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e573feda30 con 0x55e573fe4da0 2024-03-26T19:12:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.644+0000 7f3d54a0b740 10 client.0 _ll_register_callbacks cb 0x55e573f05490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:12:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.645+0000 7f3d4b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 0x55e573ff63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.645+0000 7f3d4a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe5790 0x55e573fef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.645+0000 7f3d4aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe4da0 0x55e573feee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.645+0000 7f3d4b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 0x55e573ff63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49648/0 (socket says 172.21.15.97:49648) 2024-03-26T19:12:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.645+0000 7f3d4b400640 1 -- 172.21.15.97:0/27608160 learned_addr learned my addr 172.21.15.97:0/27608160 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.645+0000 7f3d4a000640 1 -- 172.21.15.97:0/27608160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe4da0 msgr2=0x55e573feee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.645+0000 7f3d4a000640 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe4da0 0x55e573feee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.645+0000 7f3d4a000640 1 -- 172.21.15.97:0/27608160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 msgr2=0x55e573ff63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.645+0000 7f3d4a000640 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 0x55e573ff63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.645+0000 7f3d4a000640 1 -- 172.21.15.97:0/27608160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e573d2a620 con 0x55e573fe5790 2024-03-26T19:12:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.646+0000 7f3d4a000640 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe5790 0x55e573fef3a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55e573fe1280 tx=0x7f3d38011960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde8bfef1f822bc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.646+0000 7f3d34c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:12:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.647+0000 7f3d34c00640 1 -- 172.21.15.97:0/27608160 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38015040 con 0x55e573fe5790 2024-03-26T19:12:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.647+0000 7f3d34c00640 1 -- 172.21.15.97:0/27608160 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3d3801a450 con 0x55e573fe5790 2024-03-26T19:12:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.647+0000 7f3d54a0b740 1 -- 172.21.15.97:0/27608160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d3800cb40 con 0x55e573fe5790 2024-03-26T19:12:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.649+0000 7f3d34c00640 1 -- 172.21.15.97:0/27608160 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d3801a670 con 0x55e573fe5790 2024-03-26T19:12:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.649+0000 7f3d54a0b740 10 client.26243 fetch_fsmap learned FSMap version 27 2024-03-26T19:12:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.649+0000 7f3d54a0b740 1 -- 172.21.15.97:0/27608160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e573ff9750 con 0x55e573fe5790 2024-03-26T19:12:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.649+0000 7f3d34c00640 1 -- 172.21.15.97:0/27608160 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d3800cb40 con 0x55e573fe5790 2024-03-26T19:12:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.649+0000 7f3d34c00640 1 -- 172.21.15.97:0/27608160 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d3802d790 con 0x55e573fe5790 2024-03-26T19:12:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.650+0000 7f3d54a0b740 10 client.26243 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:12:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.650+0000 7f3d54a0b740 10 client.26243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:12:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.650+0000 7f3d54a0b740 20 client.26243 populate_metadata read hostname 'smithi097' 2024-03-26T19:12:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.650+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55e573ffaaa0 0x55e573ffce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.650+0000 7f3d54a0b740 4 client.26243 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:12:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.650+0000 7f3d54a0b740 1 -- 172.21.15.97:0/27608160 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e573f573c0 con 0x55e573ffaaa0 2024-03-26T19:12:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.650+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55e573ffaaa0 0x55e573ffce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.651+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55e573ffaaa0 0x55e573ffce90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8720673aa4603eb server_cookie=44abe5b1503b4533 in_seq=0 out_seq=0 2024-03-26T19:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.651+0000 7f3d34c00640 10 client.26243 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.651+0000 7f3d34c00640 1 -- 172.21.15.97:0/27608160 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e573f573c0 con 0x55e573ffaaa0 2024-03-26T19:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.651+0000 7f3d34c00640 10 client.26243 handle_command_reply: tid=0 2024-03-26T19:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.652+0000 7f3d54a0b740 1 client.26243 shutdown 2024-03-26T19:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.652+0000 7f3d54a0b740 1 -- 172.21.15.97:0/27608160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe5790 msgr2=0x55e573fef3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.652+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe5790 0x55e573fef3a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55e573fe1280 tx=0x7f3d38011960 comp rx=0 tx=0).stop 2024-03-26T19:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.652+0000 7f3d54a0b740 1 -- 172.21.15.97:0/27608160 shutdown_connections 2024-03-26T19:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.652+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55e573ffaaa0 0x55e573ffce90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.653+0000 7f3d34c00640 0 client.26243 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.653+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 0x55e573ff63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.653+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe5790 0x55e573fef3a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.653+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/27608160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe4da0 0x55e573feee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.653+0000 7f3d54a0b740 1 -- 172.21.15.97:0/27608160 >> 172.21.15.97:0/27608160 conn(0x55e573fe0310 msgr2=0x55e573fdf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.656+0000 7f3d54a0b740 1 -- 172.21.15.97:0/27608160 shutdown_connections 2024-03-26T19:12:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.656+0000 7f3d54a0b740 1 -- 172.21.15.97:0/27608160 wait complete. 2024-03-26T19:12:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.656+0000 7f3d54a0b740 20 client.26243 trim_cache size 0 max 16384 2024-03-26T19:12:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d54a0b740 1 Processor -- start 2024-03-26T19:12:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d54a0b740 1 -- start start 2024-03-26T19:12:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573fe3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 0x55e573ff63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 0x55e573fdecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c1af320 con 0x55e573fea900 2024-03-26T19:12:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c1b0e60 con 0x55e573fe4da0 2024-03-26T19:12:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c1af9f0 con 0x55e573fe5790 2024-03-26T19:12:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d4aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573fe3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d4aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573fe3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:54724/0 (socket says 172.21.15.97:54724) 2024-03-26T19:12:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d4aa00640 1 -- 172.21.15.97:0/4029552720 learned_addr learned my addr 172.21.15.97:0/4029552720 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.681+0000 7f3d4a000640 1 --2- 172.21.15.97:0/4029552720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 0x55e573ff63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.682+0000 7f3d4b400640 1 --2- 172.21.15.97:0/4029552720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 0x55e573fdecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.682+0000 7f3d4aa00640 1 -- 172.21.15.97:0/4029552720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 msgr2=0x55e573ff63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.682+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/4029552720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 0x55e573ff63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.682+0000 7f3d4aa00640 1 -- 172.21.15.97:0/4029552720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 msgr2=0x55e573fdecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.682+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/4029552720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 0x55e573fdecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.682+0000 7f3d4aa00640 1 -- 172.21.15.97:0/4029552720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e573ffd430 con 0x55e573fe4da0 2024-03-26T19:12:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.682+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/4029552720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573fe3c80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0cc7b0 tx=0x7f3d3c004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59eb49cc64c85d72 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.682+0000 7f3d37400640 1 -- 172.21.15.97:0/4029552720 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0af070 con 0x55e573fe4da0 2024-03-26T19:12:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.682+0000 7f3d37400640 1 -- 172.21.15.97:0/4029552720 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d3c09b3e0 con 0x55e573fe4da0 2024-03-26T19:12:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.682+0000 7f3d37400640 1 -- 172.21.15.97:0/4029552720 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0b4070 con 0x55e573fe4da0 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.683+0000 7f3d54a0b740 1 -- 172.21.15.97:0/4029552720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 msgr2=0x55e573fe3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.683+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/4029552720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573fe3c80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0cc7b0 tx=0x7f3d3c004590 comp rx=0 tx=0).stop 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.683+0000 7f3d54a0b740 1 -- 172.21.15.97:0/4029552720 shutdown_connections 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.683+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/4029552720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fea900 0x55e573fdecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.683+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/4029552720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fe5790 0x55e573ff63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.683+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/4029552720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573fe3c80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.683+0000 7f3d54a0b740 1 -- 172.21.15.97:0/4029552720 >> 172.21.15.97:0/4029552720 conn(0x55e573fe0310 msgr2=0x55e573fe3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.683+0000 7f3d54a0b740 1 -- 172.21.15.97:0/4029552720 shutdown_connections 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.683+0000 7f3d54a0b740 1 -- 172.21.15.97:0/4029552720 wait complete. 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.684+0000 7f3d54a0b740 1 -- start start 2024-03-26T19:12:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.684+0000 7f3d54a0b740 1 Processor -- start 2024-03-26T19:12:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573ff1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fe5790 0x55e573ff1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d54a0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fea900 0x55e573ff4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c1b0790 con 0x55e573fe5790 2024-03-26T19:12:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c1b00c0 con 0x55e573fe4da0 2024-03-26T19:12:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d54a0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c1ad300 con 0x55e573fea900 2024-03-26T19:12:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d54a0b740 10 client.0 _ll_register_callbacks cb 0x7f3d38011820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:12:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d4b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fea900 0x55e573ff4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d4b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fea900 0x55e573ff4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:42740/0 (socket says 172.21.15.97:42740) 2024-03-26T19:12:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d4b400640 1 -- 172.21.15.97:0/65834443 learned_addr learned my addr 172.21.15.97:0/65834443 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.685+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573ff1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.686+0000 7f3d4a000640 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fe5790 0x55e573ff1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.686+0000 7f3d4aa00640 1 -- 172.21.15.97:0/65834443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fea900 msgr2=0x55e573ff4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.686+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fea900 0x55e573ff4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.686+0000 7f3d4aa00640 1 -- 172.21.15.97:0/65834443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fe5790 msgr2=0x55e573ff1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.686+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fe5790 0x55e573ff1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.686+0000 7f3d4aa00640 1 -- 172.21.15.97:0/65834443 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e573ff4f50 con 0x55e573fe4da0 2024-03-26T19:12:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.686+0000 7f3d4a000640 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fe5790 0x55e573ff1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:12:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.686+0000 7f3d4aa00640 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573ff1400 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c09f6f0 tx=0x7f3d3c009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=132cb31d6a953b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.686+0000 7f3d34c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:12:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.687+0000 7f3d34c00640 1 -- 172.21.15.97:0/65834443 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0b8020 con 0x55e573fe4da0 2024-03-26T19:12:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.687+0000 7f3d34c00640 1 -- 172.21.15.97:0/65834443 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3d3c097a50 con 0x55e573fe4da0 2024-03-26T19:12:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.687+0000 7f3d54a0b740 1 -- 172.21.15.97:0/65834443 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d3c09b3e0 con 0x55e573fe4da0 2024-03-26T19:12:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.687+0000 7f3d34c00640 1 -- 172.21.15.97:0/65834443 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d3c0af070 con 0x55e573fe4da0 2024-03-26T19:12:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.687+0000 7f3d54a0b740 10 client.26249 fetch_fsmap learned FSMap version 27 2024-03-26T19:12:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.687+0000 7f3d54a0b740 1 -- 172.21.15.97:0/65834443 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e573ff6260 con 0x55e573fe4da0 2024-03-26T19:12:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.687+0000 7f3d34c00640 1 -- 172.21.15.97:0/65834443 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d3c09b3e0 con 0x55e573fe4da0 2024-03-26T19:12:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.687+0000 7f3d34c00640 1 -- 172.21.15.97:0/65834443 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d3c0b41e0 con 0x55e573fe4da0 2024-03-26T19:12:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.688+0000 7f3d54a0b740 10 client.26249 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:12:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.688+0000 7f3d54a0b740 10 client.26249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:12:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.688+0000 7f3d54a0b740 20 client.26249 populate_metadata read hostname 'smithi097' 2024-03-26T19:12:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.688+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55e573ffaaa0 0x55e574073f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.688+0000 7f3d54a0b740 4 client.26249 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:12:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.688+0000 7f3d54a0b740 1 -- 172.21.15.97:0/65834443 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e573f51710 con 0x55e573ffaaa0 2024-03-26T19:12:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.688+0000 7f3d4a000640 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55e573ffaaa0 0x55e574073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.689+0000 7f3d4a000640 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55e573ffaaa0 0x55e574073f30 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee19af71783791ad server_cookie=7ae26f7be4827423 in_seq=0 out_seq=0 2024-03-26T19:12:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.689+0000 7f3d34c00640 10 client.26249 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d34c00640 1 -- 172.21.15.97:0/65834443 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e573f51710 con 0x55e573ffaaa0 2024-03-26T19:12:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d34c00640 10 client.26249 handle_command_reply: tid=0 2024-03-26T19:12:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d54a0b740 1 client.26249 shutdown 2024-03-26T19:12:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d54a0b740 1 -- 172.21.15.97:0/65834443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 msgr2=0x55e573ff1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573ff1400 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c09f6f0 tx=0x7f3d3c009f20 comp rx=0 tx=0).stop 2024-03-26T19:12:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d54a0b740 1 -- 172.21.15.97:0/65834443 shutdown_connections 2024-03-26T19:12:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55e573ffaaa0 0x55e574073f30 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e573fea900 0x55e573ff4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e573fe5790 0x55e573ff1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d54a0b740 1 --2- 172.21.15.97:0/65834443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e573fe4da0 0x55e573ff1400 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d54a0b740 1 -- 172.21.15.97:0/65834443 >> 172.21.15.97:0/65834443 conn(0x55e573fe0310 msgr2=0x55e573fe3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d34c00640 0 client.26249 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.725+0000 7f3d54a0b740 1 -- 172.21.15.97:0/65834443 shutdown_connections 2024-03-26T19:12:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d54a0b740 1 -- 172.21.15.97:0/65834443 wait complete. 2024-03-26T19:12:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d54a0b740 20 client.26249 trim_cache size 0 max 16384 2024-03-26T19:12:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:12:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:12:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:12:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:12:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d51c00640 1 -- 172.21.15.97:0/654300601 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d300a3130 msgr2=0x7f3d300a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d51c00640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d300a3130 0x7f3d300a55f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3d4402c640 tx=0x7f3d44032590 comp rx=0 tx=0).stop 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d51c00640 1 -- 172.21.15.97:0/654300601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c103b10 msgr2=0x7f3d4c19e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d51c00640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c103b10 0x7f3d4c19e180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00d770 tx=0x7f3d3c00dc30 comp rx=0 tx=0).stop 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d51c00640 1 -- 172.21.15.97:0/654300601 shutdown_connections 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d51c00640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3d300a3130 0x7f3d300a55f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d51c00640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c1056c0 0x7f3d4c1a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d51c00640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d4c104d00 0x7f3d4c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.726+0000 7f3d51c00640 1 --2- 172.21.15.97:0/654300601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c103b10 0x7f3d4c19e180 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.727+0000 7f3d51c00640 1 -- 172.21.15.97:0/654300601 >> 172.21.15.97:0/654300601 conn(0x7f3d4c0ff080 msgr2=0x7f3d4c077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.727+0000 7f3d51c00640 1 -- 172.21.15.97:0/654300601 shutdown_connections 2024-03-26T19:12:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:20.727+0000 7f3d51c00640 1 -- 172.21.15.97:0/654300601 wait complete. 2024-03-26T19:12:20.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:20 smithi097 ceph-mon[31955]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 632 op/s 2024-03-26T19:12:20.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:20 smithi144 ceph-mon[36053]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 632 op/s 2024-03-26T19:12:20.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:20 smithi105 ceph-mon[36270]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 632 op/s 2024-03-26T19:12:22.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:22 smithi097 ceph-mon[31955]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 687 op/s 2024-03-26T19:12:22.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:22 smithi144 ceph-mon[36053]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 687 op/s 2024-03-26T19:12:22.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:22 smithi105 ceph-mon[36270]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 687 op/s 2024-03-26T19:12:24.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:24 smithi097 ceph-mon[31955]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.4 MiB/s wr, 582 op/s 2024-03-26T19:12:24.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:24 smithi144 ceph-mon[36053]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.4 MiB/s wr, 582 op/s 2024-03-26T19:12:24.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:24 smithi105 ceph-mon[36270]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.4 MiB/s wr, 582 op/s 2024-03-26T19:12:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 780.0 s, 1207.4 tps, lat 81.457 ms stddev 97.885 2024-03-26T19:12:25.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:25 smithi097 ceph-mon[31955]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 720 op/s 2024-03-26T19:12:25.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:25 smithi144 ceph-mon[36053]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 720 op/s 2024-03-26T19:12:25.970 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:25 smithi105 ceph-mon[36270]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 720 op/s 2024-03-26T19:12:28.669 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:28 smithi105 ceph-mon[36270]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 601 op/s 2024-03-26T19:12:28.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:28 smithi097 ceph-mon[31955]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 601 op/s 2024-03-26T19:12:28.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:28 smithi144 ceph-mon[36053]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 601 op/s 2024-03-26T19:12:30.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 785.0 s, 945.8 tps, lat 104.949 ms stddev 129.901 2024-03-26T19:12:30.718 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:30 smithi105 ceph-mon[36270]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.2 MiB/s wr, 594 op/s 2024-03-26T19:12:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:30 smithi097 ceph-mon[31955]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.2 MiB/s wr, 594 op/s 2024-03-26T19:12:30.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:30 smithi144 ceph-mon[36053]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.2 MiB/s wr, 594 op/s 2024-03-26T19:12:32.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:32 smithi105 ceph-mon[36270]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 671 op/s 2024-03-26T19:12:32.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:32 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:12:32.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:32 smithi097 ceph-mon[31955]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 671 op/s 2024-03-26T19:12:32.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:32 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:12:32.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:32 smithi144 ceph-mon[36053]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 671 op/s 2024-03-26T19:12:32.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:32 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.939+0000 7f34a6400640 1 -- 172.21.15.97:0/4236303425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34a0075900 msgr2=0x7f34a0073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.939+0000 7f34a6400640 1 --2- 172.21.15.97:0/4236303425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34a0075900 0x7f34a0073d50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3490009900 tx=0x7f349002bc30 comp rx=0 tx=0).stop 2024-03-26T19:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.939+0000 7f34a6400640 1 -- 172.21.15.97:0/4236303425 shutdown_connections 2024-03-26T19:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.939+0000 7f34a6400640 1 --2- 172.21.15.97:0/4236303425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a010dee0 0x7f34a01102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.939+0000 7f34a6400640 1 --2- 172.21.15.97:0/4236303425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a0074320 0x7f34a00747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.939+0000 7f34a6400640 1 --2- 172.21.15.97:0/4236303425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34a0075900 0x7f34a0073d50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.939+0000 7f34a6400640 1 -- 172.21.15.97:0/4236303425 >> 172.21.15.97:0/4236303425 conn(0x7f34a00fce80 msgr2=0x7f34a00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.940+0000 7f34a6400640 1 -- 172.21.15.97:0/4236303425 shutdown_connections 2024-03-26T19:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.940+0000 7f34a6400640 1 -- 172.21.15.97:0/4236303425 wait complete. 2024-03-26T19:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.941+0000 7f34a6400640 1 Processor -- start 2024-03-26T19:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.942+0000 7f34a6400640 1 -- start start 2024-03-26T19:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.942+0000 7f34a6400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a0074320 0x7f34a019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.942+0000 7f34a6400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a0075900 0x7f34a019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.942+0000 7f34a6400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34a010dee0 0x7f34a01a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.942+0000 7f34a6400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0113bd0 con 0x7f34a0075900 2024-03-26T19:12:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.942+0000 7f34a6400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0113a50 con 0x7f34a0074320 2024-03-26T19:12:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.942+0000 7f34a6400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0113d50 con 0x7f34a010dee0 2024-03-26T19:12:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.943+0000 7f349f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a0074320 0x7f34a019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.943+0000 7f349f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a0074320 0x7f34a019e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:36736/0 (socket says 172.21.15.97:36736) 2024-03-26T19:12:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.943+0000 7f349f400640 1 -- 172.21.15.97:0/2830824127 learned_addr learned my addr 172.21.15.97:0/2830824127 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.943+0000 7f349fe00640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34a010dee0 0x7f34a01a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.943+0000 7f349ea00640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a0075900 0x7f34a019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.944+0000 7f349f400640 1 -- 172.21.15.97:0/2830824127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34a010dee0 msgr2=0x7f34a01a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.944+0000 7f349f400640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34a010dee0 0x7f34a01a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.944+0000 7f349f400640 1 -- 172.21.15.97:0/2830824127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a0075900 msgr2=0x7f34a019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.944+0000 7f349f400640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a0075900 0x7f34a019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.944+0000 7f349f400640 1 -- 172.21.15.97:0/2830824127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a01a5d90 con 0x7f34a0074320 2024-03-26T19:12:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.944+0000 7f349f400640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a0074320 0x7f34a019e1c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3490005ab0 tx=0x7f349002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.945+0000 7f3497e00640 1 -- 172.21.15.97:0/2830824127 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490041070 con 0x7f34a0074320 2024-03-26T19:12:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.945+0000 7f3497e00640 1 -- 172.21.15.97:0/2830824127 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3490004450 con 0x7f34a0074320 2024-03-26T19:12:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.945+0000 7f3497e00640 1 -- 172.21.15.97:0/2830824127 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490003da0 con 0x7f34a0074320 2024-03-26T19:12:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.945+0000 7f34a6400640 1 -- 172.21.15.97:0/2830824127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a01a6020 con 0x7f34a0074320 2024-03-26T19:12:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.946+0000 7f34a6400640 1 -- 172.21.15.97:0/2830824127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01a6490 con 0x7f34a0074320 2024-03-26T19:12:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.950+0000 7f34a9046740 1 Processor -- start 2024-03-26T19:12:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.954+0000 7f3497e00640 1 -- 172.21.15.97:0/2830824127 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f3490044070 con 0x7f34a0074320 2024-03-26T19:12:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.955+0000 7f34a9046740 1 -- start start 2024-03-26T19:12:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.955+0000 7f34a9046740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a867da0 0x55a44a8681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.955+0000 7f34a9046740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a86d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.955+0000 7f34a9046740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a86d900 0x55a44a86fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.955+0000 7f34a9046740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a44a7cce80 con 0x55a44a868790 2024-03-26T19:12:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.955+0000 7f34a9046740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a44a775660 con 0x55a44a867da0 2024-03-26T19:12:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.956+0000 7f34a9046740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a44a7c8e30 con 0x55a44a86d900 2024-03-26T19:12:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.956+0000 7f349ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a86d900 0x55a44a86fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.957+0000 7f349fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a867da0 0x55a44a8681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.957+0000 7f349ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a86d900 0x55a44a86fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60982/0 (socket says 172.21.15.97:60982) 2024-03-26T19:12:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.957+0000 7f349ea00640 1 -- 172.21.15.97:0/3196128115 learned_addr learned my addr 172.21.15.97:0/3196128115 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.957+0000 7f349ea00640 1 -- 172.21.15.97:0/3196128115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a867da0 msgr2=0x55a44a8681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.957+0000 7f349ea00640 1 --2- 172.21.15.97:0/3196128115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a867da0 0x55a44a8681c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.957+0000 7f349ea00640 1 -- 172.21.15.97:0/3196128115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 msgr2=0x55a44a86d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.957+0000 7f349ea00640 1 --2- 172.21.15.97:0/3196128115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a86d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.957+0000 7f349ea00640 1 -- 172.21.15.97:0/3196128115 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a44a4e1210 con 0x55a44a86d900 2024-03-26T19:12:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.958+0000 7f349ea00640 1 --2- 172.21.15.97:0/3196128115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a86d900 0x55a44a86fd90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f348c009c00 tx=0x7f348c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1365b80728931c92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.960+0000 7f3494c00640 1 -- 172.21.15.97:0/3196128115 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c01b480 con 0x55a44a86d900 2024-03-26T19:12:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.960+0000 7f3494c00640 1 -- 172.21.15.97:0/3196128115 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f348c01bb00 con 0x55a44a86d900 2024-03-26T19:12:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.960+0000 7f3494c00640 1 -- 172.21.15.97:0/3196128115 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c012b50 con 0x55a44a86d900 2024-03-26T19:12:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.961+0000 7f34a9046740 1 -- 172.21.15.97:0/3196128115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a86d900 msgr2=0x55a44a86fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.961+0000 7f34a9046740 1 --2- 172.21.15.97:0/3196128115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a86d900 0x55a44a86fd90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f348c009c00 tx=0x7f348c010740 comp rx=0 tx=0).stop 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.962+0000 7f34a9046740 1 -- 172.21.15.97:0/3196128115 shutdown_connections 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.962+0000 7f34a9046740 1 --2- 172.21.15.97:0/3196128115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a86d900 0x55a44a86fd90 secure :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f348c009c00 tx=0x7f348c010740 comp rx=0 tx=0).stop 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.962+0000 7f34a9046740 1 --2- 172.21.15.97:0/3196128115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a86d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.962+0000 7f34a9046740 1 --2- 172.21.15.97:0/3196128115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a867da0 0x55a44a8681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.962+0000 7f34a9046740 1 -- 172.21.15.97:0/3196128115 >> 172.21.15.97:0/3196128115 conn(0x55a44a863310 msgr2=0x55a44a863710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.963+0000 7f3497e00640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f34840a3130 0x7f34840a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.963+0000 7f3497e00640 1 -- 172.21.15.97:0/2830824127 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f34900f0ea0 con 0x7f34a0074320 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.964+0000 7f349ea00640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f34840a3130 0x7f34840a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.966+0000 7f34a9046740 1 -- 172.21.15.97:0/3196128115 shutdown_connections 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.966+0000 7f34a9046740 1 -- 172.21.15.97:0/3196128115 wait complete. 2024-03-26T19:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.966+0000 7f349ea00640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f34840a3130 0x7f34840a55f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f348c012490 tx=0x7f348c010f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.967+0000 7f34a9046740 1 -- start start 2024-03-26T19:12:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.967+0000 7f34a9046740 1 Processor -- start 2024-03-26T19:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.971+0000 7f34a9046740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 0x55a44a882ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.971+0000 7f34a9046740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a883530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.971+0000 7f34a9046740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 0x55a44a88a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.972+0000 7f34a9046740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a44a8818c0 con 0x55a44a868790 2024-03-26T19:12:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.972+0000 7f34a9046740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a44a788ac0 con 0x55a44a888560 2024-03-26T19:12:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.972+0000 7f34a9046740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a44a881a40 con 0x55a44a867da0 2024-03-26T19:12:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.972+0000 7f34a9046740 10 client.0 _ll_register_callbacks cb 0x55a44a788490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.972+0000 7f349fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 0x55a44a88a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.972+0000 7f349fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 0x55a44a88a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:36752/0 (socket says 172.21.15.97:36752) 2024-03-26T19:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.972+0000 7f349fe00640 1 -- 172.21.15.97:0/1668808220 learned_addr learned my addr 172.21.15.97:0/1668808220 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.972+0000 7f349fe00640 1 -- 172.21.15.97:0/1668808220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 msgr2=0x55a44a882ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.972+0000 7f349fe00640 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 0x55a44a882ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.973+0000 7f349fe00640 1 -- 172.21.15.97:0/1668808220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 msgr2=0x55a44a883530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.973+0000 7f349fe00640 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a883530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.973+0000 7f349fe00640 1 -- 172.21.15.97:0/1668808220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44a5ad620 con 0x55a44a888560 2024-03-26T19:12:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.973+0000 7f349fe00640 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 0x55a44a88a950 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3498011d10 tx=0x7f349800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf937761d6e174a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.973+0000 7f3489600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:12:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.973+0000 7f3489600640 1 -- 172.21.15.97:0/1668808220 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349801a3f0 con 0x55a44a888560 2024-03-26T19:12:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.973+0000 7f34a9046740 1 -- 172.21.15.97:0/1668808220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f348c01bb00 con 0x55a44a888560 2024-03-26T19:12:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.975+0000 7f3489600640 1 -- 172.21.15.97:0/1668808220 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3498017050 con 0x55a44a888560 2024-03-26T19:12:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.975+0000 7f3489600640 1 -- 172.21.15.97:0/1668808220 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f349801a590 con 0x55a44a888560 2024-03-26T19:12:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.975+0000 7f34a9046740 10 client.26261 fetch_fsmap learned FSMap version 27 2024-03-26T19:12:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.975+0000 7f34a9046740 1 -- 172.21.15.97:0/1668808220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a44a88ddb0 con 0x55a44a888560 2024-03-26T19:12:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.975+0000 7f3489600640 1 -- 172.21.15.97:0/1668808220 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f348c01bb00 con 0x55a44a888560 2024-03-26T19:12:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.976+0000 7f3489600640 1 -- 172.21.15.97:0/1668808220 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3498019070 con 0x55a44a888560 2024-03-26T19:12:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.976+0000 7f34a9046740 10 client.26261 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:12:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.976+0000 7f34a9046740 10 client.26261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:12:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.976+0000 7f34a9046740 20 client.26261 populate_metadata read hostname 'smithi097' 2024-03-26T19:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.976+0000 7f34a9046740 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a44a88f5d0 0x55a44a891a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.976+0000 7f34a9046740 4 client.26261 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.976+0000 7f34a9046740 1 -- 172.21.15.97:0/1668808220 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a44a7da3c0 con 0x55a44a88f5d0 2024-03-26T19:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.977+0000 7f349f400640 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a44a88f5d0 0x55a44a891a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.980+0000 7f349f400640 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a44a88f5d0 0x55a44a891a90 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9e50c41b17ec113 server_cookie=cd21297fc3d5a041 in_seq=0 out_seq=0 2024-03-26T19:12:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.980+0000 7f3489600640 10 client.26261 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.981+0000 7f3489600640 1 -- 172.21.15.97:0/1668808220 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a44a7da3c0 con 0x55a44a88f5d0 2024-03-26T19:12:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.981+0000 7f3489600640 10 client.26261 handle_command_reply: tid=0 2024-03-26T19:12:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.981+0000 7f34a9046740 1 client.26261 shutdown 2024-03-26T19:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.983+0000 7f34a9046740 1 -- 172.21.15.97:0/1668808220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 msgr2=0x55a44a88a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.983+0000 7f34a9046740 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 0x55a44a88a950 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3498011d10 tx=0x7f349800f790 comp rx=0 tx=0).stop 2024-03-26T19:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.984+0000 7f34a9046740 1 -- 172.21.15.97:0/1668808220 shutdown_connections 2024-03-26T19:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.984+0000 7f34a9046740 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a44a88f5d0 0x55a44a891a90 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.984+0000 7f34a9046740 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 0x55a44a88a950 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.984+0000 7f34a9046740 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a883530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.984+0000 7f34a9046740 1 --2- 172.21.15.97:0/1668808220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 0x55a44a882ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.984+0000 7f34a9046740 1 -- 172.21.15.97:0/1668808220 >> 172.21.15.97:0/1668808220 conn(0x55a44a863310 msgr2=0x55a44a862540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.984+0000 7f3489600640 0 client.26261 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.986+0000 7f34a9046740 1 -- 172.21.15.97:0/1668808220 shutdown_connections 2024-03-26T19:12:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.986+0000 7f34a9046740 1 -- 172.21.15.97:0/1668808220 wait complete. 2024-03-26T19:12:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:33.986+0000 7f34a9046740 20 client.26261 trim_cache size 0 max 16384 2024-03-26T19:12:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.049+0000 7f34a9046740 1 Processor -- start 2024-03-26T19:12:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.050+0000 7f34a9046740 1 -- start start 2024-03-26T19:12:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.050+0000 7f34a9046740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 0x55a44a8829d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.050+0000 7f34a9046740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a866570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.050+0000 7f34a9046740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 0x55a44a866ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f34a9046740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f348000e420 con 0x55a44a868790 2024-03-26T19:12:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f34a9046740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f348000ff60 con 0x55a44a888560 2024-03-26T19:12:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f34a9046740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f348000eaf0 con 0x55a44a867da0 2024-03-26T19:12:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f349f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 0x55a44a8829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f349ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a866570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f349f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 0x55a44a8829d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60990/0 (socket says 172.21.15.97:60990) 2024-03-26T19:12:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f349f400640 1 -- 172.21.15.97:0/378698830 learned_addr learned my addr 172.21.15.97:0/378698830 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f349fe00640 1 --2- 172.21.15.97:0/378698830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 0x55a44a866ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f349f400640 1 -- 172.21.15.97:0/378698830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 msgr2=0x55a44a866ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f349f400640 1 --2- 172.21.15.97:0/378698830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 0x55a44a866ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.051+0000 7f349f400640 1 -- 172.21.15.97:0/378698830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 msgr2=0x55a44a866570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.052+0000 7f349f400640 1 --2- 172.21.15.97:0/378698830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a866570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.052+0000 7f349f400640 1 -- 172.21.15.97:0/378698830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a44a864080 con 0x55a44a867da0 2024-03-26T19:12:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.052+0000 7f349ea00640 1 --2- 172.21.15.97:0/378698830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a866570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:12:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.053+0000 7f349f400640 1 --2- 172.21.15.97:0/378698830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 0x55a44a8829d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f34900b9f70 tx=0x7f34900f0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c5d3529c5293e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.053+0000 7f348be00640 1 -- 172.21.15.97:0/378698830 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349004b040 con 0x55a44a867da0 2024-03-26T19:12:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.054+0000 7f348be00640 1 -- 172.21.15.97:0/378698830 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34900c16f0 con 0x55a44a867da0 2024-03-26T19:12:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.054+0000 7f348be00640 1 -- 172.21.15.97:0/378698830 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900035f0 con 0x55a44a867da0 2024-03-26T19:12:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.055+0000 7f34a9046740 1 -- 172.21.15.97:0/378698830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 msgr2=0x55a44a8829d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.055+0000 7f34a9046740 1 --2- 172.21.15.97:0/378698830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 0x55a44a8829d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f34900b9f70 tx=0x7f34900f0040 comp rx=0 tx=0).stop 2024-03-26T19:12:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.056+0000 7f34a9046740 1 -- 172.21.15.97:0/378698830 shutdown_connections 2024-03-26T19:12:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.056+0000 7f34a9046740 1 --2- 172.21.15.97:0/378698830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a888560 0x55a44a866ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.056+0000 7f34a9046740 1 --2- 172.21.15.97:0/378698830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a868790 0x55a44a866570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.056+0000 7f34a9046740 1 --2- 172.21.15.97:0/378698830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a867da0 0x55a44a8829d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.056+0000 7f34a9046740 1 -- 172.21.15.97:0/378698830 >> 172.21.15.97:0/378698830 conn(0x55a44a863310 msgr2=0x55a44a8822c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.056+0000 7f34a9046740 1 -- 172.21.15.97:0/378698830 shutdown_connections 2024-03-26T19:12:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.056+0000 7f34a9046740 1 -- 172.21.15.97:0/378698830 wait complete. 2024-03-26T19:12:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.056+0000 7f34a9046740 1 -- start start 2024-03-26T19:12:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.057+0000 7f34a9046740 1 Processor -- start 2024-03-26T19:12:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.058+0000 7f34a9046740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a867da0 0x55a44a8a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.058+0000 7f34a9046740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a868790 0x55a44a8a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.058+0000 7f34a9046740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a888560 0x55a44a8a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.058+0000 7f34a9046740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f348000f890 con 0x55a44a867da0 2024-03-26T19:12:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.058+0000 7f34a9046740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f348000f1c0 con 0x55a44a868790 2024-03-26T19:12:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.058+0000 7f34a9046740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f348000c400 con 0x55a44a888560 2024-03-26T19:12:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.058+0000 7f34a9046740 10 client.0 _ll_register_callbacks cb 0x7f3498011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:12:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.059+0000 7f349f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a867da0 0x55a44a8a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.059+0000 7f349fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a888560 0x55a44a8a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.059+0000 7f349fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a888560 0x55a44a8a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:32768/0 (socket says 172.21.15.97:32768) 2024-03-26T19:12:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.059+0000 7f349ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a868790 0x55a44a8a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.059+0000 7f349fe00640 1 -- 172.21.15.97:0/587803408 learned_addr learned my addr 172.21.15.97:0/587803408 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.059+0000 7f349fe00640 1 -- 172.21.15.97:0/587803408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a868790 msgr2=0x55a44a8a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.059+0000 7f349fe00640 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a868790 0x55a44a8a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.059+0000 7f349fe00640 1 -- 172.21.15.97:0/587803408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a867da0 msgr2=0x55a44a8a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.061+0000 7f349fe00640 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a867da0 0x55a44a8a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.061+0000 7f349fe00640 1 -- 172.21.15.97:0/587803408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44a8a2280 con 0x55a44a888560 2024-03-26T19:12:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.061+0000 7f349f400640 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a867da0 0x55a44a8a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:12:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.062+0000 7f349fe00640 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a888560 0x55a44a8a1c90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55a44a867af0 tx=0x7f349801a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3cde5a9b61e868 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.063+0000 7f3489600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:12:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.064+0000 7f3489600640 1 -- 172.21.15.97:0/587803408 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498004d10 con 0x55a44a888560 2024-03-26T19:12:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.064+0000 7f3489600640 1 -- 172.21.15.97:0/587803408 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3498017040 con 0x55a44a888560 2024-03-26T19:12:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.064+0000 7f34a9046740 1 -- 172.21.15.97:0/587803408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34900c16f0 con 0x55a44a888560 2024-03-26T19:12:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.065+0000 7f3489600640 1 -- 172.21.15.97:0/587803408 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34980223a0 con 0x55a44a888560 2024-03-26T19:12:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.067+0000 7f34a9046740 10 client.26248 fetch_fsmap learned FSMap version 27 2024-03-26T19:12:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.067+0000 7f34a9046740 1 -- 172.21.15.97:0/587803408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a44a8f5820 con 0x55a44a888560 2024-03-26T19:12:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.068+0000 7f3489600640 1 -- 172.21.15.97:0/587803408 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34900c16f0 con 0x55a44a888560 2024-03-26T19:12:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.068+0000 7f3489600640 1 -- 172.21.15.97:0/587803408 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3498019050 con 0x55a44a888560 2024-03-26T19:12:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.068+0000 7f34a9046740 10 client.26248 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:12:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.068+0000 7f34a9046740 10 client.26248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:12:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.068+0000 7f34a9046740 20 client.26248 populate_metadata read hostname 'smithi097' 2024-03-26T19:12:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.068+0000 7f34a9046740 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a44a8f61f0 0x55a44a8f8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.068+0000 7f34a9046740 4 client.26248 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:12:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.068+0000 7f34a9046740 1 -- 172.21.15.97:0/587803408 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a44a7d4710 con 0x55a44a8f61f0 2024-03-26T19:12:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.068+0000 7f349f400640 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a44a8f61f0 0x55a44a8f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.070+0000 7f349f400640 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a44a8f61f0 0x55a44a8f8610 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e99c1d17ea0772b3 server_cookie=f12a979881e6a518 in_seq=0 out_seq=0 2024-03-26T19:12:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.071+0000 7f3489600640 10 client.26248 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f3489600640 1 -- 172.21.15.97:0/587803408 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a44a7d4710 con 0x55a44a8f61f0 2024-03-26T19:12:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f3489600640 10 client.26248 handle_command_reply: tid=0 2024-03-26T19:12:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f34a9046740 1 client.26248 shutdown 2024-03-26T19:12:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f34a9046740 1 -- 172.21.15.97:0/587803408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a888560 msgr2=0x55a44a8a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f34a9046740 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a888560 0x55a44a8a1c90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55a44a867af0 tx=0x7f349801a3f0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f34a9046740 1 -- 172.21.15.97:0/587803408 shutdown_connections 2024-03-26T19:12:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f34a9046740 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55a44a8f61f0 0x55a44a8f8610 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f34a9046740 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a44a888560 0x55a44a8a1c90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f34a9046740 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44a868790 0x55a44a8a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f34a9046740 1 --2- 172.21.15.97:0/587803408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a44a867da0 0x55a44a8a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.134+0000 7f34a9046740 1 -- 172.21.15.97:0/587803408 >> 172.21.15.97:0/587803408 conn(0x55a44a863310 msgr2=0x55a44a866570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.133+0000 7f3489600640 0 client.26248 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.134+0000 7f34a9046740 1 -- 172.21.15.97:0/587803408 shutdown_connections 2024-03-26T19:12:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.134+0000 7f34a9046740 1 -- 172.21.15.97:0/587803408 wait complete. 2024-03-26T19:12:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.134+0000 7f34a9046740 20 client.26248 trim_cache size 0 max 16384 2024-03-26T19:12:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:12:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:12:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:12:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:12:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.138+0000 7f34a6400640 1 -- 172.21.15.97:0/2830824127 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f34840a3130 msgr2=0x7f34840a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.138+0000 7f34a6400640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f34840a3130 0x7f34840a55f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f348c012490 tx=0x7f348c010f70 comp rx=0 tx=0).stop 2024-03-26T19:12:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.138+0000 7f34a6400640 1 -- 172.21.15.97:0/2830824127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a0074320 msgr2=0x7f34a019e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.138+0000 7f34a6400640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a0074320 0x7f34a019e1c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3490005ab0 tx=0x7f349002c6c0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.138+0000 7f34a6400640 1 -- 172.21.15.97:0/2830824127 shutdown_connections 2024-03-26T19:12:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.138+0000 7f34a6400640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f34840a3130 0x7f34840a55f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.138+0000 7f34a6400640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34a010dee0 0x7f34a01a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.138+0000 7f34a6400640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a0075900 0x7f34a019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.138+0000 7f34a6400640 1 --2- 172.21.15.97:0/2830824127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a0074320 0x7f34a019e1c0 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3490005ab0 tx=0x7f349002c6c0 comp rx=0 tx=0).stop 2024-03-26T19:12:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.138+0000 7f34a6400640 1 -- 172.21.15.97:0/2830824127 >> 172.21.15.97:0/2830824127 conn(0x7f34a00fce80 msgr2=0x7f34a0101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.139+0000 7f34a6400640 1 -- 172.21.15.97:0/2830824127 shutdown_connections 2024-03-26T19:12:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:34.140+0000 7f34a6400640 1 -- 172.21.15.97:0/2830824127 wait complete. 2024-03-26T19:12:34.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:34 smithi105 ceph-mon[36270]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 MiB/s wr, 563 op/s 2024-03-26T19:12:34.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:12:34.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:12:34.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:34 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:12:34.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:34 smithi097 ceph-mon[31955]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 MiB/s wr, 563 op/s 2024-03-26T19:12:34.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:12:34.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:12:34.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:34 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:12:34.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:34 smithi144 ceph-mon[36053]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 MiB/s wr, 563 op/s 2024-03-26T19:12:34.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:12:34.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:12:34.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:34 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:12:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 790.0 s, 892.4 tps, lat 112.664 ms stddev 141.654 2024-03-26T19:12:36.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:36 smithi105 ceph-mon[36270]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.9 MiB/s wr, 718 op/s 2024-03-26T19:12:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:36 smithi097 ceph-mon[31955]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.9 MiB/s wr, 718 op/s 2024-03-26T19:12:36.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:36 smithi144 ceph-mon[36053]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.9 MiB/s wr, 718 op/s 2024-03-26T19:12:38.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:38 smithi105 ceph-mon[36270]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 MiB/s wr, 655 op/s 2024-03-26T19:12:38.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:38 smithi097 ceph-mon[31955]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 MiB/s wr, 655 op/s 2024-03-26T19:12:38.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:38 smithi144 ceph-mon[36053]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 MiB/s wr, 655 op/s 2024-03-26T19:12:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 795.0 s, 1130.8 tps, lat 88.069 ms stddev 109.399 2024-03-26T19:12:40.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:40 smithi105 ceph-mon[36270]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 MiB/s wr, 693 op/s 2024-03-26T19:12:40.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:40 smithi097 ceph-mon[31955]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 MiB/s wr, 693 op/s 2024-03-26T19:12:40.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:40 smithi144 ceph-mon[36053]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 MiB/s wr, 693 op/s 2024-03-26T19:12:42.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:42 smithi105 ceph-mon[36270]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 782 op/s 2024-03-26T19:12:42.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:42 smithi097 ceph-mon[31955]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 782 op/s 2024-03-26T19:12:42.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:42 smithi144 ceph-mon[36053]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 782 op/s 2024-03-26T19:12:44.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:44 smithi105 ceph-mon[36270]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 642 op/s 2024-03-26T19:12:44.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:44 smithi097 ceph-mon[31955]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 642 op/s 2024-03-26T19:12:44.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:44 smithi144 ceph-mon[36053]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 642 op/s 2024-03-26T19:12:45.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 800.0 s, 1203.2 tps, lat 84.049 ms stddev 97.032 2024-03-26T19:12:46.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:46 smithi097 ceph-mon[31955]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 806 op/s 2024-03-26T19:12:46.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:46 smithi144 ceph-mon[36053]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 806 op/s 2024-03-26T19:12:46.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:46 smithi105 ceph-mon[36270]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 806 op/s 2024-03-26T19:12:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.349+0000 7f5293e00640 1 -- 172.21.15.97:0/1560875878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c104d10 msgr2=0x7f528c105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.349+0000 7f5293e00640 1 --2- 172.21.15.97:0/1560875878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c104d10 0x7f528c105190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5278009900 tx=0x7f527802bc60 comp rx=0 tx=0).stop 2024-03-26T19:12:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.349+0000 7f5293e00640 1 -- 172.21.15.97:0/1560875878 shutdown_connections 2024-03-26T19:12:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.350+0000 7f5293e00640 1 --2- 172.21.15.97:0/1560875878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c1056d0 0x7f528c10bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.350+0000 7f5293e00640 1 --2- 172.21.15.97:0/1560875878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c104d10 0x7f528c105190 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.350+0000 7f5293e00640 1 --2- 172.21.15.97:0/1560875878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f528c103b20 0x7f528c103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.350+0000 7f5293e00640 1 -- 172.21.15.97:0/1560875878 >> 172.21.15.97:0/1560875878 conn(0x7f528c0ff0b0 msgr2=0x7f528c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.350+0000 7f5293e00640 1 -- 172.21.15.97:0/1560875878 shutdown_connections 2024-03-26T19:12:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.350+0000 7f5293e00640 1 -- 172.21.15.97:0/1560875878 wait complete. 2024-03-26T19:12:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.351+0000 7f5293e00640 1 Processor -- start 2024-03-26T19:12:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.352+0000 7f5293e00640 1 -- start start 2024-03-26T19:12:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.352+0000 7f5293e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c103b20 0x7f528c199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.352+0000 7f5293e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c104d10 0x7f528c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.352+0000 7f5291400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c103b20 0x7f528c199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.352+0000 7f5291400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c103b20 0x7f528c199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57480/0 (socket says 172.21.15.97:57480) 2024-03-26T19:12:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5293e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f528c1056d0 0x7f528c1a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5293e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f528c10f870 con 0x7f528c1056d0 2024-03-26T19:12:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5291400640 1 -- 172.21.15.97:0/1439778149 learned_addr learned my addr 172.21.15.97:0/1439778149 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5293e00640 1 -- 172.21.15.97:0/1439778149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f528c10f6f0 con 0x7f528c103b20 2024-03-26T19:12:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5293e00640 1 -- 172.21.15.97:0/1439778149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f528c10f9f0 con 0x7f528c104d10 2024-03-26T19:12:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5290a00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c104d10 0x7f528c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5290a00640 1 -- 172.21.15.97:0/1439778149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c103b20 msgr2=0x7f528c199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5290a00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c103b20 0x7f528c199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5290a00640 1 -- 172.21.15.97:0/1439778149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f528c1056d0 msgr2=0x7f528c1a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5290a00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f528c1056d0 0x7f528c1a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.353+0000 7f5290a00640 1 -- 172.21.15.97:0/1439778149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c1a1a20 con 0x7f528c104d10 2024-03-26T19:12:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.354+0000 7f5290a00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c104d10 0x7f528c19a340 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5278036040 tx=0x7f527802c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.354+0000 7f528a000640 1 -- 172.21.15.97:0/1439778149 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278041070 con 0x7f528c104d10 2024-03-26T19:12:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.354+0000 7f528a000640 1 -- 172.21.15.97:0/1439778149 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52780054e0 con 0x7f528c104d10 2024-03-26T19:12:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.354+0000 7f528a000640 1 -- 172.21.15.97:0/1439778149 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278034440 con 0x7f528c104d10 2024-03-26T19:12:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.355+0000 7f5293e00640 1 -- 172.21.15.97:0/1439778149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528c1a1cb0 con 0x7f528c104d10 2024-03-26T19:12:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.355+0000 7f5293e00640 1 -- 172.21.15.97:0/1439778149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f528c1a2120 con 0x7f528c104d10 2024-03-26T19:12:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.361+0000 7f5296b6e740 1 Processor -- start 2024-03-26T19:12:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.361+0000 7f5296b6e740 1 -- start start 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.361+0000 7f528a000640 1 -- 172.21.15.97:0/1439778149 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f5278044070 con 0x7f528c104d10 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.361+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef81309da0 0x55ef8130a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.361+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130a790 0x55ef8130f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.361+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81311d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.361+0000 7f5296b6e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ef8126ee80 con 0x55ef81309da0 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.361+0000 7f5296b6e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ef81217660 con 0x55ef8130f900 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.361+0000 7f5296b6e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef8126ae30 con 0x55ef8130a790 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5290a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81311d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5290a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81311d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57490/0 (socket says 172.21.15.97:57490) 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5290a00640 1 -- 172.21.15.97:0/2372520331 learned_addr learned my addr 172.21.15.97:0/2372520331 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5291400640 1 --2- 172.21.15.97:0/2372520331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130a790 0x55ef8130f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5291e00640 1 --2- 172.21.15.97:0/2372520331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef81309da0 0x55ef8130a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5291400640 1 -- 172.21.15.97:0/2372520331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 msgr2=0x55ef81311d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5291400640 1 --2- 172.21.15.97:0/2372520331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81311d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5291400640 1 -- 172.21.15.97:0/2372520331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef81309da0 msgr2=0x55ef8130a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5291400640 1 --2- 172.21.15.97:0/2372520331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef81309da0 0x55ef8130a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5291400640 1 -- 172.21.15.97:0/2372520331 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef80f83210 con 0x55ef8130a790 2024-03-26T19:12:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.362+0000 7f5290a00640 1 --2- 172.21.15.97:0/2372520331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81311d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:12:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.363+0000 7f5291400640 1 --2- 172.21.15.97:0/2372520331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130a790 0x55ef8130f290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5284007ee0 tx=0x7f528400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d346cb724ca67b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.363+0000 7f527e600640 1 -- 172.21.15.97:0/2372520331 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284003b70 con 0x55ef8130a790 2024-03-26T19:12:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.363+0000 7f528a000640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52740a3180 0x7f52740a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.363+0000 7f528a000640 1 -- 172.21.15.97:0/1439778149 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f52780f1090 con 0x7f528c104d10 2024-03-26T19:12:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.363+0000 7f527e600640 1 -- 172.21.15.97:0/2372520331 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5284003d10 con 0x55ef8130a790 2024-03-26T19:12:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.363+0000 7f527e600640 1 -- 172.21.15.97:0/2372520331 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284010c30 con 0x55ef8130a790 2024-03-26T19:12:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.364+0000 7f5291e00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52740a3180 0x7f52740a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.364+0000 7f5296b6e740 1 -- 172.21.15.97:0/2372520331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130a790 msgr2=0x55ef8130f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.364+0000 7f5296b6e740 1 --2- 172.21.15.97:0/2372520331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130a790 0x55ef8130f290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5284007ee0 tx=0x7f528400f790 comp rx=0 tx=0).stop 2024-03-26T19:12:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.364+0000 7f5296b6e740 1 -- 172.21.15.97:0/2372520331 shutdown_connections 2024-03-26T19:12:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.364+0000 7f5296b6e740 1 --2- 172.21.15.97:0/2372520331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81311d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.364+0000 7f5296b6e740 1 --2- 172.21.15.97:0/2372520331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130a790 0x55ef8130f290 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.364+0000 7f5296b6e740 1 --2- 172.21.15.97:0/2372520331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef81309da0 0x55ef8130a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.364+0000 7f5296b6e740 1 -- 172.21.15.97:0/2372520331 >> 172.21.15.97:0/2372520331 conn(0x55ef81305310 msgr2=0x55ef81305710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.365+0000 7f5296b6e740 1 -- 172.21.15.97:0/2372520331 shutdown_connections 2024-03-26T19:12:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.365+0000 7f5291e00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52740a3180 0x7f52740a5640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55ef8130aff0 tx=0x7f5280008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.365+0000 7f5296b6e740 1 -- 172.21.15.97:0/2372520331 wait complete. 2024-03-26T19:12:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.365+0000 7f5296b6e740 1 -- start start 2024-03-26T19:12:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.366+0000 7f5296b6e740 1 Processor -- start 2024-03-26T19:12:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.367+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef81309da0 0x55ef8131c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.367+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130a790 0x55ef8131ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.367+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81323aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.368+0000 7f5296b6e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ef813128b0 con 0x55ef8130a790 2024-03-26T19:12:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.368+0000 7f5291e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81323aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.368+0000 7f5291e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81323aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57506/0 (socket says 172.21.15.97:57506) 2024-03-26T19:12:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.368+0000 7f5291e00640 1 -- 172.21.15.97:0/3834207603 learned_addr learned my addr 172.21.15.97:0/3834207603 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.368+0000 7f5290a00640 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130a790 0x55ef8131ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.368+0000 7f5291400640 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef81309da0 0x55ef8131c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.368+0000 7f5296b6e740 1 -- 172.21.15.97:0/3834207603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ef8122aac0 con 0x55ef8130f900 2024-03-26T19:12:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.368+0000 7f5296b6e740 1 -- 172.21.15.97:0/3834207603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef81312a30 con 0x55ef81309da0 2024-03-26T19:12:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.369+0000 7f5291e00640 1 -- 172.21.15.97:0/3834207603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef81309da0 msgr2=0x55ef8131c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.369+0000 7f5291e00640 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef81309da0 0x55ef8131c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.369+0000 7f5296b6e740 10 client.0 _ll_register_callbacks cb 0x55ef8122a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:12:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.369+0000 7f5291e00640 1 -- 172.21.15.97:0/3834207603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130a790 msgr2=0x55ef8131ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.369+0000 7f5291e00640 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130a790 0x55ef8131ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.370+0000 7f5291e00640 1 -- 172.21.15.97:0/3834207603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef8104f620 con 0x55ef8130f900 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.370+0000 7f5296b6e740 1 -- 172.21.15.97:0/3834207603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5284003d10 con 0x55ef8130f900 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.370+0000 7f5291e00640 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81323aa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f528000dd60 tx=0x7f52800047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=168da63e010420b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.371+0000 7f5272a00640 10 client.26276 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.371+0000 7f5272a00640 1 -- 172.21.15.97:0/3834207603 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280024060 con 0x55ef8130f900 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.372+0000 7f5272a00640 1 -- 172.21.15.97:0/3834207603 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f528001f400 con 0x55ef8130f900 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.372+0000 7f5272a00640 1 -- 172.21.15.97:0/3834207603 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f528001f600 con 0x55ef8130f900 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.372+0000 7f5296b6e740 10 client.26276 fetch_fsmap learned FSMap version 27 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.372+0000 7f5296b6e740 1 -- 172.21.15.97:0/3834207603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55ef81326da0 con 0x55ef8130f900 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.372+0000 7f5272a00640 1 -- 172.21.15.97:0/3834207603 <== mon.2 v2:172.21.15.105:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5280013070 con 0x55ef8130f900 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.372+0000 7f5296b6e740 10 client.26276 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.372+0000 7f5296b6e740 10 client.26276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.372+0000 7f5296b6e740 20 client.26276 populate_metadata read hostname 'smithi097' 2024-03-26T19:12:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.372+0000 7f5296b6e740 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ef813283d0 0x55ef8132a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.373+0000 7f5296b6e740 4 client.26276 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:12:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.373+0000 7f5296b6e740 1 -- 172.21.15.97:0/3834207603 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef8127c3c0 con 0x55ef813283d0 2024-03-26T19:12:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.373+0000 7f5291400640 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ef813283d0 0x55ef8132a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.373+0000 7f5291400640 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ef813283d0 0x55ef8132a820 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d118f8f238aafec server_cookie=e324af2dee42a6c7 in_seq=0 out_seq=0 2024-03-26T19:12:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.373+0000 7f5272a00640 10 client.26276 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.375+0000 7f5272a00640 1 -- 172.21.15.97:0/3834207603 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ef8127c3c0 con 0x55ef813283d0 2024-03-26T19:12:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.375+0000 7f5272a00640 10 client.26276 handle_command_reply: tid=0 2024-03-26T19:12:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.375+0000 7f5296b6e740 1 client.26276 shutdown 2024-03-26T19:12:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.376+0000 7f5296b6e740 1 -- 172.21.15.97:0/3834207603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 msgr2=0x55ef81323aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.376+0000 7f5296b6e740 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81323aa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f528000dd60 tx=0x7f52800047e0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.376+0000 7f5296b6e740 1 -- 172.21.15.97:0/3834207603 shutdown_connections 2024-03-26T19:12:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.376+0000 7f5296b6e740 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ef813283d0 0x55ef8132a820 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.376+0000 7f5296b6e740 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130f900 0x55ef81323aa0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.376+0000 7f5296b6e740 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130a790 0x55ef8131ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.376+0000 7f5296b6e740 1 --2- 172.21.15.97:0/3834207603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef81309da0 0x55ef8131c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.376+0000 7f5296b6e740 1 -- 172.21.15.97:0/3834207603 >> 172.21.15.97:0/3834207603 conn(0x55ef81305310 msgr2=0x55ef81304540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.376+0000 7f5272a00640 0 client.26276 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.377+0000 7f5296b6e740 1 -- 172.21.15.97:0/3834207603 shutdown_connections 2024-03-26T19:12:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.377+0000 7f5296b6e740 1 -- 172.21.15.97:0/3834207603 wait complete. 2024-03-26T19:12:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.377+0000 7f5296b6e740 20 client.26276 trim_cache size 0 max 16384 2024-03-26T19:12:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.404+0000 7f5296b6e740 1 Processor -- start 2024-03-26T19:12:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5296b6e740 1 -- start start 2024-03-26T19:12:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef81309da0 0x55ef81303cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 0x55ef81323a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130f900 0x55ef813041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5296b6e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1a69d0 con 0x55ef8130f900 2024-03-26T19:12:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5296b6e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1a8510 con 0x55ef8130a790 2024-03-26T19:12:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5296b6e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1a70a0 con 0x55ef81309da0 2024-03-26T19:12:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5290a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 0x55ef81323a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5291e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130f900 0x55ef813041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5291e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130f900 0x55ef813041f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33250/0 (socket says 172.21.15.97:33250) 2024-03-26T19:12:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.405+0000 7f5291e00640 1 -- 172.21.15.97:0/4162303914 learned_addr learned my addr 172.21.15.97:0/4162303914 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.406+0000 7f5291400640 1 --2- 172.21.15.97:0/4162303914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef81309da0 0x55ef81303cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.406+0000 7f5290a00640 1 -- 172.21.15.97:0/4162303914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef81309da0 msgr2=0x55ef81303cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.406+0000 7f5290a00640 1 --2- 172.21.15.97:0/4162303914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef81309da0 0x55ef81303cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.406+0000 7f5290a00640 1 -- 172.21.15.97:0/4162303914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130f900 msgr2=0x55ef813041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.406+0000 7f5290a00640 1 --2- 172.21.15.97:0/4162303914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130f900 0x55ef813041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.406+0000 7f5290a00640 1 -- 172.21.15.97:0/4162303914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef81305c90 con 0x55ef8130a790 2024-03-26T19:12:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.406+0000 7f5291e00640 1 --2- 172.21.15.97:0/4162303914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130f900 0x55ef813041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:12:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.409+0000 7f5290a00640 1 --2- 172.21.15.97:0/4162303914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 0x55ef81323a60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f52780eec10 tx=0x7f52780f4b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81d1ee33f3958617 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.413+0000 7f527dc00640 1 -- 172.21.15.97:0/4162303914 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527804b070 con 0x55ef8130a790 2024-03-26T19:12:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.414+0000 7f527dc00640 1 -- 172.21.15.97:0/4162303914 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5278004c80 con 0x55ef8130a790 2024-03-26T19:12:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.414+0000 7f527dc00640 1 -- 172.21.15.97:0/4162303914 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52780bd880 con 0x55ef8130a790 2024-03-26T19:12:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.415+0000 7f5296b6e740 1 -- 172.21.15.97:0/4162303914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 msgr2=0x55ef81323a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.415+0000 7f5296b6e740 1 --2- 172.21.15.97:0/4162303914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 0x55ef81323a60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f52780eec10 tx=0x7f52780f4b70 comp rx=0 tx=0).stop 2024-03-26T19:12:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.415+0000 7f5296b6e740 1 -- 172.21.15.97:0/4162303914 shutdown_connections 2024-03-26T19:12:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.416+0000 7f5296b6e740 1 --2- 172.21.15.97:0/4162303914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef8130f900 0x55ef813041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.416+0000 7f5296b6e740 1 --2- 172.21.15.97:0/4162303914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 0x55ef81323a60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.416+0000 7f5296b6e740 1 --2- 172.21.15.97:0/4162303914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef81309da0 0x55ef81303cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.416+0000 7f5296b6e740 1 -- 172.21.15.97:0/4162303914 >> 172.21.15.97:0/4162303914 conn(0x55ef81305310 msgr2=0x55ef8130bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.418+0000 7f5296b6e740 1 -- 172.21.15.97:0/4162303914 shutdown_connections 2024-03-26T19:12:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.418+0000 7f5296b6e740 1 -- 172.21.15.97:0/4162303914 wait complete. 2024-03-26T19:12:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.418+0000 7f5296b6e740 1 -- start start 2024-03-26T19:12:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.419+0000 7f5296b6e740 1 Processor -- start 2024-03-26T19:12:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.419+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef81309da0 0x55ef81321460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.419+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 0x55ef813219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.419+0000 7f5296b6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130f900 0x55ef813229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.419+0000 7f5290a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 0x55ef813219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.419+0000 7f5290a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 0x55ef813219a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57526/0 (socket says 172.21.15.97:57526) 2024-03-26T19:12:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5290a00640 1 -- 172.21.15.97:0/930168302 learned_addr learned my addr 172.21.15.97:0/930168302 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:12:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1a7e40 con 0x55ef81309da0 2024-03-26T19:12:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1a7770 con 0x55ef8130a790 2024-03-26T19:12:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1a49b0 con 0x55ef8130f900 2024-03-26T19:12:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5291e00640 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130f900 0x55ef813229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5296b6e740 10 client.0 _ll_register_callbacks cb 0x7f528000bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:12:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5291e00640 1 -- 172.21.15.97:0/930168302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 msgr2=0x55ef813219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5291e00640 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 0x55ef813219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5291e00640 1 -- 172.21.15.97:0/930168302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef81309da0 msgr2=0x55ef81321460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:12:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5291e00640 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef81309da0 0x55ef81321460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.420+0000 7f5291e00640 1 -- 172.21.15.97:0/930168302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef81322fa0 con 0x55ef8130f900 2024-03-26T19:12:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.421+0000 7f5272a00640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:12:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.421+0000 7f5291e00640 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130f900 0x55ef813229b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f528000d530 tx=0x7f5280012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=875505f720a374c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:12:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.421+0000 7f5272a00640 1 -- 172.21.15.97:0/930168302 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280024040 con 0x55ef8130f900 2024-03-26T19:12:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.421+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5278004c80 con 0x55ef8130f900 2024-03-26T19:12:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.421+0000 7f5272a00640 1 -- 172.21.15.97:0/930168302 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f528000f040 con 0x55ef8130f900 2024-03-26T19:12:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.422+0000 7f5272a00640 1 -- 172.21.15.97:0/930168302 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5280024040 con 0x55ef8130f900 2024-03-26T19:12:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.422+0000 7f5296b6e740 10 client.26266 fetch_fsmap learned FSMap version 27 2024-03-26T19:12:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.422+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef81312ef0 con 0x55ef8130f900 2024-03-26T19:12:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.422+0000 7f5272a00640 1 -- 172.21.15.97:0/930168302 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5278004c80 con 0x55ef8130f900 2024-03-26T19:12:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.422+0000 7f5272a00640 1 -- 172.21.15.97:0/930168302 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5280006040 con 0x55ef8130f900 2024-03-26T19:12:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.422+0000 7f5296b6e740 10 client.26266 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:12:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.422+0000 7f5296b6e740 10 client.26266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:12:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.422+0000 7f5296b6e740 20 client.26266 populate_metadata read hostname 'smithi097' 2024-03-26T19:12:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.422+0000 7f5296b6e740 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ef81313830 0x55ef81315c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:12:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.423+0000 7f5296b6e740 4 client.26266 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:12:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.423+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef8132abb0 con 0x55ef81313830 2024-03-26T19:12:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.424+0000 7f5291400640 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ef81313830 0x55ef81315c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:12:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.425+0000 7f5291400640 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ef81313830 0x55ef81315c50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec814bb2871ffde server_cookie=f8e53ac7bf5b2741 in_seq=0 out_seq=0 2024-03-26T19:12:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.425+0000 7f5272a00640 10 client.26266 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.480+0000 7f5272a00640 1 -- 172.21.15.97:0/930168302 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef8132abb0 con 0x55ef81313830 2024-03-26T19:12:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.480+0000 7f5272a00640 10 client.26266 handle_command_reply: tid=0 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.480+0000 7f5296b6e740 1 client.26266 shutdown 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.480+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130f900 msgr2=0x55ef813229b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.480+0000 7f5296b6e740 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130f900 0x55ef813229b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f528000d530 tx=0x7f5280012000 comp rx=0 tx=0).stop 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.481+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 shutdown_connections 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.481+0000 7f5296b6e740 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ef81313830 0x55ef81315c50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.481+0000 7f5296b6e740 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef8130f900 0x55ef813229b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.481+0000 7f5296b6e740 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef8130a790 0x55ef813219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.481+0000 7f5296b6e740 1 --2- 172.21.15.97:0/930168302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef81309da0 0x55ef81321460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.481+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 >> 172.21.15.97:0/930168302 conn(0x55ef81305310 msgr2=0x55ef81308570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.481+0000 7f5272a00640 0 client.26266 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:12:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.481+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 shutdown_connections 2024-03-26T19:12:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.481+0000 7f5296b6e740 1 -- 172.21.15.97:0/930168302 wait complete. 2024-03-26T19:12:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.481+0000 7f5296b6e740 20 client.26266 trim_cache size 0 max 16384 2024-03-26T19:12:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:12:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:12:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:12:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:12:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.482+0000 7f5293e00640 1 -- 172.21.15.97:0/1439778149 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52740a3180 msgr2=0x7f52740a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.482+0000 7f5293e00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52740a3180 0x7f52740a5640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55ef8130aff0 tx=0x7f5280008040 comp rx=0 tx=0).stop 2024-03-26T19:12:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.482+0000 7f5293e00640 1 -- 172.21.15.97:0/1439778149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c104d10 msgr2=0x7f528c19a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:12:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.482+0000 7f5293e00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c104d10 0x7f528c19a340 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5278036040 tx=0x7f527802c670 comp rx=0 tx=0).stop 2024-03-26T19:12:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.482+0000 7f5293e00640 1 -- 172.21.15.97:0/1439778149 shutdown_connections 2024-03-26T19:12:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.482+0000 7f5293e00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f52740a3180 0x7f52740a5640 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.482+0000 7f5293e00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f528c1056d0 0x7f528c1a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.482+0000 7f5293e00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c104d10 0x7f528c19a340 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.482+0000 7f5293e00640 1 --2- 172.21.15.97:0/1439778149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c103b20 0x7f528c199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:12:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.482+0000 7f5293e00640 1 -- 172.21.15.97:0/1439778149 >> 172.21.15.97:0/1439778149 conn(0x7f528c0ff0b0 msgr2=0x7f528c108010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:12:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.483+0000 7f5293e00640 1 -- 172.21.15.97:0/1439778149 shutdown_connections 2024-03-26T19:12:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:12:47.483+0000 7f5293e00640 1 -- 172.21.15.97:0/1439778149 wait complete. 2024-03-26T19:12:48.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:48 smithi097 ceph-mon[31955]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 672 op/s 2024-03-26T19:12:48.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:48 smithi144 ceph-mon[36053]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 672 op/s 2024-03-26T19:12:48.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:48 smithi105 ceph-mon[36270]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 672 op/s 2024-03-26T19:12:50.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 805.0 s, 1294.2 tps, lat 76.969 ms stddev 89.310 2024-03-26T19:12:50.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:50 smithi097 ceph-mon[31955]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 670 op/s 2024-03-26T19:12:50.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:50 smithi144 ceph-mon[36053]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 670 op/s 2024-03-26T19:12:50.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:50 smithi105 ceph-mon[36270]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 670 op/s 2024-03-26T19:12:52.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:52 smithi097 ceph-mon[31955]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 744 op/s 2024-03-26T19:12:52.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:52 smithi144 ceph-mon[36053]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 744 op/s 2024-03-26T19:12:52.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:52 smithi105 ceph-mon[36270]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 744 op/s 2024-03-26T19:12:54.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:54 smithi097 ceph-mon[31955]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 615 op/s 2024-03-26T19:12:54.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:54 smithi144 ceph-mon[36053]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 615 op/s 2024-03-26T19:12:54.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:54 smithi105 ceph-mon[36270]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 615 op/s 2024-03-26T19:12:55.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 810.0 s, 1453.2 tps, lat 69.364 ms stddev 84.672 2024-03-26T19:12:56.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:56 smithi097 ceph-mon[31955]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 683 op/s 2024-03-26T19:12:56.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:56 smithi144 ceph-mon[36053]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 683 op/s 2024-03-26T19:12:56.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:56 smithi105 ceph-mon[36270]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 683 op/s 2024-03-26T19:12:57.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:12:57 smithi097 ceph-mon[31955]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 607 op/s 2024-03-26T19:12:57.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:12:57 smithi144 ceph-mon[36053]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 607 op/s 2024-03-26T19:12:57.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:12:57 smithi105 ceph-mon[36270]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 607 op/s 2024-03-26T19:13:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 815.0 s, 1295.6 tps, lat 75.609 ms stddev 95.457 2024-03-26T19:13:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.662+0000 7f8574e00640 1 -- 172.21.15.97:0/3995554641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8570104ac0 msgr2=0x7f8570106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.662+0000 7f8574e00640 1 --2- 172.21.15.97:0/3995554641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8570104ac0 0x7f8570106eb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f855c009900 tx=0x7f855c02bc30 comp rx=0 tx=0).stop 2024-03-26T19:13:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.662+0000 7f8574e00640 1 -- 172.21.15.97:0/3995554641 shutdown_connections 2024-03-26T19:13:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.662+0000 7f8574e00640 1 --2- 172.21.15.97:0/3995554641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85701073f0 0x7f8570109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.662+0000 7f8574e00640 1 --2- 172.21.15.97:0/3995554641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8570104ac0 0x7f8570106eb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.662+0000 7f8574e00640 1 --2- 172.21.15.97:0/3995554641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8570102190 0x7f8570104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.663+0000 7f8574e00640 1 -- 172.21.15.97:0/3995554641 >> 172.21.15.97:0/3995554641 conn(0x7f85700fbba0 msgr2=0x7f85700fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.663+0000 7f8574e00640 1 -- 172.21.15.97:0/3995554641 shutdown_connections 2024-03-26T19:13:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.663+0000 7f8574e00640 1 -- 172.21.15.97:0/3995554641 wait complete. 2024-03-26T19:13:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.665+0000 7f8574e00640 1 Processor -- start 2024-03-26T19:13:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.666+0000 7f8574e00640 1 -- start start 2024-03-26T19:13:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.666+0000 7f8574e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8570102190 0x7f8570070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f8574e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8570104ac0 0x7f85700707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f8574e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85701073f0 0x7f85701a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f8574e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f857010d3d0 con 0x7f85701073f0 2024-03-26T19:13:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f8574e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f857010d250 con 0x7f8570102190 2024-03-26T19:13:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f8574e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f857010d550 con 0x7f8570104ac0 2024-03-26T19:13:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8570102190 0x7f8570070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8570102190 0x7f8570070280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57338/0 (socket says 172.21.15.97:57338) 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856f400640 1 -- 172.21.15.97:0/4045585628 learned_addr learned my addr 172.21.15.97:0/4045585628 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856ea00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8570104ac0 0x7f85700707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856fe00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85701073f0 0x7f85701a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856ea00640 1 -- 172.21.15.97:0/4045585628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8570102190 msgr2=0x7f8570070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856ea00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8570102190 0x7f8570070280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856ea00640 1 -- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85701073f0 msgr2=0x7f85701a7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856ea00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85701073f0 0x7f85701a7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856ea00640 1 -- 172.21.15.97:0/4045585628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85701a8020 con 0x7f8570104ac0 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856f400640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8570102190 0x7f8570070280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.667+0000 7f856fe00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85701073f0 0x7f85701a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.668+0000 7f856ea00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8570104ac0 0x7f85700707c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f855c009230 tx=0x7f855c02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.668+0000 7f8567e00640 1 -- 172.21.15.97:0/4045585628 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c043020 con 0x7f8570104ac0 2024-03-26T19:13:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.668+0000 7f8567e00640 1 -- 172.21.15.97:0/4045585628 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855c0054e0 con 0x7f8570104ac0 2024-03-26T19:13:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.668+0000 7f8567e00640 1 -- 172.21.15.97:0/4045585628 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c003c60 con 0x7f8570104ac0 2024-03-26T19:13:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.668+0000 7f8574e00640 1 -- 172.21.15.97:0/4045585628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85701a81f0 con 0x7f8570104ac0 2024-03-26T19:13:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.668+0000 7f8574e00640 1 -- 172.21.15.97:0/4045585628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85701a8600 con 0x7f8570104ac0 2024-03-26T19:13:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.673+0000 7f85791e6740 1 Processor -- start 2024-03-26T19:13:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.673+0000 7f8567e00640 1 -- 172.21.15.97:0/4045585628 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f855c046020 con 0x7f8570104ac0 2024-03-26T19:13:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.673+0000 7f85791e6740 1 -- start start 2024-03-26T19:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f85791e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 0x563b4fea6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f85791e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 0x563b4feab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f85791e6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4feabac0 0x563b4feadf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f85791e6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563b4fe0ae80 con 0x563b4feabac0 2024-03-26T19:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f85791e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563b4fdb3660 con 0x563b4fea6950 2024-03-26T19:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f85791e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563b4fe06e30 con 0x563b4fea5f60 2024-03-26T19:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f856fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 0x563b4fea6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f856fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 0x563b4fea6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:36734/0 (socket says 172.21.15.97:36734) 2024-03-26T19:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f856fe00640 1 -- 172.21.15.97:0/1176162434 learned_addr learned my addr 172.21.15.97:0/1176162434 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f856fe00640 1 -- 172.21.15.97:0/1176162434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 msgr2=0x563b4feab450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:13:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f856fe00640 1 --2- 172.21.15.97:0/1176162434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 0x563b4feab450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f856fe00640 1 -- 172.21.15.97:0/1176162434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4feabac0 msgr2=0x563b4feadf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f856fe00640 1 --2- 172.21.15.97:0/1176162434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4feabac0 0x563b4feadf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.674+0000 7f856fe00640 1 -- 172.21.15.97:0/1176162434 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4fb1f210 con 0x563b4fea5f60 2024-03-26T19:13:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.675+0000 7f856fe00640 1 --2- 172.21.15.97:0/1176162434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 0x563b4fea6380 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8568009f10 tx=0x7f856800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8953d7c799dea23 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.675+0000 7f8564c00640 1 -- 172.21.15.97:0/1176162434 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568003b70 con 0x563b4fea5f60 2024-03-26T19:13:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.675+0000 7f8564c00640 1 -- 172.21.15.97:0/1176162434 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8568003d10 con 0x563b4fea5f60 2024-03-26T19:13:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.675+0000 7f8564c00640 1 -- 172.21.15.97:0/1176162434 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568011ae0 con 0x563b4fea5f60 2024-03-26T19:13:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.675+0000 7f8567e00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85540a2f90 0x7f85540a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.676+0000 7f8567e00640 1 -- 172.21.15.97:0/4045585628 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f855c0f1580 con 0x7f8570104ac0 2024-03-26T19:13:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.676+0000 7f85791e6740 1 -- 172.21.15.97:0/1176162434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 msgr2=0x563b4fea6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.676+0000 7f85791e6740 1 --2- 172.21.15.97:0/1176162434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 0x563b4fea6380 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8568009f10 tx=0x7f856800f730 comp rx=0 tx=0).stop 2024-03-26T19:13:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.676+0000 7f85791e6740 1 -- 172.21.15.97:0/1176162434 shutdown_connections 2024-03-26T19:13:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.676+0000 7f85791e6740 1 --2- 172.21.15.97:0/1176162434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4feabac0 0x563b4feadf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.676+0000 7f85791e6740 1 --2- 172.21.15.97:0/1176162434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 0x563b4feab450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.676+0000 7f85791e6740 1 --2- 172.21.15.97:0/1176162434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 0x563b4fea6380 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.676+0000 7f85791e6740 1 -- 172.21.15.97:0/1176162434 >> 172.21.15.97:0/1176162434 conn(0x563b4fea14d0 msgr2=0x563b4fea18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.677+0000 7f856f400640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85540a2f90 0x7f85540a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.677+0000 7f85791e6740 1 -- 172.21.15.97:0/1176162434 shutdown_connections 2024-03-26T19:13:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.677+0000 7f85791e6740 1 -- 172.21.15.97:0/1176162434 wait complete. 2024-03-26T19:13:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.678+0000 7f85791e6740 1 -- start start 2024-03-26T19:13:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.678+0000 7f856f400640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85540a2f90 0x7f85540a5450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x563b4fea2440 tx=0x7f856000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.678+0000 7f85791e6740 1 Processor -- start 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.679+0000 7f85791e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 0x563b4feb4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.679+0000 7f85791e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 0x563b4feb49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.679+0000 7f85791e6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4feabac0 0x563b4febb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.679+0000 7f85791e6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563b4feb2e40 con 0x563b4feabac0 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.679+0000 7f85791e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563b4fdc6ac0 con 0x563b4fea6950 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.679+0000 7f85791e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563b4feb2fc0 con 0x563b4fea5f60 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.679+0000 7f85791e6740 10 client.0 _ll_register_callbacks cb 0x563b4fdc6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.679+0000 7f856ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 0x563b4feb49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.679+0000 7f856ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 0x563b4feb49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57354/0 (socket says 172.21.15.97:57354) 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.679+0000 7f856ea00640 1 -- 172.21.15.97:0/3354766045 learned_addr learned my addr 172.21.15.97:0/3354766045 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.680+0000 7f856fe00640 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4feabac0 0x563b4febb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.680+0000 7f856f400640 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 0x563b4feb4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.680+0000 7f856ea00640 1 -- 172.21.15.97:0/3354766045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 msgr2=0x563b4feb4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.680+0000 7f856ea00640 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 0x563b4feb4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.680+0000 7f856ea00640 1 -- 172.21.15.97:0/3354766045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4feabac0 msgr2=0x563b4febb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.680+0000 7f856ea00640 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4feabac0 0x563b4febb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.680+0000 7f856ea00640 1 -- 172.21.15.97:0/3354766045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4fbeb620 con 0x563b4fea6950 2024-03-26T19:13:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.682+0000 7f856ea00640 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 0x563b4feb49b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f855c0ba0e0 tx=0x7f855c0bd880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba7ed278ce673ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.682+0000 7f8559600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:13:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.683+0000 7f8559600640 1 -- 172.21.15.97:0/3354766045 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0be380 con 0x563b4fea6950 2024-03-26T19:13:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.683+0000 7f8559600640 1 -- 172.21.15.97:0/3354766045 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f855c0bbf80 con 0x563b4fea6950 2024-03-26T19:13:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.685+0000 7f85791e6740 1 -- 172.21.15.97:0/3354766045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8568003d10 con 0x563b4fea6950 2024-03-26T19:13:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.685+0000 7f8559600640 1 -- 172.21.15.97:0/3354766045 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f855c0bdc80 con 0x563b4fea6950 2024-03-26T19:13:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.685+0000 7f85791e6740 10 client.26291 fetch_fsmap learned FSMap version 27 2024-03-26T19:13:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.685+0000 7f85791e6740 1 -- 172.21.15.97:0/3354766045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b4feae990 con 0x563b4fea6950 2024-03-26T19:13:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.685+0000 7f8559600640 1 -- 172.21.15.97:0/3354766045 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8568003d10 con 0x563b4fea6950 2024-03-26T19:13:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.686+0000 7f8559600640 1 -- 172.21.15.97:0/3354766045 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f855c0c6300 con 0x563b4fea6950 2024-03-26T19:13:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.686+0000 7f85791e6740 10 client.26291 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:13:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.686+0000 7f85791e6740 10 client.26291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:13:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.686+0000 7f85791e6740 20 client.26291 populate_metadata read hostname 'smithi097' 2024-03-26T19:13:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.686+0000 7f85791e6740 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x563b4feb00a0 0x563b4feb24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.686+0000 7f85791e6740 4 client.26291 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.686+0000 7f85791e6740 1 -- 172.21.15.97:0/3354766045 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b4fe183c0 con 0x563b4feb00a0 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.686+0000 7f856f400640 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x563b4feb00a0 0x563b4feb24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.687+0000 7f856f400640 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x563b4feb00a0 0x563b4feb24c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f8738e1af6e290 server_cookie=92c87e1c867f6bd8 in_seq=0 out_seq=0 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.687+0000 7f8559600640 10 client.26291 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.687+0000 7f8559600640 1 -- 172.21.15.97:0/3354766045 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563b4fe183c0 con 0x563b4feb00a0 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.687+0000 7f8559600640 10 client.26291 handle_command_reply: tid=0 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.687+0000 7f85791e6740 1 client.26291 shutdown 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.688+0000 7f85791e6740 1 -- 172.21.15.97:0/3354766045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 msgr2=0x563b4feb49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.688+0000 7f85791e6740 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 0x563b4feb49b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f855c0ba0e0 tx=0x7f855c0bd880 comp rx=0 tx=0).stop 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.688+0000 7f85791e6740 1 -- 172.21.15.97:0/3354766045 shutdown_connections 2024-03-26T19:13:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.688+0000 7f85791e6740 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x563b4feb00a0 0x563b4feb24c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.688+0000 7f85791e6740 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4feabac0 0x563b4febb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.688+0000 7f8559600640 0 client.26291 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.688+0000 7f85791e6740 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4fea6950 0x563b4feb49b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.688+0000 7f85791e6740 1 --2- 172.21.15.97:0/3354766045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea5f60 0x563b4feb4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.688+0000 7f85791e6740 1 -- 172.21.15.97:0/3354766045 >> 172.21.15.97:0/3354766045 conn(0x563b4fea14d0 msgr2=0x563b4fea06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.689+0000 7f85791e6740 1 -- 172.21.15.97:0/3354766045 shutdown_connections 2024-03-26T19:13:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.689+0000 7f85791e6740 1 -- 172.21.15.97:0/3354766045 wait complete. 2024-03-26T19:13:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.689+0000 7f85791e6740 20 client.26291 trim_cache size 0 max 16384 2024-03-26T19:13:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.713+0000 7f85791e6740 1 Processor -- start 2024-03-26T19:13:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.713+0000 7f85791e6740 1 -- start start 2024-03-26T19:13:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.713+0000 7f85791e6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 0x563b4fe9fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.713+0000 7f85791e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea6950 0x563b4febb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.713+0000 7f85791e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feabac0 0x563b4fea03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f85791e6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85701af120 con 0x563b4fea5f60 2024-03-26T19:13:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f85791e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85701b0c60 con 0x563b4feabac0 2024-03-26T19:13:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f85791e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85701af7f0 con 0x563b4fea6950 2024-03-26T19:13:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea6950 0x563b4febb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea6950 0x563b4febb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:36752/0 (socket says 172.21.15.97:36752) 2024-03-26T19:13:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856ea00640 1 -- 172.21.15.97:0/2692769424 learned_addr learned my addr 172.21.15.97:0/2692769424 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856fe00640 1 --2- 172.21.15.97:0/2692769424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feabac0 0x563b4fea03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856ea00640 1 -- 172.21.15.97:0/2692769424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feabac0 msgr2=0x563b4fea03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856f400640 1 --2- 172.21.15.97:0/2692769424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 0x563b4fe9fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856ea00640 1 --2- 172.21.15.97:0/2692769424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feabac0 0x563b4fea03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856ea00640 1 -- 172.21.15.97:0/2692769424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 msgr2=0x563b4fe9fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856ea00640 1 --2- 172.21.15.97:0/2692769424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 0x563b4fe9fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856ea00640 1 -- 172.21.15.97:0/2692769424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4fea5ba0 con 0x563b4fea6950 2024-03-26T19:13:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856fe00640 1 --2- 172.21.15.97:0/2692769424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feabac0 0x563b4fea03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:13:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.714+0000 7f856ea00640 1 --2- 172.21.15.97:0/2692769424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea6950 0x563b4febb9b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f855c02c7b0 tx=0x7f855c0bd8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d12bda57e8db49 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.715+0000 7f855be00640 1 -- 172.21.15.97:0/2692769424 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0be380 con 0x563b4fea6950 2024-03-26T19:13:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.715+0000 7f855be00640 1 -- 172.21.15.97:0/2692769424 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855c0c6690 con 0x563b4fea6950 2024-03-26T19:13:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.715+0000 7f855be00640 1 -- 172.21.15.97:0/2692769424 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0c6200 con 0x563b4fea6950 2024-03-26T19:13:00.718 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:00 smithi105 ceph-mon[36270]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 608 op/s 2024-03-26T19:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.716+0000 7f85791e6740 1 -- 172.21.15.97:0/2692769424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea6950 msgr2=0x563b4febb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.716+0000 7f85791e6740 1 --2- 172.21.15.97:0/2692769424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea6950 0x563b4febb9b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f855c02c7b0 tx=0x7f855c0bd8f0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.716+0000 7f85791e6740 1 -- 172.21.15.97:0/2692769424 shutdown_connections 2024-03-26T19:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.716+0000 7f85791e6740 1 --2- 172.21.15.97:0/2692769424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feabac0 0x563b4fea03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.716+0000 7f85791e6740 1 --2- 172.21.15.97:0/2692769424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4fea6950 0x563b4febb9b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.716+0000 7f85791e6740 1 --2- 172.21.15.97:0/2692769424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 0x563b4fe9fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.716+0000 7f85791e6740 1 -- 172.21.15.97:0/2692769424 >> 172.21.15.97:0/2692769424 conn(0x563b4fea14d0 msgr2=0x563b4fea4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.716+0000 7f85791e6740 1 -- 172.21.15.97:0/2692769424 shutdown_connections 2024-03-26T19:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.716+0000 7f85791e6740 1 -- 172.21.15.97:0/2692769424 wait complete. 2024-03-26T19:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.716+0000 7f85791e6740 1 -- start start 2024-03-26T19:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.717+0000 7f85791e6740 1 Processor -- start 2024-03-26T19:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.719+0000 7f85791e6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 0x563b4feb8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.719+0000 7f85791e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4feabac0 0x563b4feb9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.719+0000 7f85791e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feb00a0 0x563b4feba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.719+0000 7f85791e6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85701b0590 con 0x563b4fea5f60 2024-03-26T19:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.719+0000 7f85791e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85701afec0 con 0x563b4feb00a0 2024-03-26T19:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.719+0000 7f85791e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85701ad100 con 0x563b4feabac0 2024-03-26T19:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.719+0000 7f85791e6740 10 client.0 _ll_register_callbacks cb 0x7f855c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.720+0000 7f856ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4feabac0 0x563b4feb9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.720+0000 7f856f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 0x563b4feb8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.720+0000 7f856fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feb00a0 0x563b4feba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.720+0000 7f856fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feb00a0 0x563b4feba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57374/0 (socket says 172.21.15.97:57374) 2024-03-26T19:13:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.720+0000 7f856fe00640 1 -- 172.21.15.97:0/1947328502 learned_addr learned my addr 172.21.15.97:0/1947328502 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.720+0000 7f856fe00640 1 -- 172.21.15.97:0/1947328502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4feabac0 msgr2=0x563b4feb9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.720+0000 7f856fe00640 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4feabac0 0x563b4feb9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.720+0000 7f856fe00640 1 -- 172.21.15.97:0/1947328502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 msgr2=0x563b4feb8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.721+0000 7f856fe00640 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 0x563b4feb8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.721+0000 7f856fe00640 1 -- 172.21.15.97:0/1947328502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4feba930 con 0x563b4feb00a0 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.721+0000 7f856f400640 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 0x563b4feb8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.722+0000 7f856fe00640 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feb00a0 0x563b4feba340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8568009f10 tx=0x7f8568009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad1c644191ac7a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.722+0000 7f8559600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.722+0000 7f8559600640 1 -- 172.21.15.97:0/1947328502 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568004e10 con 0x563b4feb00a0 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.722+0000 7f8559600640 1 -- 172.21.15.97:0/1947328502 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8568005040 con 0x563b4feb00a0 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.723+0000 7f85791e6740 1 -- 172.21.15.97:0/1947328502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f855c0c6690 con 0x563b4feb00a0 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.723+0000 7f8559600640 1 -- 172.21.15.97:0/1947328502 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8568005280 con 0x563b4feb00a0 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.723+0000 7f85791e6740 10 client.26294 fetch_fsmap learned FSMap version 27 2024-03-26T19:13:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.723+0000 7f85791e6740 1 -- 172.21.15.97:0/1947328502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b4febbc20 con 0x563b4feb00a0 2024-03-26T19:13:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.723+0000 7f8559600640 1 -- 172.21.15.97:0/1947328502 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855c0c6690 con 0x563b4feb00a0 2024-03-26T19:13:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.724+0000 7f8559600640 1 -- 172.21.15.97:0/1947328502 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f856802e030 con 0x563b4feb00a0 2024-03-26T19:13:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.724+0000 7f85791e6740 10 client.26294 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:13:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.724+0000 7f85791e6740 10 client.26294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:13:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.724+0000 7f85791e6740 20 client.26294 populate_metadata read hostname 'smithi097' 2024-03-26T19:13:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.724+0000 7f85791e6740 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x563b4ff2af40 0x563b4ff2d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.724+0000 7f85791e6740 4 client.26294 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:13:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.724+0000 7f85791e6740 1 -- 172.21.15.97:0/1947328502 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b4fe12710 con 0x563b4ff2af40 2024-03-26T19:13:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.725+0000 7f856f400640 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x563b4ff2af40 0x563b4ff2d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.726+0000 7f856f400640 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x563b4ff2af40 0x563b4ff2d360 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27e94b13eb04208 server_cookie=b879688df39bad76 in_seq=0 out_seq=0 2024-03-26T19:13:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.726+0000 7f8559600640 10 client.26294 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.743+0000 7f8559600640 1 -- 172.21.15.97:0/1947328502 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b4fe12710 con 0x563b4ff2af40 2024-03-26T19:13:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.743+0000 7f8559600640 10 client.26294 handle_command_reply: tid=0 2024-03-26T19:13:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.743+0000 7f85791e6740 1 client.26294 shutdown 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.744+0000 7f85791e6740 1 -- 172.21.15.97:0/1947328502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feb00a0 msgr2=0x563b4feba340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.744+0000 7f85791e6740 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feb00a0 0x563b4feba340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8568009f10 tx=0x7f8568009cd0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.744+0000 7f85791e6740 1 -- 172.21.15.97:0/1947328502 shutdown_connections 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.744+0000 7f85791e6740 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x563b4ff2af40 0x563b4ff2d360 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.744+0000 7f85791e6740 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b4feb00a0 0x563b4feba340 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.744+0000 7f85791e6740 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b4feabac0 0x563b4feb9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.744+0000 7f85791e6740 1 --2- 172.21.15.97:0/1947328502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b4fea5f60 0x563b4feb8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.744+0000 7f8559600640 0 client.26294 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.744+0000 7f85791e6740 1 -- 172.21.15.97:0/1947328502 >> 172.21.15.97:0/1947328502 conn(0x563b4fea14d0 msgr2=0x563b4fea73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.744+0000 7f85791e6740 1 -- 172.21.15.97:0/1947328502 shutdown_connections 2024-03-26T19:13:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.745+0000 7f85791e6740 1 -- 172.21.15.97:0/1947328502 wait complete. 2024-03-26T19:13:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.745+0000 7f85791e6740 20 client.26294 trim_cache size 0 max 16384 2024-03-26T19:13:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:13:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:13:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:13:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:13:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 -- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85540a2f90 msgr2=0x7f85540a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85540a2f90 0x7f85540a5450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x563b4fea2440 tx=0x7f856000b4a0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 -- 172.21.15.97:0/4045585628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8570104ac0 msgr2=0x7f85700707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8570104ac0 0x7f85700707c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f855c009230 tx=0x7f855c02c6c0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 -- 172.21.15.97:0/4045585628 shutdown_connections 2024-03-26T19:13:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85540a2f90 0x7f85540a5450 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85701073f0 0x7f85701a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8570104ac0 0x7f85700707c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 --2- 172.21.15.97:0/4045585628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8570102190 0x7f8570070280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 -- 172.21.15.97:0/4045585628 >> 172.21.15.97:0/4045585628 conn(0x7f85700fbba0 msgr2=0x7f8570102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 -- 172.21.15.97:0/4045585628 shutdown_connections 2024-03-26T19:13:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:00.746+0000 7f8574e00640 1 -- 172.21.15.97:0/4045585628 wait complete. 2024-03-26T19:13:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:00 smithi097 ceph-mon[31955]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 608 op/s 2024-03-26T19:13:00.882 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:00 smithi144 ceph-mon[36053]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 608 op/s 2024-03-26T19:13:02.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:02 smithi105 ceph-mon[36270]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 684 op/s 2024-03-26T19:13:02.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:02 smithi097 ceph-mon[31955]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 684 op/s 2024-03-26T19:13:02.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:02 smithi144 ceph-mon[36053]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 684 op/s 2024-03-26T19:13:04.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:04 smithi105 ceph-mon[36270]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.8 MiB/s wr, 529 op/s 2024-03-26T19:13:04.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:04 smithi097 ceph-mon[31955]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.8 MiB/s wr, 529 op/s 2024-03-26T19:13:04.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:04 smithi144 ceph-mon[36053]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.8 MiB/s wr, 529 op/s 2024-03-26T19:13:05.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 820.0 s, 1047.8 tps, lat 95.229 ms stddev 118.466 2024-03-26T19:13:06.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:06 smithi105 ceph-mon[36270]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 629 op/s 2024-03-26T19:13:06.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:06 smithi097 ceph-mon[31955]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 629 op/s 2024-03-26T19:13:06.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:06 smithi144 ceph-mon[36053]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 629 op/s 2024-03-26T19:13:08.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:08 smithi097 ceph-mon[31955]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 655 op/s 2024-03-26T19:13:08.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:08 smithi144 ceph-mon[36053]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 655 op/s 2024-03-26T19:13:08.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:08 smithi105 ceph-mon[36270]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 655 op/s 2024-03-26T19:13:10.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 825.0 s, 1449.0 tps, lat 70.195 ms stddev 101.496 2024-03-26T19:13:10.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:10 smithi097 ceph-mon[31955]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 635 op/s 2024-03-26T19:13:10.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:10 smithi144 ceph-mon[36053]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 635 op/s 2024-03-26T19:13:10.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:10 smithi105 ceph-mon[36270]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 635 op/s 2024-03-26T19:13:12.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:12 smithi097 ceph-mon[31955]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 707 op/s 2024-03-26T19:13:12.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:12 smithi144 ceph-mon[36053]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 707 op/s 2024-03-26T19:13:12.971 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:12 smithi105 ceph-mon[36270]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 707 op/s 2024-03-26T19:13:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.941+0000 7f0db7800640 1 -- 172.21.15.97:0/3580741790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db0111fc0 msgr2=0x7f0db0114450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.941+0000 7f0db7800640 1 --2- 172.21.15.97:0/3580741790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db0111fc0 0x7f0db0114450 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009be0 tx=0x7f0da802bef0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.941+0000 7f0db7800640 1 -- 172.21.15.97:0/3580741790 shutdown_connections 2024-03-26T19:13:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.941+0000 7f0db7800640 1 --2- 172.21.15.97:0/3580741790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db0111fc0 0x7f0db0114450 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.941+0000 7f0db7800640 1 --2- 172.21.15.97:0/3580741790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db006ba40 0x7f0db006bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.941+0000 7f0db7800640 1 --2- 172.21.15.97:0/3580741790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0db006d020 0x7f0db006b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.941+0000 7f0db7800640 1 -- 172.21.15.97:0/3580741790 >> 172.21.15.97:0/3580741790 conn(0x7f0db0070890 msgr2=0x7f0db0072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.941+0000 7f0db7800640 1 -- 172.21.15.97:0/3580741790 shutdown_connections 2024-03-26T19:13:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.941+0000 7f0db7800640 1 -- 172.21.15.97:0/3580741790 wait complete. 2024-03-26T19:13:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.943+0000 7f0db7800640 1 Processor -- start 2024-03-26T19:13:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.943+0000 7f0db7800640 1 -- start start 2024-03-26T19:13:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.943+0000 7f0db7800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006ba40 0x7f0db01a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.943+0000 7f0db7800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0db006d020 0x7f0db01a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0db7800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db0111fc0 0x7f0db01a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0db7800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0db0117f90 con 0x7f0db006ba40 2024-03-26T19:13:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0db7800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0db0117e10 con 0x7f0db006d020 2024-03-26T19:13:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0db7800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0db0118110 con 0x7f0db0111fc0 2024-03-26T19:13:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0db4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006ba40 0x7f0db01a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0dafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0db006d020 0x7f0db01a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0db4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006ba40 0x7f0db01a24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47638/0 (socket says 172.21.15.97:47638) 2024-03-26T19:13:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0db4e00640 1 -- 172.21.15.97:0/4048709836 learned_addr learned my addr 172.21.15.97:0/4048709836 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0db5800640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db0111fc0 0x7f0db01a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0dafe00640 1 -- 172.21.15.97:0/4048709836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db0111fc0 msgr2=0x7f0db01a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0dafe00640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db0111fc0 0x7f0db01a9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0dafe00640 1 -- 172.21.15.97:0/4048709836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006ba40 msgr2=0x7f0db01a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0dafe00640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006ba40 0x7f0db01a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0dafe00640 1 -- 172.21.15.97:0/4048709836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db01aa0d0 con 0x7f0db006d020 2024-03-26T19:13:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.944+0000 7f0db5800640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db0111fc0 0x7f0db01a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:13:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.945+0000 7f0dafe00640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0db006d020 0x7f0db01a29f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d6d0 tx=0x7f0d9c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.945+0000 7f0dad600640 1 -- 172.21.15.97:0/4048709836 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c017070 con 0x7f0db006d020 2024-03-26T19:13:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.945+0000 7f0dad600640 1 -- 172.21.15.97:0/4048709836 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9c009820 con 0x7f0db006d020 2024-03-26T19:13:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.945+0000 7f0dad600640 1 -- 172.21.15.97:0/4048709836 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c004d00 con 0x7f0db006d020 2024-03-26T19:13:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.945+0000 7f0db7800640 1 -- 172.21.15.97:0/4048709836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db01aa3c0 con 0x7f0db006d020 2024-03-26T19:13:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.946+0000 7f0db7800640 1 -- 172.21.15.97:0/4048709836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db01aa8c0 con 0x7f0db006d020 2024-03-26T19:13:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0dba607740 1 Processor -- start 2024-03-26T19:13:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0dba607740 1 -- start start 2024-03-26T19:13:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0dba607740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x5613615279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0dba607740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561361527fa0 0x56136152caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0dba607740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56136152d110 0x56136152f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0dba607740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561361483e80 con 0x561361527fa0 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0dba607740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56136142c660 con 0x56136152d110 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0dba607740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56136147fe30 con 0x5613615275b0 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0db5800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x5613615279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0db5800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x5613615279d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:40992/0 (socket says 172.21.15.97:40992) 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0db5800640 1 -- 172.21.15.97:0/2002457028 learned_addr learned my addr 172.21.15.97:0/2002457028 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.952+0000 7f0db4e00640 1 --2- 172.21.15.97:0/2002457028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561361527fa0 0x56136152caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.953+0000 7f0dad600640 1 -- 172.21.15.97:0/4048709836 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f0d9c0099d0 con 0x7f0db006d020 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.953+0000 7f0db5800640 1 -- 172.21.15.97:0/2002457028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56136152d110 msgr2=0x56136152f5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.953+0000 7f0db5800640 1 --2- 172.21.15.97:0/2002457028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56136152d110 0x56136152f5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.953+0000 7f0db5800640 1 -- 172.21.15.97:0/2002457028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561361527fa0 msgr2=0x56136152caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.953+0000 7f0db5800640 1 --2- 172.21.15.97:0/2002457028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561361527fa0 0x56136152caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.953+0000 7f0db5800640 1 -- 172.21.15.97:0/2002457028 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561361198210 con 0x5613615275b0 2024-03-26T19:13:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.953+0000 7f0db5800640 1 --2- 172.21.15.97:0/2002457028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x5613615279d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0da8034fd0 tx=0x7f0da8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c30de4627fa648 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.954+0000 7f0da2000640 1 -- 172.21.15.97:0/2002457028 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8043070 con 0x5613615275b0 2024-03-26T19:13:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.954+0000 7f0da2000640 1 -- 172.21.15.97:0/2002457028 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da8033bc0 con 0x5613615275b0 2024-03-26T19:13:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.954+0000 7f0da2000640 1 -- 172.21.15.97:0/2002457028 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da802cb80 con 0x5613615275b0 2024-03-26T19:13:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dba607740 1 -- 172.21.15.97:0/2002457028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 msgr2=0x5613615279d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dba607740 1 --2- 172.21.15.97:0/2002457028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x5613615279d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0da8034fd0 tx=0x7f0da8009840 comp rx=0 tx=0).stop 2024-03-26T19:13:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dad600640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d980a3040 0x7f0d980a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dad600640 1 -- 172.21.15.97:0/4048709836 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0d9c0cec70 con 0x7f0db006d020 2024-03-26T19:13:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dba607740 1 -- 172.21.15.97:0/2002457028 shutdown_connections 2024-03-26T19:13:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dba607740 1 --2- 172.21.15.97:0/2002457028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56136152d110 0x56136152f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dba607740 1 --2- 172.21.15.97:0/2002457028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561361527fa0 0x56136152caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dba607740 1 --2- 172.21.15.97:0/2002457028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x5613615279d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dba607740 1 -- 172.21.15.97:0/2002457028 >> 172.21.15.97:0/2002457028 conn(0x561361522b20 msgr2=0x561361522f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dba607740 1 -- 172.21.15.97:0/2002457028 shutdown_connections 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0db5800640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d980a3040 0x7f0d980a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.955+0000 7f0dba607740 1 -- 172.21.15.97:0/2002457028 wait complete. 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.956+0000 7f0dba607740 1 -- start start 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.956+0000 7f0dba607740 1 Processor -- start 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.956+0000 7f0db5800640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d980a3040 0x7f0d980a5500 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009b30 tx=0x7f0da802bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.957+0000 7f0dba607740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613615275b0 0x5613615315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.957+0000 7f0dba607740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561361527fa0 0x561361531b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.957+0000 7f0dba607740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 0x561361538b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.957+0000 7f0dba607740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561361530060 con 0x561361527fa0 2024-03-26T19:13:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.957+0000 7f0dba607740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56136143fac0 con 0x5613615275b0 2024-03-26T19:13:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.957+0000 7f0dba607740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5613615301e0 con 0x56136152d110 2024-03-26T19:13:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.957+0000 7f0dba607740 10 client.0 _ll_register_callbacks cb 0x56136143f490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:13:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0db4e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613615275b0 0x5613615315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0dafe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561361527fa0 0x561361531b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0db4e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613615275b0 0x5613615315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:43228/0 (socket says 172.21.15.97:43228) 2024-03-26T19:13:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0db4e00640 1 -- 172.21.15.97:0/3295123441 learned_addr learned my addr 172.21.15.97:0/3295123441 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0db5800640 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 0x561361538b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0db4e00640 1 -- 172.21.15.97:0/3295123441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 msgr2=0x561361538b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0db4e00640 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 0x561361538b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0db4e00640 1 -- 172.21.15.97:0/3295123441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561361527fa0 msgr2=0x561361531b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0db4e00640 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561361527fa0 0x561361531b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0db4e00640 1 -- 172.21.15.97:0/3295123441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561361264620 con 0x5613615275b0 2024-03-26T19:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.958+0000 7f0db4e00640 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613615275b0 0x5613615315d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0da4012700 tx=0x7f0da4012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d132ec4f87b15e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.959+0000 7f0d96a00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.959+0000 7f0d96a00640 1 -- 172.21.15.97:0/3295123441 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4004030 con 0x5613615275b0 2024-03-26T19:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.959+0000 7f0d96a00640 1 -- 172.21.15.97:0/3295123441 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0da40041d0 con 0x5613615275b0 2024-03-26T19:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.959+0000 7f0dba607740 1 -- 172.21.15.97:0/3295123441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da8033bc0 con 0x5613615275b0 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.959+0000 7f0d96a00640 1 -- 172.21.15.97:0/3295123441 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0da4004410 con 0x5613615275b0 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.959+0000 7f0dba607740 10 client.26303 fetch_fsmap learned FSMap version 27 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.959+0000 7f0dba607740 1 -- 172.21.15.97:0/3295123441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56136153bf30 con 0x5613615275b0 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.959+0000 7f0d96a00640 1 -- 172.21.15.97:0/3295123441 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da8033bc0 con 0x5613615275b0 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.960+0000 7f0d96a00640 1 -- 172.21.15.97:0/3295123441 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0da4003cc0 con 0x5613615275b0 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.960+0000 7f0dba607740 10 client.26303 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.960+0000 7f0dba607740 10 client.26303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.960+0000 7f0dba607740 20 client.26303 populate_metadata read hostname 'smithi097' 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.960+0000 7f0dba607740 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56136153d600 0x56136153fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.960+0000 7f0dba607740 4 client.26303 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.960+0000 7f0dba607740 1 -- 172.21.15.97:0/3295123441 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613614913c0 con 0x56136153d600 2024-03-26T19:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.960+0000 7f0dafe00640 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56136153d600 0x56136153fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.961+0000 7f0d96a00640 10 client.26303 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.961+0000 7f0dafe00640 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56136153d600 0x56136153fa30 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f17823e24721386 server_cookie=7ced80dc2599dda1 in_seq=0 out_seq=0 2024-03-26T19:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.962+0000 7f0d96a00640 1 -- 172.21.15.97:0/3295123441 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5613614913c0 con 0x56136153d600 2024-03-26T19:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.962+0000 7f0d96a00640 10 client.26303 handle_command_reply: tid=0 2024-03-26T19:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.962+0000 7f0dba607740 1 client.26303 shutdown 2024-03-26T19:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.962+0000 7f0dba607740 1 -- 172.21.15.97:0/3295123441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613615275b0 msgr2=0x5613615315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.962+0000 7f0dba607740 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613615275b0 0x5613615315d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0da4012700 tx=0x7f0da4012bc0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.963+0000 7f0dba607740 1 -- 172.21.15.97:0/3295123441 shutdown_connections 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.963+0000 7f0dba607740 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x56136153d600 0x56136153fa30 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.963+0000 7f0dba607740 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 0x561361538b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.963+0000 7f0d96a00640 0 client.26303 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.963+0000 7f0dba607740 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561361527fa0 0x561361531b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.963+0000 7f0dba607740 1 --2- 172.21.15.97:0/3295123441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613615275b0 0x5613615315d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.963+0000 7f0dba607740 1 -- 172.21.15.97:0/3295123441 >> 172.21.15.97:0/3295123441 conn(0x561361522b20 msgr2=0x561361519540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.963+0000 7f0dba607740 1 -- 172.21.15.97:0/3295123441 shutdown_connections 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.963+0000 7f0dba607740 1 -- 172.21.15.97:0/3295123441 wait complete. 2024-03-26T19:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.963+0000 7f0dba607740 20 client.26303 trim_cache size 0 max 16384 2024-03-26T19:13:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.986+0000 7f0dba607740 1 Processor -- start 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.986+0000 7f0dba607740 1 -- start start 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0dba607740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x561361528830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0dba607740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361538b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0dba607740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56136152d110 0x561361528d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0dba607740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0db01af2c0 con 0x56136152d110 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0dba607740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0db01b0e00 con 0x561361527fa0 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0dba607740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0db01af990 con 0x5613615275b0 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0db4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x561361528830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0dafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361538b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0db4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x561361528830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:41016/0 (socket says 172.21.15.97:41016) 2024-03-26T19:13:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0db4e00640 1 -- 172.21.15.97:0/2094079450 learned_addr learned my addr 172.21.15.97:0/2094079450 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0db5800640 1 --2- 172.21.15.97:0/2094079450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56136152d110 0x561361528d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0db4e00640 1 -- 172.21.15.97:0/2094079450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 msgr2=0x561361538b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0db4e00640 1 --2- 172.21.15.97:0/2094079450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361538b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0db4e00640 1 -- 172.21.15.97:0/2094079450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56136152d110 msgr2=0x561361528d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0db4e00640 1 --2- 172.21.15.97:0/2094079450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56136152d110 0x561361528d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0db4e00640 1 -- 172.21.15.97:0/2094079450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56136152c2d0 con 0x5613615275b0 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.987+0000 7f0dafe00640 1 --2- 172.21.15.97:0/2094079450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361538b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.988+0000 7f0db4e00640 1 --2- 172.21.15.97:0/2094079450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x561361528830 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0da4001eb0 tx=0x7f0da40099d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad49e2f0e6b378a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.988+0000 7f0da1600640 1 -- 172.21.15.97:0/2094079450 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da400f9a0 con 0x5613615275b0 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.988+0000 7f0da1600640 1 -- 172.21.15.97:0/2094079450 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da4005510 con 0x5613615275b0 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.988+0000 7f0da1600640 1 -- 172.21.15.97:0/2094079450 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4010e30 con 0x5613615275b0 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.989+0000 7f0dba607740 1 -- 172.21.15.97:0/2094079450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 msgr2=0x561361528830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.989+0000 7f0dba607740 1 --2- 172.21.15.97:0/2094079450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x561361528830 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0da4001eb0 tx=0x7f0da40099d0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.989+0000 7f0dba607740 1 -- 172.21.15.97:0/2094079450 shutdown_connections 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.989+0000 7f0dba607740 1 --2- 172.21.15.97:0/2094079450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56136152d110 0x561361528d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.989+0000 7f0dba607740 1 --2- 172.21.15.97:0/2094079450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361538b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.989+0000 7f0dba607740 1 --2- 172.21.15.97:0/2094079450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613615275b0 0x561361528830 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.989+0000 7f0dba607740 1 -- 172.21.15.97:0/2094079450 >> 172.21.15.97:0/2094079450 conn(0x561361522b20 msgr2=0x561361526610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.989+0000 7f0dba607740 1 -- 172.21.15.97:0/2094079450 shutdown_connections 2024-03-26T19:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.989+0000 7f0dba607740 1 -- 172.21.15.97:0/2094079450 wait complete. 2024-03-26T19:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.990+0000 7f0dba607740 1 -- start start 2024-03-26T19:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.990+0000 7f0dba607740 1 Processor -- start 2024-03-26T19:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dba607740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613615275b0 0x561361535d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dba607740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361536260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dba607740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 0x561361537270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dba607740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0db01b0730 con 0x5613615275b0 2024-03-26T19:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dba607740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0db01b0060 con 0x561361527fa0 2024-03-26T19:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dba607740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0db01ad2c0 con 0x56136152d110 2024-03-26T19:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dba607740 10 client.0 _ll_register_callbacks cb 0x7f0da4009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0db4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613615275b0 0x561361535d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0db5800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 0x561361537270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361536260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0db5800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 0x561361537270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:41026/0 (socket says 172.21.15.97:41026) 2024-03-26T19:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0db4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613615275b0 0x561361535d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47682/0 (socket says 172.21.15.97:47682) 2024-03-26T19:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0db5800640 1 -- 172.21.15.97:0/445794919 learned_addr learned my addr 172.21.15.97:0/445794919 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361536260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:43260/0 (socket says 172.21.15.97:43260) 2024-03-26T19:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dafe00640 1 -- 172.21.15.97:0/445794919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 msgr2=0x561361537270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dafe00640 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 0x561361537270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dafe00640 1 -- 172.21.15.97:0/445794919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613615275b0 msgr2=0x561361535d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dafe00640 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613615275b0 0x561361535d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0dafe00640 1 -- 172.21.15.97:0/445794919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561361537860 con 0x561361527fa0 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.991+0000 7f0db5800640 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 0x561361537270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.992+0000 7f0dafe00640 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361536260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d6a0 tx=0x7f0d9c097040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdc40d6aad8d785e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.992+0000 7f0d96a00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.992+0000 7f0d96a00640 1 -- 172.21.15.97:0/445794919 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c020040 con 0x561361527fa0 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.992+0000 7f0d96a00640 1 -- 172.21.15.97:0/445794919 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0d9c009d40 con 0x561361527fa0 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.992+0000 7f0dba607740 1 -- 172.21.15.97:0/445794919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da4005510 con 0x561361527fa0 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.992+0000 7f0d96a00640 1 -- 172.21.15.97:0/445794919 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d9c0a3de0 con 0x561361527fa0 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.992+0000 7f0dba607740 10 client.26309 fetch_fsmap learned FSMap version 27 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.992+0000 7f0dba607740 1 -- 172.21.15.97:0/445794919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561361538be0 con 0x561361527fa0 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.993+0000 7f0d96a00640 1 -- 172.21.15.97:0/445794919 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da4005510 con 0x561361527fa0 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.993+0000 7f0d96a00640 1 -- 172.21.15.97:0/445794919 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0d9c0a3950 con 0x561361527fa0 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.993+0000 7f0dba607740 10 client.26309 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.993+0000 7f0dba607740 10 client.26309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:13:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.993+0000 7f0dba607740 20 client.26309 populate_metadata read hostname 'smithi097' 2024-03-26T19:13:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.993+0000 7f0dba607740 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613615a3dd0 0x5613615a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.993+0000 7f0dba607740 4 client.26309 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:13:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.993+0000 7f0dba607740 1 -- 172.21.15.97:0/445794919 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56136148b710 con 0x5613615a3dd0 2024-03-26T19:13:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.993+0000 7f0db4e00640 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613615a3dd0 0x5613615a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.994+0000 7f0db4e00640 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613615a3dd0 0x5613615a61f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f61b3471132abd server_cookie=95b675eba517fdaf in_seq=0 out_seq=0 2024-03-26T19:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:13.994+0000 7f0d96a00640 10 client.26309 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.030+0000 7f0d96a00640 1 -- 172.21.15.97:0/445794919 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56136148b710 con 0x5613615a3dd0 2024-03-26T19:13:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0d96a00640 10 client.26309 handle_command_reply: tid=0 2024-03-26T19:13:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0dba607740 1 client.26309 shutdown 2024-03-26T19:13:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0dba607740 1 -- 172.21.15.97:0/445794919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 msgr2=0x561361536260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0dba607740 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361536260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d6a0 tx=0x7f0d9c097040 comp rx=0 tx=0).stop 2024-03-26T19:13:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0dba607740 1 -- 172.21.15.97:0/445794919 shutdown_connections 2024-03-26T19:13:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0dba607740 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x5613615a3dd0 0x5613615a61f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0dba607740 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56136152d110 0x561361537270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0d96a00640 0 client.26309 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0dba607740 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561361527fa0 0x561361536260 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0dba607740 1 --2- 172.21.15.97:0/445794919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613615275b0 0x561361535d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.031+0000 7f0dba607740 1 -- 172.21.15.97:0/445794919 >> 172.21.15.97:0/445794919 conn(0x561361522b20 msgr2=0x561361518cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.032+0000 7f0dba607740 1 -- 172.21.15.97:0/445794919 shutdown_connections 2024-03-26T19:13:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.032+0000 7f0dba607740 1 -- 172.21.15.97:0/445794919 wait complete. 2024-03-26T19:13:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.032+0000 7f0dba607740 20 client.26309 trim_cache size 0 max 16384 2024-03-26T19:13:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:13:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:13:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:13:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:13:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.033+0000 7f0db7800640 1 -- 172.21.15.97:0/4048709836 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d980a3040 msgr2=0x7f0d980a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.033+0000 7f0db7800640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d980a3040 0x7f0d980a5500 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009b30 tx=0x7f0da802bef0 comp rx=0 tx=0).stop 2024-03-26T19:13:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.033+0000 7f0db7800640 1 -- 172.21.15.97:0/4048709836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0db006d020 msgr2=0x7f0db01a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.033+0000 7f0db7800640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0db006d020 0x7f0db01a29f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d6d0 tx=0x7f0d9c00db90 comp rx=0 tx=0).stop 2024-03-26T19:13:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.034+0000 7f0db7800640 1 -- 172.21.15.97:0/4048709836 shutdown_connections 2024-03-26T19:13:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.034+0000 7f0db7800640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0d980a3040 0x7f0d980a5500 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.034+0000 7f0db7800640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db0111fc0 0x7f0db01a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.034+0000 7f0db7800640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0db006d020 0x7f0db01a29f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.034+0000 7f0db7800640 1 --2- 172.21.15.97:0/4048709836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0db006ba40 0x7f0db01a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.034+0000 7f0db7800640 1 -- 172.21.15.97:0/4048709836 >> 172.21.15.97:0/4048709836 conn(0x7f0db0070890 msgr2=0x7f0db010dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.035+0000 7f0db7800640 1 -- 172.21.15.97:0/4048709836 shutdown_connections 2024-03-26T19:13:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:14.035+0000 7f0db7800640 1 -- 172.21.15.97:0/4048709836 wait complete. 2024-03-26T19:13:14.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:14 smithi097 ceph-mon[31955]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.3 MiB/s wr, 570 op/s 2024-03-26T19:13:14.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:14 smithi144 ceph-mon[36053]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.3 MiB/s wr, 570 op/s 2024-03-26T19:13:14.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:14 smithi105 ceph-mon[36270]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.3 MiB/s wr, 570 op/s 2024-03-26T19:13:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 830.0 s, 1120.2 tps, lat 89.710 ms stddev 128.087 2024-03-26T19:13:16.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:16 smithi097 ceph-mon[31955]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 685 op/s 2024-03-26T19:13:16.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:16 smithi144 ceph-mon[36053]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 685 op/s 2024-03-26T19:13:16.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:16 smithi105 ceph-mon[36270]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 685 op/s 2024-03-26T19:13:18.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:18 smithi097 ceph-mon[31955]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 730 op/s 2024-03-26T19:13:18.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:18 smithi144 ceph-mon[36053]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 730 op/s 2024-03-26T19:13:18.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:18 smithi105 ceph-mon[36270]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 730 op/s 2024-03-26T19:13:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 835.0 s, 1181.4 tps, lat 84.452 ms stddev 98.789 2024-03-26T19:13:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:20 smithi097 ceph-mon[31955]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 682 op/s 2024-03-26T19:13:20.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:20 smithi144 ceph-mon[36053]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 682 op/s 2024-03-26T19:13:20.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:20 smithi105 ceph-mon[36270]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 682 op/s 2024-03-26T19:13:22.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:22 smithi097 ceph-mon[31955]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 746 op/s 2024-03-26T19:13:22.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:22 smithi144 ceph-mon[36053]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 746 op/s 2024-03-26T19:13:22.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:22 smithi105 ceph-mon[36270]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 746 op/s 2024-03-26T19:13:24.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:24 smithi097 ceph-mon[31955]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.2 MiB/s wr, 603 op/s 2024-03-26T19:13:24.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:24 smithi144 ceph-mon[36053]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.2 MiB/s wr, 603 op/s 2024-03-26T19:13:24.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:24 smithi105 ceph-mon[36270]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.2 MiB/s wr, 603 op/s 2024-03-26T19:13:25.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 840.0 s, 1334.6 tps, lat 73.695 ms stddev 88.214 2024-03-26T19:13:26.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:26 smithi097 ceph-mon[31955]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.0 MiB/s wr, 681 op/s 2024-03-26T19:13:26.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:26 smithi144 ceph-mon[36053]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.0 MiB/s wr, 681 op/s 2024-03-26T19:13:26.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:26 smithi105 ceph-mon[36270]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.0 MiB/s wr, 681 op/s 2024-03-26T19:13:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.226+0000 7f7286a00640 1 -- 172.21.15.97:0/377950525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7280102360 msgr2=0x7f7280104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.226+0000 7f7286a00640 1 --2- 172.21.15.97:0/377950525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7280102360 0x7f7280104750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7278009ca0 tx=0x7f727802c0f0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.226+0000 7f7286a00640 1 -- 172.21.15.97:0/377950525 shutdown_connections 2024-03-26T19:13:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.226+0000 7f7286a00640 1 --2- 172.21.15.97:0/377950525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72801075c0 0x7f7280109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.226+0000 7f7286a00640 1 --2- 172.21.15.97:0/377950525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7280104c90 0x7f7280107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.226+0000 7f7286a00640 1 --2- 172.21.15.97:0/377950525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7280102360 0x7f7280104750 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.226+0000 7f7286a00640 1 -- 172.21.15.97:0/377950525 >> 172.21.15.97:0/377950525 conn(0x7f72800fbd50 msgr2=0x7f72800fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.227+0000 7f7286a00640 1 -- 172.21.15.97:0/377950525 shutdown_connections 2024-03-26T19:13:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.227+0000 7f7286a00640 1 -- 172.21.15.97:0/377950525 wait complete. 2024-03-26T19:13:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.229+0000 7f7286a00640 1 Processor -- start 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.229+0000 7f7286a00640 1 -- start start 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f7286a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7280102360 0x7f7280195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f7286a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7280104c90 0x7f7280195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f7286a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72801075c0 0x7f72801a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f7286a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f728010d580 con 0x7f7280102360 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f7286a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f728010d400 con 0x7f7280104c90 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f7286a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f728010d700 con 0x7f72801075c0 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f7284a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72801075c0 0x7f72801a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f727f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7280104c90 0x7f7280195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f7284a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72801075c0 0x7f72801a58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:59882/0 (socket says 172.21.15.97:59882) 2024-03-26T19:13:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f727f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7280104c90 0x7f7280195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46210/0 (socket says 172.21.15.97:46210) 2024-03-26T19:13:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.230+0000 7f7284a00640 1 -- 172.21.15.97:0/941414642 learned_addr learned my addr 172.21.15.97:0/941414642 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.231+0000 7f727fe00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7280102360 0x7f7280195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.231+0000 7f7284a00640 1 -- 172.21.15.97:0/941414642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7280104c90 msgr2=0x7f7280195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.231+0000 7f7284a00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7280104c90 0x7f7280195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.231+0000 7f7284a00640 1 -- 172.21.15.97:0/941414642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7280102360 msgr2=0x7f7280195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.231+0000 7f7284a00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7280102360 0x7f7280195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.231+0000 7f7284a00640 1 -- 172.21.15.97:0/941414642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728006b2e0 con 0x7f72801075c0 2024-03-26T19:13:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.231+0000 7f7284a00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72801075c0 0x7f72801a58b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f727000e970 tx=0x7f727000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.231+0000 7f727cc00640 1 -- 172.21.15.97:0/941414642 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727000ccc0 con 0x7f72801075c0 2024-03-26T19:13:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.232+0000 7f7286a00640 1 -- 172.21.15.97:0/941414642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728006b5d0 con 0x7f72801075c0 2024-03-26T19:13:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.232+0000 7f727cc00640 1 -- 172.21.15.97:0/941414642 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f727000ce60 con 0x7f72801075c0 2024-03-26T19:13:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.232+0000 7f727cc00640 1 -- 172.21.15.97:0/941414642 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270010490 con 0x7f72801075c0 2024-03-26T19:13:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.232+0000 7f7286a00640 1 -- 172.21.15.97:0/941414642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728006bb20 con 0x7f72801075c0 2024-03-26T19:13:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.237+0000 7f728961d740 1 Processor -- start 2024-03-26T19:13:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.237+0000 7f727cc00640 1 -- 172.21.15.97:0/941414642 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f7270005000 con 0x7f72801075c0 2024-03-26T19:13:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.237+0000 7f728961d740 1 -- start start 2024-03-26T19:13:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f728961d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f728961d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3e8790 0x55c2fe3ed290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f728961d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3ed900 0x55c2fe3efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f728961d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c2fe34ce80 con 0x55c2fe3e8790 2024-03-26T19:13:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f728961d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c2fe2f5660 con 0x55c2fe3e7da0 2024-03-26T19:13:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f728961d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c2fe348e30 con 0x55c2fe3ed900 2024-03-26T19:13:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f727f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f727f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3e81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46226/0 (socket says 172.21.15.97:46226) 2024-03-26T19:13:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f727f400640 1 -- 172.21.15.97:0/550295939 learned_addr learned my addr 172.21.15.97:0/550295939 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f727f400640 1 --2- 172.21.15.97:0/550295939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3ed900 0x55c2fe3efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.238+0000 7f727fe00640 1 --2- 172.21.15.97:0/550295939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3e8790 0x55c2fe3ed290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.239+0000 7f727f400640 1 -- 172.21.15.97:0/550295939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3ed900 msgr2=0x55c2fe3efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.239+0000 7f727f400640 1 --2- 172.21.15.97:0/550295939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3ed900 0x55c2fe3efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.239+0000 7f727f400640 1 -- 172.21.15.97:0/550295939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3e8790 msgr2=0x55c2fe3ed290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.239+0000 7f727f400640 1 --2- 172.21.15.97:0/550295939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3e8790 0x55c2fe3ed290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.239+0000 7f727f400640 1 -- 172.21.15.97:0/550295939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2fe061210 con 0x55c2fe3e7da0 2024-03-26T19:13:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.240+0000 7f727cc00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f72640a2fc0 0x7f72640a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.240+0000 7f727f400640 1 --2- 172.21.15.97:0/550295939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3e81c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f726c012770 tx=0x7f726c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503de9c060846c0d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.241+0000 7f7275200640 1 -- 172.21.15.97:0/550295939 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c012cf0 con 0x55c2fe3e7da0 2024-03-26T19:13:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.241+0000 7f7275200640 1 -- 172.21.15.97:0/550295939 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f726c010770 con 0x55c2fe3e7da0 2024-03-26T19:13:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.241+0000 7f7275200640 1 -- 172.21.15.97:0/550295939 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c003850 con 0x55c2fe3e7da0 2024-03-26T19:13:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.241+0000 7f727cc00640 1 -- 172.21.15.97:0/941414642 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f72700cf8d0 con 0x7f72801075c0 2024-03-26T19:13:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.242+0000 7f727fe00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f72640a2fc0 0x7f72640a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.243+0000 7f728961d740 1 -- 172.21.15.97:0/550295939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 msgr2=0x55c2fe3e81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.243+0000 7f728961d740 1 --2- 172.21.15.97:0/550295939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3e81c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f726c012770 tx=0x7f726c004730 comp rx=0 tx=0).stop 2024-03-26T19:13:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.243+0000 7f727fe00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f72640a2fc0 0x7f72640a5480 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f727802c700 tx=0x7f7278005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.243+0000 7f728961d740 1 -- 172.21.15.97:0/550295939 shutdown_connections 2024-03-26T19:13:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.243+0000 7f728961d740 1 --2- 172.21.15.97:0/550295939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3ed900 0x55c2fe3efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.243+0000 7f728961d740 1 --2- 172.21.15.97:0/550295939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3e8790 0x55c2fe3ed290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.243+0000 7f728961d740 1 --2- 172.21.15.97:0/550295939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3e81c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.243+0000 7f728961d740 1 -- 172.21.15.97:0/550295939 >> 172.21.15.97:0/550295939 conn(0x55c2fe3e3310 msgr2=0x55c2fe3e3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.243+0000 7f728961d740 1 -- 172.21.15.97:0/550295939 shutdown_connections 2024-03-26T19:13:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.244+0000 7f728961d740 1 -- 172.21.15.97:0/550295939 wait complete. 2024-03-26T19:13:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.244+0000 7f728961d740 1 -- start start 2024-03-26T19:13:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.244+0000 7f728961d740 1 Processor -- start 2024-03-26T19:13:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.245+0000 7f728961d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.245+0000 7f728961d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.245+0000 7f728961d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3ed900 0x55c2fe3f9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.245+0000 7f728961d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c2fe3f08b0 con 0x55c2fe3e7da0 2024-03-26T19:13:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f7284a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3ed900 0x55c2fe3f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f728961d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c2fe308ac0 con 0x55c2fe3ed900 2024-03-26T19:13:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f728961d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c2fe3f0a30 con 0x55c2fe3e8790 2024-03-26T19:13:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f7284a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3ed900 0x55c2fe3f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46230/0 (socket says 172.21.15.97:46230) 2024-03-26T19:13:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f7284a00640 1 -- 172.21.15.97:0/3677004924 learned_addr learned my addr 172.21.15.97:0/3677004924 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f728961d740 10 client.0 _ll_register_callbacks cb 0x55c2fe308490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:13:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f727fe00640 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f7284a00640 1 -- 172.21.15.97:0/3677004924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e8790 msgr2=0x55c2fe3f23c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:13:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f7284a00640 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f23c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f7284a00640 1 -- 172.21.15.97:0/3677004924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3e7da0 msgr2=0x55c2fe3f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f7284a00640 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.246+0000 7f7284a00640 1 -- 172.21.15.97:0/3677004924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2fe12d620 con 0x55c2fe3ed900 2024-03-26T19:13:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.247+0000 7f7284a00640 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3ed900 0x55c2fe3f9390 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7270098a00 tx=0x7f727009c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a9e543696f2d74 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.247+0000 7f7269600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:13:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.247+0000 7f7269600640 1 -- 172.21.15.97:0/3677004924 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72700052f0 con 0x55c2fe3ed900 2024-03-26T19:13:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.247+0000 7f7269600640 1 -- 172.21.15.97:0/3677004924 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f72700bc070 con 0x55c2fe3ed900 2024-03-26T19:13:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.248+0000 7f728961d740 1 -- 172.21.15.97:0/3677004924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f726c010770 con 0x55c2fe3ed900 2024-03-26T19:13:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.248+0000 7f7269600640 1 -- 172.21.15.97:0/3677004924 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7270005490 con 0x55c2fe3ed900 2024-03-26T19:13:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.248+0000 7f728961d740 10 client.26321 fetch_fsmap learned FSMap version 27 2024-03-26T19:13:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.248+0000 7f728961d740 1 -- 172.21.15.97:0/3677004924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2fe3fc770 con 0x55c2fe3ed900 2024-03-26T19:13:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.249+0000 7f7269600640 1 -- 172.21.15.97:0/3677004924 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f726c010770 con 0x55c2fe3ed900 2024-03-26T19:13:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.249+0000 7f7269600640 1 -- 172.21.15.97:0/3677004924 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f727009b510 con 0x55c2fe3ed900 2024-03-26T19:13:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.249+0000 7f728961d740 10 client.26321 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:13:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.249+0000 7f728961d740 10 client.26321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:13:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.249+0000 7f728961d740 20 client.26321 populate_metadata read hostname 'smithi097' 2024-03-26T19:13:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.249+0000 7f728961d740 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c2fe3fde00 0x55c2fe400250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.249+0000 7f728961d740 4 client.26321 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:13:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.249+0000 7f728961d740 1 -- 172.21.15.97:0/3677004924 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2fe35a3c0 con 0x55c2fe3fde00 2024-03-26T19:13:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.250+0000 7f727fe00640 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c2fe3fde00 0x55c2fe400250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.250+0000 7f727fe00640 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c2fe3fde00 0x55c2fe400250 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54cf4f7299beb25 server_cookie=9ddf390ddcbb1587 in_seq=0 out_seq=0 2024-03-26T19:13:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.251+0000 7f7269600640 10 client.26321 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.252+0000 7f7269600640 1 -- 172.21.15.97:0/3677004924 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c2fe35a3c0 con 0x55c2fe3fde00 2024-03-26T19:13:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.252+0000 7f7269600640 10 client.26321 handle_command_reply: tid=0 2024-03-26T19:13:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.252+0000 7f728961d740 1 client.26321 shutdown 2024-03-26T19:13:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.253+0000 7f728961d740 1 -- 172.21.15.97:0/3677004924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3ed900 msgr2=0x55c2fe3f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.253+0000 7f728961d740 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3ed900 0x55c2fe3f9390 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7270098a00 tx=0x7f727009c870 comp rx=0 tx=0).stop 2024-03-26T19:13:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.253+0000 7f728961d740 1 -- 172.21.15.97:0/3677004924 shutdown_connections 2024-03-26T19:13:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.253+0000 7f728961d740 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c2fe3fde00 0x55c2fe400250 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.253+0000 7f728961d740 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3ed900 0x55c2fe3f9390 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.253+0000 7f728961d740 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.253+0000 7f728961d740 1 --2- 172.21.15.97:0/3677004924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.253+0000 7f728961d740 1 -- 172.21.15.97:0/3677004924 >> 172.21.15.97:0/3677004924 conn(0x55c2fe3e3310 msgr2=0x55c2fe3e2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.254+0000 7f7269600640 0 client.26321 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.254+0000 7f728961d740 1 -- 172.21.15.97:0/3677004924 shutdown_connections 2024-03-26T19:13:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.254+0000 7f728961d740 1 -- 172.21.15.97:0/3677004924 wait complete. 2024-03-26T19:13:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.254+0000 7f728961d740 20 client.26321 trim_cache size 0 max 16384 2024-03-26T19:13:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.276+0000 7f728961d740 1 Processor -- start 2024-03-26T19:13:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f728961d740 1 -- start start 2024-03-26T19:13:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f728961d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f728961d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f728961d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3ed900 0x55c2fe3e9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f728961d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b7c70 con 0x55c2fe3ed900 2024-03-26T19:13:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f728961d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b97b0 con 0x55c2fe3e8790 2024-03-26T19:13:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f728961d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b8340 con 0x55c2fe3e7da0 2024-03-26T19:13:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f727f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f727f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46232/0 (socket says 172.21.15.97:46232) 2024-03-26T19:13:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f727f400640 1 -- 172.21.15.97:0/4247000754 learned_addr learned my addr 172.21.15.97:0/4247000754 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f727fe00640 1 --2- 172.21.15.97:0/4247000754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.277+0000 7f7284a00640 1 --2- 172.21.15.97:0/4247000754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3ed900 0x55c2fe3e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.278+0000 7f727f400640 1 -- 172.21.15.97:0/4247000754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e7da0 msgr2=0x55c2fe3e6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.278+0000 7f727f400640 1 --2- 172.21.15.97:0/4247000754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3e6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.278+0000 7f727f400640 1 -- 172.21.15.97:0/4247000754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3ed900 msgr2=0x55c2fe3e9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.278+0000 7f727f400640 1 --2- 172.21.15.97:0/4247000754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3ed900 0x55c2fe3e9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.278+0000 7f727f400640 1 -- 172.21.15.97:0/4247000754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2fe3e2eb0 con 0x55c2fe3e8790 2024-03-26T19:13:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.278+0000 7f727f400640 1 --2- 172.21.15.97:0/4247000754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f9350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f726c00bfb0 tx=0x7f726c00bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b8f1820f0b7a386 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.278+0000 7f726be00640 1 -- 172.21.15.97:0/4247000754 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c024070 con 0x55c2fe3e8790 2024-03-26T19:13:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.278+0000 7f726be00640 1 -- 172.21.15.97:0/4247000754 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f726c016040 con 0x55c2fe3e8790 2024-03-26T19:13:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.278+0000 7f726be00640 1 -- 172.21.15.97:0/4247000754 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c004a00 con 0x55c2fe3e8790 2024-03-26T19:13:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.279+0000 7f728961d740 1 -- 172.21.15.97:0/4247000754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e8790 msgr2=0x55c2fe3f9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.279+0000 7f728961d740 1 --2- 172.21.15.97:0/4247000754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f9350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f726c00bfb0 tx=0x7f726c00bc40 comp rx=0 tx=0).stop 2024-03-26T19:13:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.279+0000 7f728961d740 1 -- 172.21.15.97:0/4247000754 shutdown_connections 2024-03-26T19:13:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.279+0000 7f728961d740 1 --2- 172.21.15.97:0/4247000754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3ed900 0x55c2fe3e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.279+0000 7f728961d740 1 --2- 172.21.15.97:0/4247000754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f9350 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.279+0000 7f728961d740 1 --2- 172.21.15.97:0/4247000754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.279+0000 7f728961d740 1 -- 172.21.15.97:0/4247000754 >> 172.21.15.97:0/4247000754 conn(0x55c2fe3e3310 msgr2=0x55c2fe3e6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.279+0000 7f728961d740 1 -- 172.21.15.97:0/4247000754 shutdown_connections 2024-03-26T19:13:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.279+0000 7f728961d740 1 -- 172.21.15.97:0/4247000754 wait complete. 2024-03-26T19:13:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.280+0000 7f728961d740 1 -- start start 2024-03-26T19:13:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.280+0000 7f728961d740 1 Processor -- start 2024-03-26T19:13:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.281+0000 7f728961d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.281+0000 7f728961d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.281+0000 7f728961d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3ed900 0x55c2fe3f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.281+0000 7f728961d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b90e0 con 0x55c2fe3ed900 2024-03-26T19:13:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.281+0000 7f728961d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b8a10 con 0x55c2fe3e7da0 2024-03-26T19:13:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.281+0000 7f728961d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7280072940 con 0x55c2fe3e8790 2024-03-26T19:13:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.281+0000 7f728961d740 10 client.0 _ll_register_callbacks cb 0x7f727009ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:13:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.281+0000 7f727fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.282+0000 7f727fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3f66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:46242/0 (socket says 172.21.15.97:46242) 2024-03-26T19:13:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.282+0000 7f727fe00640 1 -- 172.21.15.97:0/2145408961 learned_addr learned my addr 172.21.15.97:0/2145408961 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.282+0000 7f727f400640 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.282+0000 7f7284a00640 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3ed900 0x55c2fe3f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.283+0000 7f727f400640 1 -- 172.21.15.97:0/2145408961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 msgr2=0x55c2fe3f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.283+0000 7f727f400640 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.283+0000 7f727f400640 1 -- 172.21.15.97:0/2145408961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3ed900 msgr2=0x55c2fe3f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.283+0000 7f727f400640 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3ed900 0x55c2fe3f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.283+0000 7f727f400640 1 -- 172.21.15.97:0/2145408961 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2fe3f8190 con 0x55c2fe3e8790 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.283+0000 7f727f400640 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f6be0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f726c00bc10 tx=0x7f726c005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=703c01e6a6533b7f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.283+0000 7f7269600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.284+0000 7f7269600640 1 -- 172.21.15.97:0/2145408961 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c024070 con 0x55c2fe3e8790 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.284+0000 7f7269600640 1 -- 172.21.15.97:0/2145408961 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f726c010940 con 0x55c2fe3e8790 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.284+0000 7f728961d740 1 -- 172.21.15.97:0/2145408961 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f726c016040 con 0x55c2fe3e8790 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.284+0000 7f7269600640 1 -- 172.21.15.97:0/2145408961 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f726c005290 con 0x55c2fe3e8790 2024-03-26T19:13:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.284+0000 7f728961d740 10 client.26317 fetch_fsmap learned FSMap version 27 2024-03-26T19:13:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.284+0000 7f728961d740 1 -- 172.21.15.97:0/2145408961 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2fe3f94a0 con 0x55c2fe3e8790 2024-03-26T19:13:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.285+0000 7f7269600640 1 -- 172.21.15.97:0/2145408961 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f726c016040 con 0x55c2fe3e8790 2024-03-26T19:13:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.285+0000 7f7269600640 1 -- 172.21.15.97:0/2145408961 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f726c003cc0 con 0x55c2fe3e8790 2024-03-26T19:13:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.285+0000 7f728961d740 10 client.26317 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T19:13:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.285+0000 7f728961d740 10 client.26317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:13:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.285+0000 7f728961d740 20 client.26317 populate_metadata read hostname 'smithi097' 2024-03-26T19:13:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.285+0000 7f728961d740 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c2fe464500 0x55c2fe466920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.285+0000 7f728961d740 4 client.26317 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:13:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.285+0000 7f728961d740 1 -- 172.21.15.97:0/2145408961 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2fe354710 con 0x55c2fe464500 2024-03-26T19:13:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.285+0000 7f727fe00640 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c2fe464500 0x55c2fe466920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.286+0000 7f727fe00640 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c2fe464500 0x55c2fe466920 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d0d1cfabbc2635 server_cookie=5f5b96f3eeb9e6f1 in_seq=0 out_seq=0 2024-03-26T19:13:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.286+0000 7f7269600640 10 client.26317 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.456+0000 7f7269600640 1 -- 172.21.15.97:0/2145408961 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2fe354710 con 0x55c2fe464500 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.456+0000 7f7269600640 10 client.26317 handle_command_reply: tid=0 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.456+0000 7f728961d740 1 client.26317 shutdown 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.456+0000 7f728961d740 1 -- 172.21.15.97:0/2145408961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e8790 msgr2=0x55c2fe3f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.456+0000 7f728961d740 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f6be0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f726c00bc10 tx=0x7f726c005750 comp rx=0 tx=0).stop 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.456+0000 7f728961d740 1 -- 172.21.15.97:0/2145408961 shutdown_connections 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.456+0000 7f728961d740 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55c2fe464500 0x55c2fe466920 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.457+0000 7f728961d740 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fe3ed900 0x55c2fe3f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.457+0000 7f728961d740 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c2fe3e8790 0x55c2fe3f6be0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.457+0000 7f728961d740 1 --2- 172.21.15.97:0/2145408961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c2fe3e7da0 0x55c2fe3f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.457+0000 7f728961d740 1 -- 172.21.15.97:0/2145408961 >> 172.21.15.97:0/2145408961 conn(0x55c2fe3e3310 msgr2=0x55c2fe3e1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.457+0000 7f7269600640 0 client.26317 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.457+0000 7f728961d740 1 -- 172.21.15.97:0/2145408961 shutdown_connections 2024-03-26T19:13:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.457+0000 7f728961d740 1 -- 172.21.15.97:0/2145408961 wait complete. 2024-03-26T19:13:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.457+0000 7f728961d740 20 client.26317 trim_cache size 0 max 16384 2024-03-26T19:13:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:13:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:13:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:13:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:13:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 -- 172.21.15.97:0/941414642 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f72640a2fc0 msgr2=0x7f72640a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f72640a2fc0 0x7f72640a5480 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f727802c700 tx=0x7f7278005d80 comp rx=0 tx=0).stop 2024-03-26T19:13:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 -- 172.21.15.97:0/941414642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72801075c0 msgr2=0x7f72801a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72801075c0 0x7f72801a58b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f727000e970 tx=0x7f727000ed70 comp rx=0 tx=0).stop 2024-03-26T19:13:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 -- 172.21.15.97:0/941414642 shutdown_connections 2024-03-26T19:13:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f72640a2fc0 0x7f72640a5480 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72801075c0 0x7f72801a58b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7280104c90 0x7f7280195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 --2- 172.21.15.97:0/941414642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7280102360 0x7f7280195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 -- 172.21.15.97:0/941414642 >> 172.21.15.97:0/941414642 conn(0x7f72800fbd50 msgr2=0x7f7280101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 -- 172.21.15.97:0/941414642 shutdown_connections 2024-03-26T19:13:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:27.458+0000 7f7286a00640 1 -- 172.21.15.97:0/941414642 wait complete. 2024-03-26T19:13:27.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:27 smithi097 ceph-mon[31955]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.05k op/s 2024-03-26T19:13:27.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:27 smithi144 ceph-mon[36053]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.05k op/s 2024-03-26T19:13:27.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:27 smithi105 ceph-mon[36270]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.05k op/s 2024-03-26T19:13:27.967 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:13:27 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h[65989]: ignoring --setuser ceph since I am not root 2024-03-26T19:13:27.968 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:13:27 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h[65989]: ignoring --setgroup ceph since I am not root 2024-03-26T19:13:27.968 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:13:27 smithi105 ceph-mds[65993]: -- 172.21.15.105:0/1439221860 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c30585a4e0 con 0x55c305830800 2024-03-26T19:13:27.968 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:13:27 smithi105 ceph-mds[65993]: -- 172.21.15.105:0/1439221860 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c30585da40 con 0x55c305830800 2024-03-26T19:13:27.968 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:13:27 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h[65989]: starting mds.h at 2024-03-26T19:13:28.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:28 smithi097 ceph-mon[31955]: daemon mds.h restarted 2024-03-26T19:13:28.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:28 smithi144 ceph-mon[36053]: daemon mds.h restarted 2024-03-26T19:13:28.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:28 smithi105 ceph-mon[36270]: daemon mds.h restarted 2024-03-26T19:13:29.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:29 smithi097 ceph-mon[31955]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T19:13:29.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:29 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6836/4179427850,v1:172.21.15.105:6837/4179427850] up:boot 2024-03-26T19:13:29.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:29 smithi097 ceph-mon[31955]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T19:13:29.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:29 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T19:13:29.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:29 smithi097 ceph-mon[31955]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T19:13:29.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:29 smithi097 ceph-mon[31955]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 1.31k op/s 2024-03-26T19:13:29.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:29 smithi144 ceph-mon[36053]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T19:13:29.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:29 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6836/4179427850,v1:172.21.15.105:6837/4179427850] up:boot 2024-03-26T19:13:29.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:29 smithi144 ceph-mon[36053]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T19:13:29.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:29 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T19:13:29.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:29 smithi144 ceph-mon[36053]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T19:13:29.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:29 smithi144 ceph-mon[36053]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 1.31k op/s 2024-03-26T19:13:29.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:29 smithi105 ceph-mon[36270]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T19:13:29.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:29 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6836/4179427850,v1:172.21.15.105:6837/4179427850] up:boot 2024-03-26T19:13:29.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:29 smithi105 ceph-mon[36270]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T19:13:29.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:29 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T19:13:29.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:29 smithi105 ceph-mon[36270]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T19:13:29.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:29 smithi105 ceph-mon[36270]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 1.31k op/s 2024-03-26T19:13:30.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 845.0 s, 1092.4 tps, lat 93.198 ms stddev 177.179 2024-03-26T19:13:32.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:32 smithi105 ceph-mon[36270]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-03-26T19:13:32.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:32 smithi097 ceph-mon[31955]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-03-26T19:13:32.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:32 smithi144 ceph-mon[36053]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-03-26T19:13:34.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:34 smithi105 ceph-mon[36270]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 31 MiB/s wr, 1.94k op/s 2024-03-26T19:13:34.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:34 smithi097 ceph-mon[31955]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 31 MiB/s wr, 1.94k op/s 2024-03-26T19:13:34.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:34 smithi144 ceph-mon[36053]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 31 MiB/s wr, 1.94k op/s 2024-03-26T19:13:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 850.0 s, 1180.0 tps, lat 83.684 ms stddev 108.191 2024-03-26T19:13:35.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:35 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:13:35.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:35 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:13:35.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:35 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:13:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:36 smithi097 ceph-mon[31955]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 31 MiB/s wr, 2.00k op/s 2024-03-26T19:13:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:13:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:13:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:36 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:13:36.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:36 smithi144 ceph-mon[36053]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 31 MiB/s wr, 2.00k op/s 2024-03-26T19:13:36.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:13:36.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:13:36.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:36 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:13:36.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:36 smithi105 ceph-mon[36270]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 31 MiB/s wr, 2.00k op/s 2024-03-26T19:13:36.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:36 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:13:36.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:36 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:13:36.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:36 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:13:38.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:38 smithi097 ceph-mon[31955]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 1.59k op/s 2024-03-26T19:13:38.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:38 smithi144 ceph-mon[36053]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 1.59k op/s 2024-03-26T19:13:38.970 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:38 smithi105 ceph-mon[36270]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 1.59k op/s 2024-03-26T19:13:40.032 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 855.0 s, 1344.0 tps, lat 74.712 ms stddev 95.711 2024-03-26T19:13:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.632+0000 7f9831e00640 1 -- 172.21.15.97:0/3576839979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f982c075900 msgr2=0x7f982c073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.632+0000 7f9831e00640 1 --2- 172.21.15.97:0/3576839979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f982c075900 0x7f982c073d50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9820009900 tx=0x7f982002bc40 comp rx=0 tx=0).stop 2024-03-26T19:13:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.632+0000 7f9831e00640 1 -- 172.21.15.97:0/3576839979 shutdown_connections 2024-03-26T19:13:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.632+0000 7f9831e00640 1 --2- 172.21.15.97:0/3576839979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f982c1095f0 0x7f982c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.632+0000 7f9831e00640 1 --2- 172.21.15.97:0/3576839979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f982c074320 0x7f982c0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.632+0000 7f9831e00640 1 --2- 172.21.15.97:0/3576839979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f982c075900 0x7f982c073d50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.632+0000 7f9831e00640 1 -- 172.21.15.97:0/3576839979 >> 172.21.15.97:0/3576839979 conn(0x7f982c0fcca0 msgr2=0x7f982c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.632+0000 7f9831e00640 1 -- 172.21.15.97:0/3576839979 shutdown_connections 2024-03-26T19:13:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.632+0000 7f9831e00640 1 -- 172.21.15.97:0/3576839979 wait complete. 2024-03-26T19:13:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.633+0000 7f9831e00640 1 Processor -- start 2024-03-26T19:13:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.634+0000 7f9831e00640 1 -- start start 2024-03-26T19:13:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.634+0000 7f9831e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f982c074320 0x7f982c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.634+0000 7f9831e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f982c075900 0x7f982c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.634+0000 7f9831e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f982c1095f0 0x7f982c1a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.635+0000 7f9831e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f982c10f640 con 0x7f982c074320 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.635+0000 7f9831e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f982c10f4c0 con 0x7f982c1095f0 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.635+0000 7f9831e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f982c10f7c0 con 0x7f982c075900 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.635+0000 7f9831400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f982c1095f0 0x7f982c1a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.635+0000 7f9831400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f982c1095f0 0x7f982c1a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57042/0 (socket says 172.21.15.97:57042) 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.635+0000 7f9831400640 1 -- 172.21.15.97:0/464538649 learned_addr learned my addr 172.21.15.97:0/464538649 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.636+0000 7f9831400640 1 -- 172.21.15.97:0/464538649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f982c075900 msgr2=0x7f982c19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.636+0000 7f9831400640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f982c075900 0x7f982c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.636+0000 7f9831400640 1 -- 172.21.15.97:0/464538649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f982c074320 msgr2=0x7f982c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.636+0000 7f9831400640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f982c074320 0x7f982c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.636+0000 7f9831400640 1 -- 172.21.15.97:0/464538649 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c1a1890 con 0x7f982c1095f0 2024-03-26T19:13:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.637+0000 7f9831400640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f982c1095f0 0x7f982c1a11f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9824007440 tx=0x7f9824007900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.647+0000 7f9829600640 1 -- 172.21.15.97:0/464538649 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824012d30 con 0x7f982c1095f0 2024-03-26T19:13:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.647+0000 7f9829600640 1 -- 172.21.15.97:0/464538649 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98240140d0 con 0x7f982c1095f0 2024-03-26T19:13:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.647+0000 7f9831e00640 1 -- 172.21.15.97:0/464538649 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c1010e0 con 0x7f982c1095f0 2024-03-26T19:13:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.647+0000 7f9829600640 1 -- 172.21.15.97:0/464538649 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240052f0 con 0x7f982c1095f0 2024-03-26T19:13:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.647+0000 7f9831e00640 1 -- 172.21.15.97:0/464538649 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c101630 con 0x7f982c1095f0 2024-03-26T19:13:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.652+0000 7f9829600640 1 -- 172.21.15.97:0/464538649 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f9824014280 con 0x7f982c1095f0 2024-03-26T19:13:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.654+0000 7f9829600640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98140a2f90 0x7f98140a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.654+0000 7f9829600640 1 -- 172.21.15.97:0/464538649 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f98240d6f90 con 0x7f982c1095f0 2024-03-26T19:13:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.655+0000 7f9830a00640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98140a2f90 0x7f98140a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.657+0000 7f9830a00640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98140a2f90 0x7f98140a5450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9820036040 tx=0x7f9820000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.662+0000 7f9835ffb740 1 Processor -- start 2024-03-26T19:13:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.662+0000 7f9835ffb740 1 -- start start 2024-03-26T19:13:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.663+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39ef60 0x55ac1e39f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.663+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e39f950 0x55ac1e3a4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.663+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3a6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.663+0000 7f9835ffb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e303e80 con 0x55ac1e3a4ac0 2024-03-26T19:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.663+0000 7f9835ffb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e2ac660 con 0x55ac1e39f950 2024-03-26T19:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.663+0000 7f9835ffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e2ffe30 con 0x55ac1e39ef60 2024-03-26T19:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.663+0000 7f9831400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3a6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.663+0000 7f9831400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3a6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52510/0 (socket says 172.21.15.97:52510) 2024-03-26T19:13:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.663+0000 7f9831400640 1 -- 172.21.15.97:0/1760674403 learned_addr learned my addr 172.21.15.97:0/1760674403 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.664+0000 7f9831400640 1 -- 172.21.15.97:0/1760674403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39ef60 msgr2=0x55ac1e39f380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:13:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.664+0000 7f9831400640 1 --2- 172.21.15.97:0/1760674403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39ef60 0x55ac1e39f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.664+0000 7f9831400640 1 -- 172.21.15.97:0/1760674403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e39f950 msgr2=0x55ac1e3a4450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:13:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.664+0000 7f9831400640 1 --2- 172.21.15.97:0/1760674403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e39f950 0x55ac1e3a4450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.664+0000 7f9831400640 1 -- 172.21.15.97:0/1760674403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1e018210 con 0x55ac1e3a4ac0 2024-03-26T19:13:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.664+0000 7f9831400640 1 --2- 172.21.15.97:0/1760674403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3a6f50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f982409fb60 tx=0x7f982409fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a477f54eed5133e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.665+0000 7f981dc00640 1 -- 172.21.15.97:0/1760674403 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240ac1a0 con 0x55ac1e3a4ac0 2024-03-26T19:13:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.665+0000 7f981dc00640 1 -- 172.21.15.97:0/1760674403 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98240a1a90 con 0x55ac1e3a4ac0 2024-03-26T19:13:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.665+0000 7f981dc00640 1 -- 172.21.15.97:0/1760674403 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240a2ed0 con 0x55ac1e3a4ac0 2024-03-26T19:13:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.666+0000 7f9835ffb740 1 -- 172.21.15.97:0/1760674403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 msgr2=0x55ac1e3a6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.666+0000 7f9835ffb740 1 --2- 172.21.15.97:0/1760674403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3a6f50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f982409fb60 tx=0x7f982409fa20 comp rx=0 tx=0).stop 2024-03-26T19:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.668+0000 7f9835ffb740 1 -- 172.21.15.97:0/1760674403 shutdown_connections 2024-03-26T19:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.668+0000 7f9835ffb740 1 --2- 172.21.15.97:0/1760674403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3a6f50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.668+0000 7f9835ffb740 1 --2- 172.21.15.97:0/1760674403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e39f950 0x55ac1e3a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.668+0000 7f9835ffb740 1 --2- 172.21.15.97:0/1760674403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39ef60 0x55ac1e39f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.668+0000 7f9835ffb740 1 -- 172.21.15.97:0/1760674403 >> 172.21.15.97:0/1760674403 conn(0x55ac1e39a4d0 msgr2=0x55ac1e39a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.670+0000 7f9835ffb740 1 -- 172.21.15.97:0/1760674403 shutdown_connections 2024-03-26T19:13:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.671+0000 7f9835ffb740 1 -- 172.21.15.97:0/1760674403 wait complete. 2024-03-26T19:13:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.672+0000 7f9835ffb740 1 -- start start 2024-03-26T19:13:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.672+0000 7f9835ffb740 1 Processor -- start 2024-03-26T19:13:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.673+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e39ef60 0x55ac1e3a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.673+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e3a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.674+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.674+0000 7f9835ffb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e3a7a10 con 0x55ac1e3a4ac0 2024-03-26T19:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.674+0000 7f9835ffb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e2bfac0 con 0x55ac1e39ef60 2024-03-26T19:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.674+0000 7f9835ffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e3a7b90 con 0x55ac1e39f950 2024-03-26T19:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.674+0000 7f9835ffb740 10 client.0 _ll_register_callbacks cb 0x55ac1e2bf490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.674+0000 7f982be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e3a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.674+0000 7f982be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e3a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60964/0 (socket says 172.21.15.97:60964) 2024-03-26T19:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.674+0000 7f982be00640 1 -- 172.21.15.97:0/580090150 learned_addr learned my addr 172.21.15.97:0/580090150 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.674+0000 7f9830a00640 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e39ef60 0x55ac1e3a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.675+0000 7f982be00640 1 -- 172.21.15.97:0/580090150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e39ef60 msgr2=0x55ac1e3a8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.676+0000 7f982be00640 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e39ef60 0x55ac1e3a8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.676+0000 7f982be00640 1 -- 172.21.15.97:0/580090150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 msgr2=0x55ac1e3b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.676+0000 7f982be00640 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.676+0000 7f982be00640 1 -- 172.21.15.97:0/580090150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1e0e4620 con 0x55ac1e39f950 2024-03-26T19:13:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.677+0000 7f9830a00640 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e39ef60 0x55ac1e3a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:13:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.677+0000 7f982be00640 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e3a94d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9818009e10 tx=0x7f981800b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de84894a24ab168 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.677+0000 7f9812000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:13:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.677+0000 7f9812000640 1 -- 172.21.15.97:0/580090150 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818015020 con 0x55ac1e39f950 2024-03-26T19:13:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.678+0000 7f9835ffb740 1 -- 172.21.15.97:0/580090150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98240a1a90 con 0x55ac1e39f950 2024-03-26T19:13:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.678+0000 7f9812000640 1 -- 172.21.15.97:0/580090150 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f9818011620 con 0x55ac1e39f950 2024-03-26T19:13:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.678+0000 7f9812000640 1 -- 172.21.15.97:0/580090150 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9818011860 con 0x55ac1e39f950 2024-03-26T19:13:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.680+0000 7f9835ffb740 10 client.26329 fetch_fsmap learned FSMap version 29 2024-03-26T19:13:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.680+0000 7f9835ffb740 1 -- 172.21.15.97:0/580090150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac1e3b39a0 con 0x55ac1e39f950 2024-03-26T19:13:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.680+0000 7f9812000640 1 -- 172.21.15.97:0/580090150 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98240a1a90 con 0x55ac1e39f950 2024-03-26T19:13:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.681+0000 7f9812000640 1 -- 172.21.15.97:0/580090150 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9818022020 con 0x55ac1e39f950 2024-03-26T19:13:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.682+0000 7f9835ffb740 10 client.26329 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T19:13:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.682+0000 7f9835ffb740 10 client.26329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:13:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.682+0000 7f9835ffb740 20 client.26329 populate_metadata read hostname 'smithi097' 2024-03-26T19:13:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.682+0000 7f9835ffb740 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ac1e3b5130 0x55ac1e3b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.682+0000 7f9835ffb740 4 client.26329 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:13:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.682+0000 7f9835ffb740 1 -- 172.21.15.97:0/580090150 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac1e3113c0 con 0x55ac1e3b5130 2024-03-26T19:13:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.682+0000 7f9830a00640 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ac1e3b5130 0x55ac1e3b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.684+0000 7f9812000640 10 client.26329 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.684+0000 7f9830a00640 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ac1e3b5130 0x55ac1e3b7560 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e0c96e420dcf937 server_cookie=8ff7c570e7aa44e2 in_seq=0 out_seq=0 2024-03-26T19:13:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.685+0000 7f9812000640 1 -- 172.21.15.97:0/580090150 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ac1e3113c0 con 0x55ac1e3b5130 2024-03-26T19:13:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.685+0000 7f9812000640 10 client.26329 handle_command_reply: tid=0 2024-03-26T19:13:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 client.26329 shutdown 2024-03-26T19:13:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 -- 172.21.15.97:0/580090150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 msgr2=0x55ac1e3a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e3a94d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9818009e10 tx=0x7f981800b4b0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 -- 172.21.15.97:0/580090150 shutdown_connections 2024-03-26T19:13:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ac1e3b5130 0x55ac1e3b7560 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e3a94d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9812000640 0 client.26329 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 --2- 172.21.15.97:0/580090150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e39ef60 0x55ac1e3a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 -- 172.21.15.97:0/580090150 >> 172.21.15.97:0/580090150 conn(0x55ac1e39a4d0 msgr2=0x55ac1e3996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 -- 172.21.15.97:0/580090150 shutdown_connections 2024-03-26T19:13:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.686+0000 7f9835ffb740 1 -- 172.21.15.97:0/580090150 wait complete. 2024-03-26T19:13:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.687+0000 7f9835ffb740 20 client.26329 trim_cache size 0 max 16384 2024-03-26T19:13:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.722+0000 7f9835ffb740 1 Processor -- start 2024-03-26T19:13:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.723+0000 7f9835ffb740 1 -- start start 2024-03-26T19:13:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.723+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e39ef60 0x55ac1e399570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.723+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e399ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.723+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.723+0000 7f9835ffb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a69f0 con 0x55ac1e39ef60 2024-03-26T19:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.723+0000 7f9835ffb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a8530 con 0x55ac1e3a4ac0 2024-03-26T19:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.723+0000 7f9835ffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a70c0 con 0x55ac1e39f950 2024-03-26T19:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.723+0000 7f982be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e399ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.723+0000 7f982be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e399ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60966/0 (socket says 172.21.15.97:60966) 2024-03-26T19:13:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.724+0000 7f982be00640 1 -- 172.21.15.97:0/13115819 learned_addr learned my addr 172.21.15.97:0/13115819 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.724+0000 7f9830a00640 1 --2- 172.21.15.97:0/13115819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e39ef60 0x55ac1e399570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.725+0000 7f9831400640 1 --2- 172.21.15.97:0/13115819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.725+0000 7f9830a00640 1 -- 172.21.15.97:0/13115819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 msgr2=0x55ac1e399ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.725+0000 7f9830a00640 1 --2- 172.21.15.97:0/13115819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e399ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.725+0000 7f9830a00640 1 -- 172.21.15.97:0/13115819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3a4ac0 msgr2=0x55ac1e3b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.725+0000 7f9830a00640 1 --2- 172.21.15.97:0/13115819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.725+0000 7f9830a00640 1 -- 172.21.15.97:0/13115819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1e39eba0 con 0x55ac1e39ef60 2024-03-26T19:13:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.725+0000 7f982be00640 1 --2- 172.21.15.97:0/13115819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e399ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:13:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.726+0000 7f9830a00640 1 --2- 172.21.15.97:0/13115819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e39ef60 0x55ac1e399570 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f982002c7c0 tx=0x7f9820040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e62acfa870c3b71 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.726+0000 7f981d200640 1 -- 172.21.15.97:0/13115819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98200044a0 con 0x55ac1e39ef60 2024-03-26T19:13:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.727+0000 7f981d200640 1 -- 172.21.15.97:0/13115819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9820043040 con 0x55ac1e39ef60 2024-03-26T19:13:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.727+0000 7f981d200640 1 -- 172.21.15.97:0/13115819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820052690 con 0x55ac1e39ef60 2024-03-26T19:13:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.728+0000 7f9835ffb740 1 -- 172.21.15.97:0/13115819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e39ef60 msgr2=0x55ac1e399570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.728+0000 7f9835ffb740 1 --2- 172.21.15.97:0/13115819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e39ef60 0x55ac1e399570 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f982002c7c0 tx=0x7f9820040000 comp rx=0 tx=0).stop 2024-03-26T19:13:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.728+0000 7f9835ffb740 1 -- 172.21.15.97:0/13115819 shutdown_connections 2024-03-26T19:13:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.728+0000 7f9835ffb740 1 --2- 172.21.15.97:0/13115819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.728+0000 7f9835ffb740 1 --2- 172.21.15.97:0/13115819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e399ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.728+0000 7f9835ffb740 1 --2- 172.21.15.97:0/13115819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e39ef60 0x55ac1e399570 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.728+0000 7f9835ffb740 1 -- 172.21.15.97:0/13115819 >> 172.21.15.97:0/13115819 conn(0x55ac1e39a4d0 msgr2=0x55ac1e398e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.728+0000 7f9835ffb740 1 -- 172.21.15.97:0/13115819 shutdown_connections 2024-03-26T19:13:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.728+0000 7f9835ffb740 1 -- 172.21.15.97:0/13115819 wait complete. 2024-03-26T19:13:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.729+0000 7f9835ffb740 1 -- start start 2024-03-26T19:13:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.729+0000 7f9835ffb740 1 Processor -- start 2024-03-26T19:13:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.731+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e3ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.731+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.731+0000 7f9835ffb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3af770 0x55ac1e3afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9831400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3af770 0x55ac1e3afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9831400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3af770 0x55ac1e3afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:57082/0 (socket says 172.21.15.97:57082) 2024-03-26T19:13:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9831400640 1 -- 172.21.15.97:0/1853965866 learned_addr learned my addr 172.21.15.97:0/1853965866 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9830a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e3ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a7e60 con 0x55ac1e3a4ac0 2024-03-26T19:13:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a7790 con 0x55ac1e3af770 2024-03-26T19:13:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1a49f0 con 0x55ac1e39f950 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9831400640 1 -- 172.21.15.97:0/1853965866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 msgr2=0x55ac1e3ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9831400640 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e3ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9831400640 1 -- 172.21.15.97:0/1853965866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 msgr2=0x55ac1e3ae7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.732+0000 7f9835ffb740 10 client.0 _ll_register_callbacks cb 0x7f98180099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.733+0000 7f982be00640 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.733+0000 7f9831400640 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.733+0000 7f9831400640 1 -- 172.21.15.97:0/1853965866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1e395ae0 con 0x55ac1e3af770 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.733+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9820043040 con 0x55ac1e3af770 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.733+0000 7f9831400640 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3af770 0x55ac1e3afbf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9824007e00 tx=0x7f98240a3040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=531f2f8f7c5b89d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.733+0000 7f9812000640 10 client.26348 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.734+0000 7f9812000640 1 -- 172.21.15.97:0/1853965866 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240bb070 con 0x55ac1e3af770 2024-03-26T19:13:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.734+0000 7f9812000640 1 -- 172.21.15.97:0/1853965866 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f982409fd70 con 0x55ac1e3af770 2024-03-26T19:13:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.734+0000 7f9812000640 1 -- 172.21.15.97:0/1853965866 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f98240c22d0 con 0x55ac1e3af770 2024-03-26T19:13:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.734+0000 7f9835ffb740 10 client.26348 fetch_fsmap learned FSMap version 29 2024-03-26T19:13:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.734+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55ac1e39b240 con 0x55ac1e3af770 2024-03-26T19:13:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.735+0000 7f9812000640 1 -- 172.21.15.97:0/1853965866 <== mon.2 v2:172.21.15.105:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f982409f450 con 0x55ac1e3af770 2024-03-26T19:13:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.735+0000 7f9835ffb740 10 client.26348 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T19:13:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.735+0000 7f9835ffb740 10 client.26348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:13:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.735+0000 7f9835ffb740 20 client.26348 populate_metadata read hostname 'smithi097' 2024-03-26T19:13:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.735+0000 7f9835ffb740 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ac1e41be80 0x55ac1e41e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.735+0000 7f9835ffb740 4 client.26348 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:13:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.735+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac1e30b710 con 0x55ac1e41be80 2024-03-26T19:13:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.736+0000 7f9830a00640 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ac1e41be80 0x55ac1e41e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.737+0000 7f9830a00640 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ac1e41be80 0x55ac1e41e2a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8694d25a0eab319b server_cookie=44c0907be99a4351 in_seq=0 out_seq=0 2024-03-26T19:13:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.737+0000 7f9812000640 10 client.26348 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9812000640 1 -- 172.21.15.97:0/1853965866 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac1e30b710 con 0x55ac1e41be80 2024-03-26T19:13:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9812000640 10 client.26348 handle_command_reply: tid=0 2024-03-26T19:13:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9835ffb740 1 client.26348 shutdown 2024-03-26T19:13:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3af770 msgr2=0x55ac1e3afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9835ffb740 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3af770 0x55ac1e3afbf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9824007e00 tx=0x7f98240a3040 comp rx=0 tx=0).stop 2024-03-26T19:13:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 shutdown_connections 2024-03-26T19:13:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9835ffb740 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ac1e41be80 0x55ac1e41e2a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9835ffb740 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac1e3af770 0x55ac1e3afbf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9835ffb740 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac1e3a4ac0 0x55ac1e3ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9835ffb740 1 --2- 172.21.15.97:0/1853965866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac1e39f950 0x55ac1e3ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 >> 172.21.15.97:0/1853965866 conn(0x55ac1e39a4d0 msgr2=0x55ac1e3996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.798+0000 7f9812000640 0 client.26348 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.799+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 shutdown_connections 2024-03-26T19:13:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.799+0000 7f9835ffb740 1 -- 172.21.15.97:0/1853965866 wait complete. 2024-03-26T19:13:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.799+0000 7f9835ffb740 20 client.26348 trim_cache size 0 max 16384 2024-03-26T19:13:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:13:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:13:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:13:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:13:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.800+0000 7f9831e00640 1 -- 172.21.15.97:0/464538649 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98140a2f90 msgr2=0x7f98140a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:40.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.800+0000 7f9831e00640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98140a2f90 0x7f98140a5450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9820036040 tx=0x7f9820000bc0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.800+0000 7f9831e00640 1 -- 172.21.15.97:0/464538649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f982c1095f0 msgr2=0x7f982c1a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:40.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.800+0000 7f9831e00640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f982c1095f0 0x7f982c1a11f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9824007440 tx=0x7f9824007900 comp rx=0 tx=0).stop 2024-03-26T19:13:40.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.800+0000 7f9831e00640 1 -- 172.21.15.97:0/464538649 shutdown_connections 2024-03-26T19:13:40.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.800+0000 7f9831e00640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f98140a2f90 0x7f98140a5450 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.800+0000 7f9831e00640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f982c1095f0 0x7f982c1a11f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.800+0000 7f9831e00640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f982c075900 0x7f982c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.800+0000 7f9831e00640 1 --2- 172.21.15.97:0/464538649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f982c074320 0x7f982c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.800+0000 7f9831e00640 1 -- 172.21.15.97:0/464538649 >> 172.21.15.97:0/464538649 conn(0x7f982c0fcca0 msgr2=0x7f982c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.801+0000 7f9831e00640 1 -- 172.21.15.97:0/464538649 shutdown_connections 2024-03-26T19:13:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:40.801+0000 7f9831e00640 1 -- 172.21.15.97:0/464538649 wait complete. 2024-03-26T19:13:40.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:40 smithi097 ceph-mon[31955]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-03-26T19:13:40.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:40 smithi144 ceph-mon[36053]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-03-26T19:13:40.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:40 smithi105 ceph-mon[36270]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-03-26T19:13:42.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:42 smithi097 ceph-mon[31955]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-03-26T19:13:42.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:42 smithi144 ceph-mon[36053]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-03-26T19:13:42.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:42 smithi105 ceph-mon[36270]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-03-26T19:13:44.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:44 smithi097 ceph-mon[31955]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 757 op/s 2024-03-26T19:13:44.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:44 smithi144 ceph-mon[36053]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 757 op/s 2024-03-26T19:13:44.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:44 smithi105 ceph-mon[36270]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 757 op/s 2024-03-26T19:13:45.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 860.0 s, 1494.8 tps, lat 66.978 ms stddev 82.986 2024-03-26T19:13:46.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:46 smithi097 ceph-mon[31955]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-03-26T19:13:46.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:46 smithi144 ceph-mon[36053]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-03-26T19:13:46.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:46 smithi105 ceph-mon[36270]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-03-26T19:13:48.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:48 smithi097 ceph-mon[31955]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 693 op/s 2024-03-26T19:13:48.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:48 smithi144 ceph-mon[36053]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 693 op/s 2024-03-26T19:13:48.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:48 smithi105 ceph-mon[36270]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 693 op/s 2024-03-26T19:13:50.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 865.0 s, 1303.2 tps, lat 76.514 ms stddev 100.538 2024-03-26T19:13:50.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:50 smithi097 ceph-mon[31955]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 MiB/s wr, 611 op/s 2024-03-26T19:13:50.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:50 smithi144 ceph-mon[36053]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 MiB/s wr, 611 op/s 2024-03-26T19:13:50.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:50 smithi105 ceph-mon[36270]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 MiB/s wr, 611 op/s 2024-03-26T19:13:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:51 smithi097 ceph-mon[31955]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 708 op/s 2024-03-26T19:13:51.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:51 smithi144 ceph-mon[36053]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 708 op/s 2024-03-26T19:13:51.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:51 smithi105 ceph-mon[36270]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 708 op/s 2024-03-26T19:13:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.991+0000 7fd699e00640 1 -- 172.21.15.97:0/1762286111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6941023d0 msgr2=0x7fd6941047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.991+0000 7fd699e00640 1 --2- 172.21.15.97:0/1762286111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6941023d0 0x7fd6941047c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009ca0 tx=0x7fd68c02c100 comp rx=0 tx=0).stop 2024-03-26T19:13:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.992+0000 7fd699e00640 1 -- 172.21.15.97:0/1762286111 shutdown_connections 2024-03-26T19:13:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.992+0000 7fd699e00640 1 --2- 172.21.15.97:0/1762286111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd694107630 0x7fd694109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.992+0000 7fd699e00640 1 --2- 172.21.15.97:0/1762286111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd694104d00 0x7fd6941070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.992+0000 7fd699e00640 1 --2- 172.21.15.97:0/1762286111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6941023d0 0x7fd6941047c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.992+0000 7fd699e00640 1 -- 172.21.15.97:0/1762286111 >> 172.21.15.97:0/1762286111 conn(0x7fd6940fbdc0 msgr2=0x7fd6940fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.992+0000 7fd699e00640 1 -- 172.21.15.97:0/1762286111 shutdown_connections 2024-03-26T19:13:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.992+0000 7fd699e00640 1 -- 172.21.15.97:0/1762286111 wait complete. 2024-03-26T19:13:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.994+0000 7fd699e00640 1 Processor -- start 2024-03-26T19:13:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.994+0000 7fd699e00640 1 -- start start 2024-03-26T19:13:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.995+0000 7fd699e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6941023d0 0x7fd694199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.995+0000 7fd699e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd694104d00 0x7fd69419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.995+0000 7fd699e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd694107630 0x7fd6941a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.995+0000 7fd699e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd69410d560 con 0x7fd694107630 2024-03-26T19:13:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.995+0000 7fd699e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd69410d3e0 con 0x7fd6941023d0 2024-03-26T19:13:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.995+0000 7fd699e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd69410d6e0 con 0x7fd694104d00 2024-03-26T19:13:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.995+0000 7fd692000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd694104d00 0x7fd69419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.995+0000 7fd692000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd694104d00 0x7fd69419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60352/0 (socket says 172.21.15.97:60352) 2024-03-26T19:13:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.995+0000 7fd692000640 1 -- 172.21.15.97:0/4131673673 learned_addr learned my addr 172.21.15.97:0/4131673673 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.995+0000 7fd692a00640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6941023d0 0x7fd694199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.996+0000 7fd692000640 1 -- 172.21.15.97:0/4131673673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6941023d0 msgr2=0x7fd694199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.996+0000 7fd692000640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6941023d0 0x7fd694199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.996+0000 7fd692000640 1 -- 172.21.15.97:0/4131673673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd694107630 msgr2=0x7fd6941a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.996+0000 7fd692000640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd694107630 0x7fd6941a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.996+0000 7fd692000640 1 -- 172.21.15.97:0/4131673673 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6941a1990 con 0x7fd694104d00 2024-03-26T19:13:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.996+0000 7fd692000640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd694104d00 0x7fd69419a2b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd680005fd0 tx=0x7fd68000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.997+0000 7fd698a00640 1 -- 172.21.15.97:0/4131673673 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680017070 con 0x7fd694104d00 2024-03-26T19:13:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.997+0000 7fd698a00640 1 -- 172.21.15.97:0/4131673673 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680009c30 con 0x7fd694104d00 2024-03-26T19:13:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.997+0000 7fd699e00640 1 -- 172.21.15.97:0/4131673673 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6941a1c80 con 0x7fd694104d00 2024-03-26T19:13:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.997+0000 7fd698a00640 1 -- 172.21.15.97:0/4131673673 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680010460 con 0x7fd694104d00 2024-03-26T19:13:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:53.997+0000 7fd699e00640 1 -- 172.21.15.97:0/4131673673 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6941a21d0 con 0x7fd694104d00 2024-03-26T19:13:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.005+0000 7fd69ca62740 1 Processor -- start 2024-03-26T19:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.006+0000 7fd698a00640 1 -- 172.21.15.97:0/4131673673 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fd680003e70 con 0x7fd694104d00 2024-03-26T19:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.007+0000 7fd69ca62740 1 -- start start 2024-03-26T19:13:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.007+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.007+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 0x55ee3c218290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c218900 0x55ee3c21ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd69ca62740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c177e80 con 0x55ee3c213790 2024-03-26T19:13:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd69ca62740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c120660 con 0x55ee3c212da0 2024-03-26T19:13:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd69ca62740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c173e30 con 0x55ee3c218900 2024-03-26T19:13:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd693400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd693400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:49604/0 (socket says 172.21.15.97:49604) 2024-03-26T19:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd693400640 1 -- 172.21.15.97:0/1887148308 learned_addr learned my addr 172.21.15.97:0/1887148308 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd698a00640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd6780a3040 0x7fd6780a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd693400640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd6780a3040 0x7fd6780a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd692a00640 1 --2- 172.21.15.97:0/1887148308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 0x55ee3c218290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.008+0000 7fd698a00640 1 -- 172.21.15.97:0/4131673673 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd6800cefb0 con 0x7fd694104d00 2024-03-26T19:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.009+0000 7fd693400640 1 -- 172.21.15.97:0/1887148308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c218900 msgr2=0x55ee3c21ad90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.009+0000 7fd693400640 1 --2- 172.21.15.97:0/1887148308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c218900 0x55ee3c21ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.009+0000 7fd693400640 1 -- 172.21.15.97:0/1887148308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 msgr2=0x55ee3c218290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.009+0000 7fd693400640 1 --2- 172.21.15.97:0/1887148308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 0x55ee3c218290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.009+0000 7fd693400640 1 -- 172.21.15.97:0/1887148308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee3be8c210 con 0x55ee3c212da0 2024-03-26T19:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.009+0000 7fd693400640 1 --2- 172.21.15.97:0/1887148308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2131c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd684013500 tx=0x7fd6840139c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ddc345af07ba10 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.009+0000 7fd67fe00640 1 -- 172.21.15.97:0/1887148308 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684020070 con 0x55ee3c212da0 2024-03-26T19:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.010+0000 7fd67fe00640 1 -- 172.21.15.97:0/1887148308 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd684011c70 con 0x55ee3c212da0 2024-03-26T19:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.010+0000 7fd67fe00640 1 -- 172.21.15.97:0/1887148308 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68400a330 con 0x55ee3c212da0 2024-03-26T19:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.010+0000 7fd69ca62740 1 -- 172.21.15.97:0/1887148308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 msgr2=0x55ee3c2131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.010+0000 7fd69ca62740 1 --2- 172.21.15.97:0/1887148308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2131c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd684013500 tx=0x7fd6840139c0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.011+0000 7fd69ca62740 1 -- 172.21.15.97:0/1887148308 shutdown_connections 2024-03-26T19:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.011+0000 7fd69ca62740 1 --2- 172.21.15.97:0/1887148308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c218900 0x55ee3c21ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.011+0000 7fd69ca62740 1 --2- 172.21.15.97:0/1887148308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 0x55ee3c218290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.011+0000 7fd69ca62740 1 --2- 172.21.15.97:0/1887148308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2131c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.011+0000 7fd69ca62740 1 -- 172.21.15.97:0/1887148308 >> 172.21.15.97:0/1887148308 conn(0x55ee3c20e310 msgr2=0x55ee3c20e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.011+0000 7fd69ca62740 1 -- 172.21.15.97:0/1887148308 shutdown_connections 2024-03-26T19:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.011+0000 7fd69ca62740 1 -- 172.21.15.97:0/1887148308 wait complete. 2024-03-26T19:13:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.012+0000 7fd69ca62740 1 -- start start 2024-03-26T19:13:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.012+0000 7fd693400640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd6780a3040 0x7fd6780a5500 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd684004700 tx=0x7fd6840139c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.012+0000 7fd69ca62740 1 Processor -- start 2024-03-26T19:13:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.013+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c212da0 0x55ee3c21ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.013+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 0x55ee3c21d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.013+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c224340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.013+0000 7fd69ca62740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c21b950 con 0x55ee3c213790 2024-03-26T19:13:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.013+0000 7fd69ca62740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c133ac0 con 0x55ee3c218900 2024-03-26T19:13:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.013+0000 7fd69ca62740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee3c21bad0 con 0x55ee3c212da0 2024-03-26T19:13:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.013+0000 7fd69ca62740 10 client.0 _ll_register_callbacks cb 0x55ee3c133490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.013+0000 7fd693400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c224340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.013+0000 7fd692a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c212da0 0x55ee3c21ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.014+0000 7fd692a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c212da0 0x55ee3c21ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60368/0 (socket says 172.21.15.97:60368) 2024-03-26T19:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.014+0000 7fd692a00640 1 -- 172.21.15.97:0/3833131959 learned_addr learned my addr 172.21.15.97:0/3833131959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.013+0000 7fd693400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c224340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:49608/0 (socket says 172.21.15.97:49608) 2024-03-26T19:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.014+0000 7fd692000640 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 0x55ee3c21d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.016+0000 7fd693400640 1 -- 172.21.15.97:0/3833131959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c212da0 msgr2=0x55ee3c21ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.016+0000 7fd693400640 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c212da0 0x55ee3c21ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.016+0000 7fd693400640 1 -- 172.21.15.97:0/3833131959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 msgr2=0x55ee3c21d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.016+0000 7fd693400640 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 0x55ee3c21d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.016+0000 7fd693400640 1 -- 172.21.15.97:0/3833131959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee3bf58620 con 0x55ee3c218900 2024-03-26T19:13:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.016+0000 7fd692000640 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 0x55ee3c21d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:13:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.016+0000 7fd693400640 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c224340 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd6840134d0 tx=0x7fd684005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a54d41ea512a1a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.016+0000 7fd67cc00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:13:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.017+0000 7fd67cc00640 1 -- 172.21.15.97:0/3833131959 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684020070 con 0x55ee3c218900 2024-03-26T19:13:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.018+0000 7fd67cc00640 1 -- 172.21.15.97:0/3833131959 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd68401b410 con 0x55ee3c218900 2024-03-26T19:13:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.018+0000 7fd69ca62740 1 -- 172.21.15.97:0/3833131959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd684011c70 con 0x55ee3c218900 2024-03-26T19:13:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.019+0000 7fd67cc00640 1 -- 172.21.15.97:0/3833131959 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd68400ec90 con 0x55ee3c218900 2024-03-26T19:13:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.019+0000 7fd69ca62740 10 client.26357 fetch_fsmap learned FSMap version 29 2024-03-26T19:13:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.019+0000 7fd69ca62740 1 -- 172.21.15.97:0/3833131959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee3c227700 con 0x55ee3c218900 2024-03-26T19:13:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.019+0000 7fd67cc00640 1 -- 172.21.15.97:0/3833131959 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd684011c70 con 0x55ee3c218900 2024-03-26T19:13:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.020+0000 7fd67cc00640 1 -- 172.21.15.97:0/3833131959 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd68400f040 con 0x55ee3c218900 2024-03-26T19:13:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.020+0000 7fd69ca62740 10 client.26357 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T19:13:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.020+0000 7fd69ca62740 10 client.26357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:13:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.020+0000 7fd69ca62740 20 client.26357 populate_metadata read hostname 'smithi097' 2024-03-26T19:13:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.020+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ee3c228b70 0x55ee3c22af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.020+0000 7fd69ca62740 4 client.26357 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:13:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.020+0000 7fd69ca62740 1 -- 172.21.15.97:0/3833131959 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee3c1853c0 con 0x55ee3c228b70 2024-03-26T19:13:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.021+0000 7fd692a00640 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ee3c228b70 0x55ee3c22af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.021+0000 7fd692a00640 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ee3c228b70 0x55ee3c22af60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788dd518ed0d2821 server_cookie=35f860e43361aa36 in_seq=0 out_seq=0 2024-03-26T19:13:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.021+0000 7fd67cc00640 10 client.26357 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.022+0000 7fd67cc00640 1 -- 172.21.15.97:0/3833131959 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ee3c1853c0 con 0x55ee3c228b70 2024-03-26T19:13:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.022+0000 7fd67cc00640 10 client.26357 handle_command_reply: tid=0 2024-03-26T19:13:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.022+0000 7fd69ca62740 1 client.26357 shutdown 2024-03-26T19:13:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.023+0000 7fd69ca62740 1 -- 172.21.15.97:0/3833131959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 msgr2=0x55ee3c224340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.023+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c224340 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd6840134d0 tx=0x7fd684005eb0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.023+0000 7fd69ca62740 1 -- 172.21.15.97:0/3833131959 shutdown_connections 2024-03-26T19:13:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.023+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ee3c228b70 0x55ee3c22af60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.023+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c224340 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.023+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c213790 0x55ee3c21d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.023+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3833131959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c212da0 0x55ee3c21ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.023+0000 7fd69ca62740 1 -- 172.21.15.97:0/3833131959 >> 172.21.15.97:0/3833131959 conn(0x55ee3c20e310 msgr2=0x55ee3c20d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.023+0000 7fd67cc00640 0 client.26357 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.024+0000 7fd69ca62740 1 -- 172.21.15.97:0/3833131959 shutdown_connections 2024-03-26T19:13:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.024+0000 7fd69ca62740 1 -- 172.21.15.97:0/3833131959 wait complete. 2024-03-26T19:13:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.024+0000 7fd69ca62740 20 client.26357 trim_cache size 0 max 16384 2024-03-26T19:13:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd69ca62740 1 Processor -- start 2024-03-26T19:13:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd69ca62740 1 -- start start 2024-03-26T19:13:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c212da0 0x55ee3c211c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c224300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c20ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd69ca62740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd6941aaf40 con 0x55ee3c212da0 2024-03-26T19:13:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd69ca62740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6941aca80 con 0x55ee3c218900 2024-03-26T19:13:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd69ca62740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6941ab610 con 0x55ee3c213790 2024-03-26T19:13:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd692000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c224300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd692000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c224300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60384/0 (socket says 172.21.15.97:60384) 2024-03-26T19:13:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd692000640 1 -- 172.21.15.97:0/1207302556 learned_addr learned my addr 172.21.15.97:0/1207302556 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd693400640 1 --2- 172.21.15.97:0/1207302556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c20ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.050+0000 7fd692a00640 1 --2- 172.21.15.97:0/1207302556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c212da0 0x55ee3c211c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.051+0000 7fd693400640 1 -- 172.21.15.97:0/1207302556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 msgr2=0x55ee3c224300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.051+0000 7fd693400640 1 --2- 172.21.15.97:0/1207302556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c224300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.051+0000 7fd693400640 1 -- 172.21.15.97:0/1207302556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c212da0 msgr2=0x55ee3c211c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.051+0000 7fd693400640 1 --2- 172.21.15.97:0/1207302556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c212da0 0x55ee3c211c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.051+0000 7fd693400640 1 -- 172.21.15.97:0/1207302556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee3c20cab0 con 0x55ee3c218900 2024-03-26T19:13:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.051+0000 7fd692000640 1 --2- 172.21.15.97:0/1207302556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c224300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:13:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.051+0000 7fd693400640 1 --2- 172.21.15.97:0/1207302556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c20ccb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd68401e040 tx=0x7fd68400e7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae5b6f61b06fd6b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.052+0000 7fd67f400640 1 -- 172.21.15.97:0/1207302556 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684020040 con 0x55ee3c218900 2024-03-26T19:13:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.052+0000 7fd67f400640 1 -- 172.21.15.97:0/1207302556 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68400efe0 con 0x55ee3c218900 2024-03-26T19:13:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.052+0000 7fd67f400640 1 -- 172.21.15.97:0/1207302556 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68401bb40 con 0x55ee3c218900 2024-03-26T19:13:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.053+0000 7fd69ca62740 1 -- 172.21.15.97:0/1207302556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 msgr2=0x55ee3c20ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.053+0000 7fd69ca62740 1 --2- 172.21.15.97:0/1207302556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c20ccb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd68401e040 tx=0x7fd68400e7d0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.053+0000 7fd69ca62740 1 -- 172.21.15.97:0/1207302556 shutdown_connections 2024-03-26T19:13:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.053+0000 7fd69ca62740 1 --2- 172.21.15.97:0/1207302556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c218900 0x55ee3c20ccb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.053+0000 7fd69ca62740 1 --2- 172.21.15.97:0/1207302556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c224300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.053+0000 7fd69ca62740 1 --2- 172.21.15.97:0/1207302556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c212da0 0x55ee3c211c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.053+0000 7fd69ca62740 1 -- 172.21.15.97:0/1207302556 >> 172.21.15.97:0/1207302556 conn(0x55ee3c20e310 msgr2=0x55ee3c211570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.054+0000 7fd69ca62740 1 -- 172.21.15.97:0/1207302556 shutdown_connections 2024-03-26T19:13:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.054+0000 7fd69ca62740 1 -- 172.21.15.97:0/1207302556 wait complete. 2024-03-26T19:13:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.054+0000 7fd69ca62740 1 -- start start 2024-03-26T19:13:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.054+0000 7fd69ca62740 1 Processor -- start 2024-03-26T19:13:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c221b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd69ca62740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c218900 0x55ee3c222b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd692a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd692a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2215d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:49634/0 (socket says 172.21.15.97:49634) 2024-03-26T19:13:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd692000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c221b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd692000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c221b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:60394/0 (socket says 172.21.15.97:60394) 2024-03-26T19:13:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd69ca62740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd6941ac3b0 con 0x55ee3c218900 2024-03-26T19:13:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd69ca62740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6941abce0 con 0x55ee3c212da0 2024-03-26T19:13:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd69ca62740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6941a8f20 con 0x55ee3c213790 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.055+0000 7fd69ca62740 10 client.0 _ll_register_callbacks cb 0x7fd68400ef10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.056+0000 7fd692a00640 1 -- 172.21.15.97:0/3320978629 learned_addr learned my addr 172.21.15.97:0/3320978629 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.056+0000 7fd693400640 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c218900 0x55ee3c222b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.056+0000 7fd692a00640 1 -- 172.21.15.97:0/3320978629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 msgr2=0x55ee3c221b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.056+0000 7fd692a00640 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c221b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.056+0000 7fd692a00640 1 -- 172.21.15.97:0/3320978629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c218900 msgr2=0x55ee3c222b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.056+0000 7fd692a00640 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c218900 0x55ee3c222b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.056+0000 7fd692a00640 1 -- 172.21.15.97:0/3320978629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee3c223110 con 0x55ee3c212da0 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.056+0000 7fd693400640 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c218900 0x55ee3c222b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.056+0000 7fd692a00640 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2215d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd68c037040 tx=0x7fd68c032590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b7170755b6e044 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.057+0000 7fd67cc00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.057+0000 7fd67cc00640 1 -- 172.21.15.97:0/3320978629 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c04d070 con 0x55ee3c212da0 2024-03-26T19:13:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.057+0000 7fd69ca62740 1 -- 172.21.15.97:0/3320978629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68400efe0 con 0x55ee3c212da0 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.057+0000 7fd67cc00640 1 -- 172.21.15.97:0/3320978629 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd68c032820 con 0x55ee3c212da0 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.057+0000 7fd67cc00640 1 -- 172.21.15.97:0/3320978629 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd68c059050 con 0x55ee3c212da0 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.057+0000 7fd69ca62740 10 client.26363 fetch_fsmap learned FSMap version 29 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.057+0000 7fd69ca62740 1 -- 172.21.15.97:0/3320978629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee3c224490 con 0x55ee3c212da0 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.058+0000 7fd67cc00640 1 -- 172.21.15.97:0/3320978629 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68400efe0 con 0x55ee3c212da0 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.058+0000 7fd67cc00640 1 -- 172.21.15.97:0/3320978629 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd68c0492e0 con 0x55ee3c212da0 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.058+0000 7fd69ca62740 10 client.26363 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.058+0000 7fd69ca62740 10 client.26363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.058+0000 7fd69ca62740 20 client.26363 populate_metadata read hostname 'smithi097' 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.058+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ee3c297c10 0x55ee3c29a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:13:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.058+0000 7fd69ca62740 4 client.26363 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:13:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.058+0000 7fd69ca62740 1 -- 172.21.15.97:0/3320978629 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee3c17f710 con 0x55ee3c297c10 2024-03-26T19:13:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.058+0000 7fd692000640 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ee3c297c10 0x55ee3c29a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:13:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.060+0000 7fd692000640 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ee3c297c10 0x55ee3c29a030 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=137c3bec3493ad81 server_cookie=5413acf75f3402e2 in_seq=0 out_seq=0 2024-03-26T19:13:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.061+0000 7fd67cc00640 10 client.26363 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.138+0000 7fd67cc00640 1 -- 172.21.15.97:0/3320978629 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee3c17f710 con 0x55ee3c297c10 2024-03-26T19:13:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.138+0000 7fd67cc00640 10 client.26363 handle_command_reply: tid=0 2024-03-26T19:13:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 client.26363 shutdown 2024-03-26T19:13:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 -- 172.21.15.97:0/3320978629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 msgr2=0x55ee3c2215d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2215d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd68c037040 tx=0x7fd68c032590 comp rx=0 tx=0).stop 2024-03-26T19:13:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 -- 172.21.15.97:0/3320978629 shutdown_connections 2024-03-26T19:13:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55ee3c297c10 0x55ee3c29a030 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee3c218900 0x55ee3c222b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee3c213790 0x55ee3c221b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 --2- 172.21.15.97:0/3320978629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee3c212da0 0x55ee3c2215d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 -- 172.21.15.97:0/3320978629 >> 172.21.15.97:0/3320978629 conn(0x55ee3c20e310 msgr2=0x55ee3c211e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd67cc00640 0 client.26363 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:13:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 -- 172.21.15.97:0/3320978629 shutdown_connections 2024-03-26T19:13:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.139+0000 7fd69ca62740 1 -- 172.21.15.97:0/3320978629 wait complete. 2024-03-26T19:13:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.140+0000 7fd69ca62740 20 client.26363 trim_cache size 0 max 16384 2024-03-26T19:13:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:13:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:13:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:13:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:13:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.141+0000 7fd699e00640 1 -- 172.21.15.97:0/4131673673 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd6780a3040 msgr2=0x7fd6780a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.141+0000 7fd699e00640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd6780a3040 0x7fd6780a5500 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd684004700 tx=0x7fd6840139c0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.141+0000 7fd699e00640 1 -- 172.21.15.97:0/4131673673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd694104d00 msgr2=0x7fd69419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:13:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.141+0000 7fd699e00640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd694104d00 0x7fd69419a2b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd680005fd0 tx=0x7fd68000dde0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.142+0000 7fd699e00640 1 -- 172.21.15.97:0/4131673673 shutdown_connections 2024-03-26T19:13:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.142+0000 7fd699e00640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fd6780a3040 0x7fd6780a5500 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.142+0000 7fd699e00640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd694107630 0x7fd6941a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.142+0000 7fd699e00640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd694104d00 0x7fd69419a2b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.142+0000 7fd699e00640 1 --2- 172.21.15.97:0/4131673673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6941023d0 0x7fd694199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:13:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.142+0000 7fd699e00640 1 -- 172.21.15.97:0/4131673673 >> 172.21.15.97:0/4131673673 conn(0x7fd6940fbdc0 msgr2=0x7fd694101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:13:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.142+0000 7fd699e00640 1 -- 172.21.15.97:0/4131673673 shutdown_connections 2024-03-26T19:13:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:13:54.142+0000 7fd699e00640 1 -- 172.21.15.97:0/4131673673 wait complete. 2024-03-26T19:13:54.718 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:54 smithi105 ceph-mon[36270]: pgmap v840: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.0 MiB/s wr, 605 op/s 2024-03-26T19:13:54.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:54 smithi097 ceph-mon[31955]: pgmap v840: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.0 MiB/s wr, 605 op/s 2024-03-26T19:13:54.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:54 smithi144 ceph-mon[36053]: pgmap v840: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.0 MiB/s wr, 605 op/s 2024-03-26T19:13:55.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 870.0 s, 1134.2 tps, lat 87.917 ms stddev 100.614 2024-03-26T19:13:56.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:56 smithi097 ceph-mon[31955]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.8 MiB/s wr, 658 op/s 2024-03-26T19:13:56.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:56 smithi144 ceph-mon[36053]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.8 MiB/s wr, 658 op/s 2024-03-26T19:13:56.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:56 smithi105 ceph-mon[36270]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.8 MiB/s wr, 658 op/s 2024-03-26T19:13:58.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:13:58 smithi097 ceph-mon[31955]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 661 op/s 2024-03-26T19:13:58.886 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:13:58 smithi144 ceph-mon[36053]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 661 op/s 2024-03-26T19:13:58.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:13:58 smithi105 ceph-mon[36270]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 661 op/s 2024-03-26T19:14:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 875.0 s, 1194.4 tps, lat 84.860 ms stddev 103.326 2024-03-26T19:14:00.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:00 smithi097 ceph-mon[31955]: pgmap v843: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 567 op/s 2024-03-26T19:14:00.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:00 smithi144 ceph-mon[36053]: pgmap v843: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 567 op/s 2024-03-26T19:14:00.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:00 smithi105 ceph-mon[36270]: pgmap v843: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 567 op/s 2024-03-26T19:14:02.769 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:02 smithi105 ceph-mon[36270]: pgmap v844: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-03-26T19:14:02.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:02 smithi097 ceph-mon[31955]: pgmap v844: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-03-26T19:14:02.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:02 smithi144 ceph-mon[36053]: pgmap v844: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-03-26T19:14:04.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:04 smithi097 ceph-mon[31955]: pgmap v845: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 MiB/s wr, 556 op/s 2024-03-26T19:14:04.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:04 smithi144 ceph-mon[36053]: pgmap v845: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 MiB/s wr, 556 op/s 2024-03-26T19:14:04.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:04 smithi105 ceph-mon[36270]: pgmap v845: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 MiB/s wr, 556 op/s 2024-03-26T19:14:05.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 880.0 s, 1127.0 tps, lat 86.896 ms stddev 110.050 2024-03-26T19:14:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:06 smithi097 ceph-mon[31955]: pgmap v846: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 MiB/s wr, 636 op/s 2024-03-26T19:14:06.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:06 smithi144 ceph-mon[36053]: pgmap v846: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 MiB/s wr, 636 op/s 2024-03-26T19:14:06.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:06 smithi105 ceph-mon[36270]: pgmap v846: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 MiB/s wr, 636 op/s 2024-03-26T19:14:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.324+0000 7f8599400640 1 -- 172.21.15.97:0/2837244259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8594109220 msgr2=0x7f859410fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.324+0000 7f8599400640 1 --2- 172.21.15.97:0/2837244259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8594109220 0x7f859410fc00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f857c009860 tx=0x7f857c02bb70 comp rx=0 tx=0).stop 2024-03-26T19:14:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.324+0000 7f8599400640 1 -- 172.21.15.97:0/2837244259 shutdown_connections 2024-03-26T19:14:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.324+0000 7f8599400640 1 --2- 172.21.15.97:0/2837244259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8594110140 0x7f8594112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.324+0000 7f8599400640 1 --2- 172.21.15.97:0/2837244259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8594109220 0x7f859410fc00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.324+0000 7f8599400640 1 --2- 172.21.15.97:0/2837244259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85941088c0 0x7f8594108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.324+0000 7f8599400640 1 -- 172.21.15.97:0/2837244259 >> 172.21.15.97:0/2837244259 conn(0x7f859406e900 msgr2=0x7f8594070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.324+0000 7f8599400640 1 -- 172.21.15.97:0/2837244259 shutdown_connections 2024-03-26T19:14:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.324+0000 7f8599400640 1 -- 172.21.15.97:0/2837244259 wait complete. 2024-03-26T19:14:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.326+0000 7f8599400640 1 Processor -- start 2024-03-26T19:14:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.326+0000 7f8599400640 1 -- start start 2024-03-26T19:14:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.327+0000 7f8599400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85941088c0 0x7f85941a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.327+0000 7f8599400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8594109220 0x7f85941a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.327+0000 7f8599400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8594110140 0x7f85941a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.327+0000 7f8599400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8594115d60 con 0x7f8594109220 2024-03-26T19:14:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.327+0000 7f8599400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8594115be0 con 0x7f85941088c0 2024-03-26T19:14:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.327+0000 7f8599400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8594115ee0 con 0x7f8594110140 2024-03-26T19:14:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.327+0000 7f8593200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8594110140 0x7f85941a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.327+0000 7f8593200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8594110140 0x7f85941a78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:36814/0 (socket says 172.21.15.97:36814) 2024-03-26T19:14:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.327+0000 7f8593200640 1 -- 172.21.15.97:0/2707266443 learned_addr learned my addr 172.21.15.97:0/2707266443 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.327+0000 7f8592800640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85941088c0 0x7f85941a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.328+0000 7f8593200640 1 -- 172.21.15.97:0/2707266443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85941088c0 msgr2=0x7f85941a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.328+0000 7f8593200640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85941088c0 0x7f85941a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.328+0000 7f8593200640 1 -- 172.21.15.97:0/2707266443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8594109220 msgr2=0x7f85941a0850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T19:14:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.329+0000 7f8593200640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8594109220 0x7f85941a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.329+0000 7f8593200640 1 -- 172.21.15.97:0/2707266443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85941a7f30 con 0x7f8594110140 2024-03-26T19:14:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.329+0000 7f8593200640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8594110140 0x7f85941a78c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f858400d950 tx=0x7f858400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.330+0000 7f858b400640 1 -- 172.21.15.97:0/2707266443 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584017030 con 0x7f8594110140 2024-03-26T19:14:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.330+0000 7f858b400640 1 -- 172.21.15.97:0/2707266443 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8584004020 con 0x7f8594110140 2024-03-26T19:14:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.330+0000 7f858b400640 1 -- 172.21.15.97:0/2707266443 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584003c00 con 0x7f8594110140 2024-03-26T19:14:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.330+0000 7f8599400640 1 -- 172.21.15.97:0/2707266443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8594109a30 con 0x7f8594110140 2024-03-26T19:14:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.330+0000 7f8599400640 1 -- 172.21.15.97:0/2707266443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8594109f80 con 0x7f8594110140 2024-03-26T19:14:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.337+0000 7f859c1e4740 1 Processor -- start 2024-03-26T19:14:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.339+0000 7f858b400640 1 -- 172.21.15.97:0/2707266443 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f858401a070 con 0x7f8594110140 2024-03-26T19:14:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.342+0000 7f858b400640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85780a7540 0x7f85780a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.342+0000 7f858b400640 1 -- 172.21.15.97:0/2707266443 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f85840cee70 con 0x7f8594110140 2024-03-26T19:14:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.342+0000 7f859c1e4740 1 -- start start 2024-03-26T19:14:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.343+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769ae01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.343+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 0x558769ae5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.343+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 0x558769ae7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.343+0000 7f859c1e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558769a44e80 con 0x558769ae0790 2024-03-26T19:14:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.343+0000 7f859c1e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5587699ed660 con 0x558769ae5900 2024-03-26T19:14:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.343+0000 7f859c1e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558769a40e30 con 0x558769adfda0 2024-03-26T19:14:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.343+0000 7f8591e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 0x558769ae5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.343+0000 7f8592800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769ae01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.344+0000 7f8592800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769ae01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:36828/0 (socket says 172.21.15.97:36828) 2024-03-26T19:14:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.344+0000 7f8592800640 1 -- 172.21.15.97:0/2385972945 learned_addr learned my addr 172.21.15.97:0/2385972945 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.344+0000 7f8593200640 1 --2- 172.21.15.97:0/2385972945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 0x558769ae7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.344+0000 7f8591e00640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85780a7540 0x7f85780a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.345+0000 7f8592800640 1 -- 172.21.15.97:0/2385972945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 msgr2=0x558769ae7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.345+0000 7f8592800640 1 --2- 172.21.15.97:0/2385972945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 0x558769ae7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.345+0000 7f8592800640 1 -- 172.21.15.97:0/2385972945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 msgr2=0x558769ae5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.345+0000 7f8592800640 1 --2- 172.21.15.97:0/2385972945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 0x558769ae5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.345+0000 7f8592800640 1 -- 172.21.15.97:0/2385972945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558769759210 con 0x558769adfda0 2024-03-26T19:14:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.345+0000 7f8591e00640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85780a7540 0x7f85780a9a00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f857c0095f0 tx=0x7f857c005730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.346+0000 7f8592800640 1 --2- 172.21.15.97:0/2385972945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769ae01c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f858c00e5d0 tx=0x7f858c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f66729ff8453b13d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.346+0000 7f8583400640 1 -- 172.21.15.97:0/2385972945 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c00dad0 con 0x558769adfda0 2024-03-26T19:14:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.347+0000 7f8583400640 1 -- 172.21.15.97:0/2385972945 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f858c01c070 con 0x558769adfda0 2024-03-26T19:14:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.347+0000 7f8583400640 1 -- 172.21.15.97:0/2385972945 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c00dad0 con 0x558769adfda0 2024-03-26T19:14:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.350+0000 7f859c1e4740 1 -- 172.21.15.97:0/2385972945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 msgr2=0x558769ae01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.350+0000 7f859c1e4740 1 --2- 172.21.15.97:0/2385972945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769ae01c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f858c00e5d0 tx=0x7f858c0095c0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.350+0000 7f859c1e4740 1 -- 172.21.15.97:0/2385972945 shutdown_connections 2024-03-26T19:14:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.350+0000 7f859c1e4740 1 --2- 172.21.15.97:0/2385972945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 0x558769ae7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.350+0000 7f859c1e4740 1 --2- 172.21.15.97:0/2385972945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 0x558769ae5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.350+0000 7f859c1e4740 1 --2- 172.21.15.97:0/2385972945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769ae01c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.350+0000 7f859c1e4740 1 -- 172.21.15.97:0/2385972945 >> 172.21.15.97:0/2385972945 conn(0x558769adb310 msgr2=0x558769adb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.351+0000 7f859c1e4740 1 -- 172.21.15.97:0/2385972945 shutdown_connections 2024-03-26T19:14:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.351+0000 7f859c1e4740 1 -- 172.21.15.97:0/2385972945 wait complete. 2024-03-26T19:14:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.352+0000 7f859c1e4740 1 -- start start 2024-03-26T19:14:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.353+0000 7f859c1e4740 1 Processor -- start 2024-03-26T19:14:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.355+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769adfda0 0x558769ae9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.355+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae0790 0x558769aea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.355+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae5900 0x558769af9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.355+0000 7f859c1e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558769ae88b0 con 0x558769adfda0 2024-03-26T19:14:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.355+0000 7f859c1e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558769a00ac0 con 0x558769ae0790 2024-03-26T19:14:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.355+0000 7f859c1e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558769ae8a30 con 0x558769ae5900 2024-03-26T19:14:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.355+0000 7f859c1e4740 10 client.0 _ll_register_callbacks cb 0x558769a00490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:14:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.355+0000 7f8591e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae0790 0x558769aea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.355+0000 7f8591e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae0790 0x558769aea290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53928/0 (socket says 172.21.15.97:53928) 2024-03-26T19:14:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.355+0000 7f8591e00640 1 -- 172.21.15.97:0/2742084906 learned_addr learned my addr 172.21.15.97:0/2742084906 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.356+0000 7f8593200640 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae5900 0x558769af9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.356+0000 7f8591e00640 1 -- 172.21.15.97:0/2742084906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae5900 msgr2=0x558769af9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.356+0000 7f8591e00640 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae5900 0x558769af9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.356+0000 7f8591e00640 1 -- 172.21.15.97:0/2742084906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769adfda0 msgr2=0x558769ae9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.356+0000 7f8591e00640 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769adfda0 0x558769ae9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.356+0000 7f8591e00640 1 -- 172.21.15.97:0/2742084906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558769825620 con 0x558769ae0790 2024-03-26T19:14:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.357+0000 7f8591e00640 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae0790 0x558769aea290 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f857c009230 tx=0x7f857c005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21fd7201049a1d74 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.357+0000 7f8577e00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:14:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.358+0000 7f8577e00640 1 -- 172.21.15.97:0/2742084906 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c039040 con 0x558769ae0790 2024-03-26T19:14:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.358+0000 7f8577e00640 1 -- 172.21.15.97:0/2742084906 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f857c004050 con 0x558769ae0790 2024-03-26T19:14:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.358+0000 7f859c1e4740 1 -- 172.21.15.97:0/2742084906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f858c01c070 con 0x558769ae0790 2024-03-26T19:14:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.359+0000 7f8577e00640 1 -- 172.21.15.97:0/2742084906 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f857c004250 con 0x558769ae0790 2024-03-26T19:14:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.359+0000 7f859c1e4740 10 client.26375 fetch_fsmap learned FSMap version 29 2024-03-26T19:14:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.359+0000 7f859c1e4740 1 -- 172.21.15.97:0/2742084906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558769aecea0 con 0x558769ae0790 2024-03-26T19:14:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.359+0000 7f8577e00640 1 -- 172.21.15.97:0/2742084906 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f858c01c070 con 0x558769ae0790 2024-03-26T19:14:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.359+0000 7f8577e00640 1 -- 172.21.15.97:0/2742084906 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f857c03f030 con 0x558769ae0790 2024-03-26T19:14:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.360+0000 7f859c1e4740 10 client.26375 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T19:14:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.360+0000 7f859c1e4740 10 client.26375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:14:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.360+0000 7f859c1e4740 20 client.26375 populate_metadata read hostname 'smithi097' 2024-03-26T19:14:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.360+0000 7f859c1e4740 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558769aee590 0x558769af0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.360+0000 7f859c1e4740 4 client.26375 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:14:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.360+0000 7f859c1e4740 1 -- 172.21.15.97:0/2742084906 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558769a523c0 con 0x558769aee590 2024-03-26T19:14:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.360+0000 7f8592800640 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558769aee590 0x558769af0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.361+0000 7f8592800640 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558769aee590 0x558769af0a50 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d16fedf5d123126e server_cookie=72c728e0177e2eba in_seq=0 out_seq=0 2024-03-26T19:14:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.361+0000 7f8577e00640 10 client.26375 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.362+0000 7f8577e00640 1 -- 172.21.15.97:0/2742084906 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558769a523c0 con 0x558769aee590 2024-03-26T19:14:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.362+0000 7f8577e00640 10 client.26375 handle_command_reply: tid=0 2024-03-26T19:14:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.362+0000 7f859c1e4740 1 client.26375 shutdown 2024-03-26T19:14:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f859c1e4740 1 -- 172.21.15.97:0/2742084906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae0790 msgr2=0x558769aea290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f859c1e4740 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae0790 0x558769aea290 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f857c009230 tx=0x7f857c005a90 comp rx=0 tx=0).stop 2024-03-26T19:14:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f859c1e4740 1 -- 172.21.15.97:0/2742084906 shutdown_connections 2024-03-26T19:14:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f859c1e4740 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558769aee590 0x558769af0a50 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f859c1e4740 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae5900 0x558769af9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f859c1e4740 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae0790 0x558769aea290 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f859c1e4740 1 --2- 172.21.15.97:0/2742084906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769adfda0 0x558769ae9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f859c1e4740 1 -- 172.21.15.97:0/2742084906 >> 172.21.15.97:0/2742084906 conn(0x558769adb310 msgr2=0x558769ada540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f8577e00640 0 client.26375 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f859c1e4740 1 -- 172.21.15.97:0/2742084906 shutdown_connections 2024-03-26T19:14:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.363+0000 7f859c1e4740 1 -- 172.21.15.97:0/2742084906 wait complete. 2024-03-26T19:14:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.364+0000 7f859c1e4740 20 client.26375 trim_cache size 0 max 16384 2024-03-26T19:14:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f859c1e4740 1 Processor -- start 2024-03-26T19:14:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f859c1e4740 1 -- start start 2024-03-26T19:14:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769adfda0 0x558769ade570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae0790 0x558769ae93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae5900 0x558769ae9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f859c1e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85941af250 con 0x558769ae5900 2024-03-26T19:14:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f859c1e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85941b0d90 con 0x558769adfda0 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f859c1e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85941af920 con 0x558769ae0790 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f8592800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769adfda0 0x558769ade570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f8592800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769adfda0 0x558769ade570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53936/0 (socket says 172.21.15.97:53936) 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f8592800640 1 -- 172.21.15.97:0/3377151809 learned_addr learned my addr 172.21.15.97:0/3377151809 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.383+0000 7f8591e00640 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae0790 0x558769ae93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8593200640 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae5900 0x558769ae9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8592800640 1 -- 172.21.15.97:0/3377151809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae0790 msgr2=0x558769ae93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8592800640 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae0790 0x558769ae93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8592800640 1 -- 172.21.15.97:0/3377151809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae5900 msgr2=0x558769ae9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8592800640 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae5900 0x558769ae9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8592800640 1 -- 172.21.15.97:0/3377151809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558769ae1020 con 0x558769adfda0 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8591e00640 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae0790 0x558769ae93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8593200640 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae5900 0x558769ae9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8592800640 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769adfda0 0x558769ade570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x558769ae0ff0 tx=0x7f858c005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25d75d5f25094493 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8582a00640 1 -- 172.21.15.97:0/3377151809 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c02bb10 con 0x558769adfda0 2024-03-26T19:14:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8582a00640 1 -- 172.21.15.97:0/3377151809 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f858c01c040 con 0x558769adfda0 2024-03-26T19:14:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.384+0000 7f8582a00640 1 -- 172.21.15.97:0/3377151809 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c017540 con 0x558769adfda0 2024-03-26T19:14:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.385+0000 7f859c1e4740 1 -- 172.21.15.97:0/3377151809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769adfda0 msgr2=0x558769ade570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.385+0000 7f859c1e4740 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769adfda0 0x558769ade570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x558769ae0ff0 tx=0x7f858c005bd0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.385+0000 7f859c1e4740 1 -- 172.21.15.97:0/3377151809 shutdown_connections 2024-03-26T19:14:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.385+0000 7f859c1e4740 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae5900 0x558769ae9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.385+0000 7f859c1e4740 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769ae0790 0x558769ae93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.385+0000 7f859c1e4740 1 --2- 172.21.15.97:0/3377151809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769adfda0 0x558769ade570 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.385+0000 7f859c1e4740 1 -- 172.21.15.97:0/3377151809 >> 172.21.15.97:0/3377151809 conn(0x558769adb310 msgr2=0x558769ae1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.385+0000 7f859c1e4740 1 -- 172.21.15.97:0/3377151809 shutdown_connections 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.385+0000 7f859c1e4740 1 -- 172.21.15.97:0/3377151809 wait complete. 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.386+0000 7f859c1e4740 1 -- start start 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.386+0000 7f859c1e4740 1 Processor -- start 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.386+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769af3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.386+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 0x558769af3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.386+0000 7f859c1e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 0x558769b6b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.386+0000 7f859c1e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85941b06c0 con 0x558769ae0790 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.386+0000 7f859c1e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85941afff0 con 0x558769ae5900 2024-03-26T19:14:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.386+0000 7f859c1e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85941ad230 con 0x558769adfda0 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.386+0000 7f859c1e4740 10 client.0 _ll_register_callbacks cb 0x7f857c0094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8593200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 0x558769b6b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8593200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 0x558769b6b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:53950/0 (socket says 172.21.15.97:53950) 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8593200640 1 -- 172.21.15.97:0/3117603083 learned_addr learned my addr 172.21.15.97:0/3117603083 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8592800640 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769af3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8593200640 1 -- 172.21.15.97:0/3117603083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 msgr2=0x558769af3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8593200640 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769af3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8593200640 1 -- 172.21.15.97:0/3117603083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 msgr2=0x558769af3a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8591e00640 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 0x558769af3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8593200640 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 0x558769af3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8593200640 1 -- 172.21.15.97:0/3117603083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558769af4100 con 0x558769ae5900 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8592800640 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769af3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:14:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.387+0000 7f8591e00640 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 0x558769af3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.388+0000 7f8593200640 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 0x558769b6b810 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8584097fa0 tx=0x7f8584097200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c98033c2d98edce6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.388+0000 7f8577e00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.388+0000 7f8577e00640 1 -- 172.21.15.97:0/3117603083 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85840cce60 con 0x558769ae5900 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.388+0000 7f8577e00640 1 -- 172.21.15.97:0/3117603083 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f8584097c40 con 0x558769ae5900 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.388+0000 7f859c1e4740 1 -- 172.21.15.97:0/3117603083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f858c01c040 con 0x558769ae5900 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.388+0000 7f8577e00640 1 -- 172.21.15.97:0/3117603083 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85840a3fe0 con 0x558769ae5900 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.388+0000 7f859c1e4740 10 client.26381 fetch_fsmap learned FSMap version 29 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.388+0000 7f859c1e4740 1 -- 172.21.15.97:0/3117603083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558769b6cf00 con 0x558769ae5900 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.389+0000 7f8577e00640 1 -- 172.21.15.97:0/3117603083 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f858c01c040 con 0x558769ae5900 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.389+0000 7f8577e00640 1 -- 172.21.15.97:0/3117603083 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8584024060 con 0x558769ae5900 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.389+0000 7f859c1e4740 10 client.26381 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.389+0000 7f859c1e4740 10 client.26381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.389+0000 7f859c1e4740 20 client.26381 populate_metadata read hostname 'smithi097' 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.389+0000 7f859c1e4740 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558769b6d8b0 0x558769b6fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.389+0000 7f859c1e4740 4 client.26381 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:14:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.389+0000 7f859c1e4740 1 -- 172.21.15.97:0/3117603083 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558769a4c710 con 0x558769b6d8b0 2024-03-26T19:14:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.389+0000 7f8592800640 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558769b6d8b0 0x558769b6fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.392+0000 7f8592800640 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558769b6d8b0 0x558769b6fcd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea627086203103c server_cookie=5aed6860c50b32e8 in_seq=0 out_seq=0 2024-03-26T19:14:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.392+0000 7f8577e00640 10 client.26381 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.415+0000 7f8577e00640 1 -- 172.21.15.97:0/3117603083 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558769a4c710 con 0x558769b6d8b0 2024-03-26T19:14:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.415+0000 7f8577e00640 10 client.26381 handle_command_reply: tid=0 2024-03-26T19:14:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.415+0000 7f859c1e4740 1 client.26381 shutdown 2024-03-26T19:14:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.416+0000 7f859c1e4740 1 -- 172.21.15.97:0/3117603083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 msgr2=0x558769b6b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.416+0000 7f859c1e4740 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 0x558769b6b810 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8584097fa0 tx=0x7f8584097200 comp rx=0 tx=0).stop 2024-03-26T19:14:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.416+0000 7f859c1e4740 1 -- 172.21.15.97:0/3117603083 shutdown_connections 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.416+0000 7f859c1e4740 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x558769b6d8b0 0x558769b6fcd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.416+0000 7f859c1e4740 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558769ae5900 0x558769b6b810 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.416+0000 7f859c1e4740 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558769ae0790 0x558769af3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.416+0000 7f859c1e4740 1 --2- 172.21.15.97:0/3117603083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558769adfda0 0x558769af3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.416+0000 7f859c1e4740 1 -- 172.21.15.97:0/3117603083 >> 172.21.15.97:0/3117603083 conn(0x558769adb310 msgr2=0x558769ae1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.416+0000 7f8577e00640 0 client.26381 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.416+0000 7f859c1e4740 1 -- 172.21.15.97:0/3117603083 shutdown_connections 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.417+0000 7f859c1e4740 1 -- 172.21.15.97:0/3117603083 wait complete. 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.417+0000 7f859c1e4740 20 client.26381 trim_cache size 0 max 16384 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:14:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:14:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:14:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:14:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.418+0000 7f8599400640 1 -- 172.21.15.97:0/2707266443 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85780a7540 msgr2=0x7f85780a9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.418+0000 7f8599400640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85780a7540 0x7f85780a9a00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f857c0095f0 tx=0x7f857c005730 comp rx=0 tx=0).stop 2024-03-26T19:14:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.418+0000 7f8599400640 1 -- 172.21.15.97:0/2707266443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8594110140 msgr2=0x7f85941a78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.418+0000 7f8599400640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8594110140 0x7f85941a78c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f858400d950 tx=0x7f858400de10 comp rx=0 tx=0).stop 2024-03-26T19:14:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.419+0000 7f8599400640 1 -- 172.21.15.97:0/2707266443 shutdown_connections 2024-03-26T19:14:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.419+0000 7f8599400640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f85780a7540 0x7f85780a9a00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.419+0000 7f8599400640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8594110140 0x7f85941a78c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.419+0000 7f8599400640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8594109220 0x7f85941a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.419+0000 7f8599400640 1 --2- 172.21.15.97:0/2707266443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85941088c0 0x7f85941a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.419+0000 7f8599400640 1 -- 172.21.15.97:0/2707266443 >> 172.21.15.97:0/2707266443 conn(0x7f859406e900 msgr2=0x7f85940704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.419+0000 7f8599400640 1 -- 172.21.15.97:0/2707266443 shutdown_connections 2024-03-26T19:14:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:07.420+0000 7f8599400640 1 -- 172.21.15.97:0/2707266443 wait complete. 2024-03-26T19:14:08.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:08 smithi097 ceph-mon[31955]: pgmap v847: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 619 op/s 2024-03-26T19:14:08.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:08 smithi144 ceph-mon[36053]: pgmap v847: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 619 op/s 2024-03-26T19:14:08.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:08 smithi105 ceph-mon[36270]: pgmap v847: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 619 op/s 2024-03-26T19:14:10.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 885.0 s, 902.2 tps, lat 111.313 ms stddev 130.554 2024-03-26T19:14:10.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:10 smithi097 ceph-mon[31955]: pgmap v848: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.8 MiB/s wr, 560 op/s 2024-03-26T19:14:10.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:10 smithi144 ceph-mon[36053]: pgmap v848: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.8 MiB/s wr, 560 op/s 2024-03-26T19:14:10.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:10 smithi105 ceph-mon[36270]: pgmap v848: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.8 MiB/s wr, 560 op/s 2024-03-26T19:14:12.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:12 smithi097 ceph-mon[31955]: pgmap v849: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 647 op/s 2024-03-26T19:14:12.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:12 smithi144 ceph-mon[36053]: pgmap v849: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 647 op/s 2024-03-26T19:14:12.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:12 smithi105 ceph-mon[36270]: pgmap v849: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 647 op/s 2024-03-26T19:14:14.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:14 smithi097 ceph-mon[31955]: pgmap v850: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.6 MiB/s wr, 628 op/s 2024-03-26T19:14:14.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:14 smithi144 ceph-mon[36053]: pgmap v850: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.6 MiB/s wr, 628 op/s 2024-03-26T19:14:14.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:14 smithi105 ceph-mon[36270]: pgmap v850: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.6 MiB/s wr, 628 op/s 2024-03-26T19:14:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 890.0 s, 1539.0 tps, lat 66.104 ms stddev 79.838 2024-03-26T19:14:15.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:15 smithi097 ceph-mon[31955]: pgmap v851: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.4 MiB/s wr, 627 op/s 2024-03-26T19:14:15.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:15 smithi144 ceph-mon[36053]: pgmap v851: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.4 MiB/s wr, 627 op/s 2024-03-26T19:14:15.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:15 smithi105 ceph-mon[36270]: pgmap v851: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.4 MiB/s wr, 627 op/s 2024-03-26T19:14:18.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:18 smithi097 ceph-mon[31955]: pgmap v852: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 618 op/s 2024-03-26T19:14:18.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:18 smithi144 ceph-mon[36053]: pgmap v852: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 618 op/s 2024-03-26T19:14:18.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:18 smithi105 ceph-mon[36270]: pgmap v852: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 618 op/s 2024-03-26T19:14:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 895.0 s, 1199.8 tps, lat 81.081 ms stddev 91.939 2024-03-26T19:14:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.603+0000 7f3ba7000640 1 -- 172.21.15.97:0/2412226860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0076280 msgr2=0x7f3ba00766a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.603+0000 7f3ba7000640 1 --2- 172.21.15.97:0/2412226860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0076280 0x7f3ba00766a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b90009900 tx=0x7f3b9002bc40 comp rx=0 tx=0).stop 2024-03-26T19:14:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.603+0000 7f3ba7000640 1 -- 172.21.15.97:0/2412226860 shutdown_connections 2024-03-26T19:14:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.603+0000 7f3ba7000640 1 --2- 172.21.15.97:0/2412226860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ba0077e60 0x7f3ba0078700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.603+0000 7f3ba7000640 1 --2- 172.21.15.97:0/2412226860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba00774c0 0x7f3ba0077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.603+0000 7f3ba7000640 1 --2- 172.21.15.97:0/2412226860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0076280 0x7f3ba00766a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.603+0000 7f3ba7000640 1 -- 172.21.15.97:0/2412226860 >> 172.21.15.97:0/2412226860 conn(0x7f3ba00ff0b0 msgr2=0x7f3ba01014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.603+0000 7f3ba7000640 1 -- 172.21.15.97:0/2412226860 shutdown_connections 2024-03-26T19:14:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.603+0000 7f3ba7000640 1 -- 172.21.15.97:0/2412226860 wait complete. 2024-03-26T19:14:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.605+0000 7f3ba7000640 1 Processor -- start 2024-03-26T19:14:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.606+0000 7f3ba7000640 1 -- start start 2024-03-26T19:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.607+0000 7f3ba7000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba0076280 0x7f3ba019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.607+0000 7f3ba7000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ba00774c0 0x7f3ba019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.607+0000 7f3ba7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0077e60 0x7f3ba01a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.607+0000 7f3ba7000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba0113bf0 con 0x7f3ba0077e60 2024-03-26T19:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.607+0000 7f3ba7000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba0113a70 con 0x7f3ba0076280 2024-03-26T19:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.607+0000 7f3ba7000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba0113d70 con 0x7f3ba00774c0 2024-03-26T19:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3b9f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ba00774c0 0x7f3ba019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3b9f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ba00774c0 0x7f3ba019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:40524/0 (socket says 172.21.15.97:40524) 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3b9f400640 1 -- 172.21.15.97:0/2628909497 learned_addr learned my addr 172.21.15.97:0/2628909497 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3b9fe00640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba0076280 0x7f3ba019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3ba5000640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0077e60 0x7f3ba01a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3b9f400640 1 -- 172.21.15.97:0/2628909497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba0076280 msgr2=0x7f3ba019e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3b9f400640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba0076280 0x7f3ba019e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3b9f400640 1 -- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0077e60 msgr2=0x7f3ba01a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3b9f400640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0077e60 0x7f3ba01a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3b9f400640 1 -- 172.21.15.97:0/2628909497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba01a5d90 con 0x7f3ba00774c0 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3b9fe00640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba0076280 0x7f3ba019e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.608+0000 7f3ba5000640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0077e60 0x7f3ba01a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.609+0000 7f3b9f400640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ba00774c0 0x7f3ba019e700 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00e9f0 tx=0x7f3b8c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.609+0000 7f3b9cc00640 1 -- 172.21.15.97:0/2628909497 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00cd80 con 0x7f3ba00774c0 2024-03-26T19:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.609+0000 7f3b9cc00640 1 -- 172.21.15.97:0/2628909497 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b8c010910 con 0x7f3ba00774c0 2024-03-26T19:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.609+0000 7f3b9cc00640 1 -- 172.21.15.97:0/2628909497 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c018990 con 0x7f3ba00774c0 2024-03-26T19:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.609+0000 7f3ba7000640 1 -- 172.21.15.97:0/2628909497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba01a6080 con 0x7f3ba00774c0 2024-03-26T19:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.610+0000 7f3ba7000640 1 -- 172.21.15.97:0/2628909497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba01a6580 con 0x7f3ba00774c0 2024-03-26T19:14:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.614+0000 7f3ba9bf6740 1 Processor -- start 2024-03-26T19:14:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.615+0000 7f3ba9bf6740 1 -- start start 2024-03-26T19:14:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.616+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b01da0 0x561090b021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.616+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b02790 0x561090b07290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.616+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b07900 0x561090b09d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.616+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561090a66e80 con 0x561090b02790 2024-03-26T19:14:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.616+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561090a0f660 con 0x561090b07900 2024-03-26T19:14:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.616+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561090a62e30 con 0x561090b01da0 2024-03-26T19:14:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.616+0000 7f3ba5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b01da0 0x561090b021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.616+0000 7f3ba5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b01da0 0x561090b021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:40538/0 (socket says 172.21.15.97:40538) 2024-03-26T19:14:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.617+0000 7f3ba5000640 1 -- 172.21.15.97:0/4227347389 learned_addr learned my addr 172.21.15.97:0/4227347389 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.617+0000 7f3b9fe00640 1 --2- 172.21.15.97:0/4227347389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b02790 0x561090b07290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.617+0000 7f3ba5000640 1 -- 172.21.15.97:0/4227347389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b07900 msgr2=0x561090b09d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:14:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.617+0000 7f3b9cc00640 1 -- 172.21.15.97:0/2628909497 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f3b8c010ac0 con 0x7f3ba00774c0 2024-03-26T19:14:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.617+0000 7f3ba5000640 1 --2- 172.21.15.97:0/4227347389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b07900 0x561090b09d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.617+0000 7f3ba5000640 1 -- 172.21.15.97:0/4227347389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b02790 msgr2=0x561090b07290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.617+0000 7f3ba5000640 1 --2- 172.21.15.97:0/4227347389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b02790 0x561090b07290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.617+0000 7f3ba5000640 1 -- 172.21.15.97:0/4227347389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109077b210 con 0x561090b01da0 2024-03-26T19:14:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.617+0000 7f3b9fe00640 1 --2- 172.21.15.97:0/4227347389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b02790 0x561090b07290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.618+0000 7f3ba5000640 1 --2- 172.21.15.97:0/4227347389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b01da0 0x561090b021c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009f90 tx=0x7f3b9800f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44434bd4ab42f58b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.618+0000 7f3b95600640 1 -- 172.21.15.97:0/4227347389 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98003b70 con 0x561090b01da0 2024-03-26T19:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.618+0000 7f3b95600640 1 -- 172.21.15.97:0/4227347389 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b98003d10 con 0x561090b01da0 2024-03-26T19:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.618+0000 7f3b95600640 1 -- 172.21.15.97:0/4227347389 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98011a30 con 0x561090b01da0 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.619+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/4227347389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b01da0 msgr2=0x561090b021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.619+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/4227347389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b01da0 0x561090b021c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009f90 tx=0x7f3b9800f7f0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.620+0000 7f3b9cc00640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b840a3040 0x7f3b840a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.620+0000 7f3b9cc00640 1 -- 172.21.15.97:0/2628909497 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3b8c0cf9f0 con 0x7f3ba00774c0 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.620+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/4227347389 shutdown_connections 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.620+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/4227347389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b07900 0x561090b09d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.620+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/4227347389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b02790 0x561090b07290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.620+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/4227347389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b01da0 0x561090b021c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.620+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/4227347389 >> 172.21.15.97:0/4227347389 conn(0x561090afd310 msgr2=0x561090afd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.620+0000 7f3ba5000640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b840a3040 0x7f3b840a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.621+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/4227347389 shutdown_connections 2024-03-26T19:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.621+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/4227347389 wait complete. 2024-03-26T19:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.621+0000 7f3ba9bf6740 1 -- start start 2024-03-26T19:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.622+0000 7f3ba5000640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b840a3040 0x7f3b840a5500 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b98005fd0 tx=0x7f3b98000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.622+0000 7f3ba9bf6740 1 Processor -- start 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.622+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b0bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.622+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b0c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.622+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b13330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.622+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561090b0a8b0 con 0x561090b01da0 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.622+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561090a22ac0 con 0x561090b02790 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.622+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561090b0aa30 con 0x561090b07900 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3ba9bf6740 10 client.0 _ll_register_callbacks cb 0x561090a22490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3b9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b0bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3b9f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b0c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3b9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b0bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47074/0 (socket says 172.21.15.97:47074) 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3ba5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b13330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3b9fe00640 1 -- 172.21.15.97:0/3878338956 learned_addr learned my addr 172.21.15.97:0/3878338956 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3b9f400640 1 -- 172.21.15.97:0/3878338956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 msgr2=0x561090b13330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3b9f400640 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b13330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3b9f400640 1 -- 172.21.15.97:0/3878338956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 msgr2=0x561090b0bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3b9f400640 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b0bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.623+0000 7f3b9f400640 1 -- 172.21.15.97:0/3878338956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561090847620 con 0x561090b02790 2024-03-26T19:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.625+0000 7f3b9f400640 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b0c2f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c09bdf0 tx=0x7f3b8c09c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de39a879bdea3a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.625+0000 7f3b8a000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.626+0000 7f3b8a000640 1 -- 172.21.15.97:0/3878338956 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0bae60 con 0x561090b02790 2024-03-26T19:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.626+0000 7f3b8a000640 1 -- 172.21.15.97:0/3878338956 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3b8c0bc2c0 con 0x561090b02790 2024-03-26T19:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.626+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3878338956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b98003d10 con 0x561090b02790 2024-03-26T19:14:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.627+0000 7f3b8a000640 1 -- 172.21.15.97:0/3878338956 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3b8c0b5220 con 0x561090b02790 2024-03-26T19:14:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.627+0000 7f3ba9bf6740 10 client.26390 fetch_fsmap learned FSMap version 29 2024-03-26T19:14:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.627+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3878338956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561090b16710 con 0x561090b02790 2024-03-26T19:14:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.627+0000 7f3b8a000640 1 -- 172.21.15.97:0/3878338956 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b98003d10 con 0x561090b02790 2024-03-26T19:14:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.628+0000 7f3b8a000640 1 -- 172.21.15.97:0/3878338956 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3b8c09b540 con 0x561090b02790 2024-03-26T19:14:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.628+0000 7f3ba9bf6740 10 client.26390 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T19:14:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.628+0000 7f3ba9bf6740 10 client.26390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:14:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.628+0000 7f3ba9bf6740 20 client.26390 populate_metadata read hostname 'smithi097' 2024-03-26T19:14:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.628+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561090b17d60 0x561090b1a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.628+0000 7f3ba9bf6740 4 client.26390 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:14:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.628+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3878338956 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561090a743c0 con 0x561090b17d60 2024-03-26T19:14:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.628+0000 7f3b9fe00640 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561090b17d60 0x561090b1a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.629+0000 7f3b9fe00640 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561090b17d60 0x561090b1a170 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc0f7a6c4847695 server_cookie=98d988657fafc62b in_seq=0 out_seq=0 2024-03-26T19:14:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.629+0000 7f3b8a000640 10 client.26390 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3b8a000640 1 -- 172.21.15.97:0/3878338956 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561090a743c0 con 0x561090b17d60 2024-03-26T19:14:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3b8a000640 10 client.26390 handle_command_reply: tid=0 2024-03-26T19:14:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 client.26390 shutdown 2024-03-26T19:14:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3878338956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 msgr2=0x561090b0c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b0c2f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c09bdf0 tx=0x7f3b8c09c7f0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3878338956 shutdown_connections 2024-03-26T19:14:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561090b17d60 0x561090b1a170 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b13330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b0c2f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3878338956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b0bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3878338956 >> 172.21.15.97:0/3878338956 conn(0x561090afd310 msgr2=0x561090afc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3b8a000640 0 client.26390 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3878338956 shutdown_connections 2024-03-26T19:14:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.630+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3878338956 wait complete. 2024-03-26T19:14:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.631+0000 7f3ba9bf6740 20 client.26390 trim_cache size 0 max 16384 2024-03-26T19:14:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.655+0000 7f3ba9bf6740 1 Processor -- start 2024-03-26T19:14:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.655+0000 7f3ba9bf6740 1 -- start start 2024-03-26T19:14:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.655+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b1a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.655+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b132f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.655+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b00c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.655+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01af440 con 0x561090b01da0 2024-03-26T19:14:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.655+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01b0f80 con 0x561090b02790 2024-03-26T19:14:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.655+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01afb10 con 0x561090b07900 2024-03-26T19:14:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3b9f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3b9f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b132f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:59116/0 (socket says 172.21.15.97:59116) 2024-03-26T19:14:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3b9f400640 1 -- 172.21.15.97:0/253983436 learned_addr learned my addr 172.21.15.97:0/253983436 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3b9fe00640 1 --2- 172.21.15.97:0/253983436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b1a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3ba5000640 1 --2- 172.21.15.97:0/253983436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b00c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3ba5000640 1 -- 172.21.15.97:0/253983436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 msgr2=0x561090b132f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3ba5000640 1 --2- 172.21.15.97:0/253983436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b132f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3ba5000640 1 -- 172.21.15.97:0/253983436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 msgr2=0x561090b1a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3ba5000640 1 --2- 172.21.15.97:0/253983436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b1a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3ba5000640 1 -- 172.21.15.97:0/253983436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561090b02b70 con 0x561090b07900 2024-03-26T19:14:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.656+0000 7f3b9f400640 1 --2- 172.21.15.97:0/253983436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:14:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.657+0000 7f3ba5000640 1 --2- 172.21.15.97:0/253983436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b00c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009f60 tx=0x7f3b98024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f283a286f63bdaeb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.657+0000 7f3b94c00640 1 -- 172.21.15.97:0/253983436 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98037070 con 0x561090b07900 2024-03-26T19:14:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.657+0000 7f3b94c00640 1 -- 172.21.15.97:0/253983436 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b98012d10 con 0x561090b07900 2024-03-26T19:14:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.657+0000 7f3b94c00640 1 -- 172.21.15.97:0/253983436 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98029660 con 0x561090b07900 2024-03-26T19:14:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.658+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/253983436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 msgr2=0x561090b00c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.658+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/253983436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b00c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009f60 tx=0x7f3b98024000 comp rx=0 tx=0).stop 2024-03-26T19:14:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.658+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/253983436 shutdown_connections 2024-03-26T19:14:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.658+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/253983436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b00c80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.658+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/253983436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.658+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/253983436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b1a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.658+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/253983436 >> 172.21.15.97:0/253983436 conn(0x561090afd310 msgr2=0x561090b00570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.658+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/253983436 shutdown_connections 2024-03-26T19:14:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.658+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/253983436 wait complete. 2024-03-26T19:14:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.658+0000 7f3ba9bf6740 1 -- start start 2024-03-26T19:14:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.659+0000 7f3ba9bf6740 1 Processor -- start 2024-03-26T19:14:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.659+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b10830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.659+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b10d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.659+0000 7f3ba9bf6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b11d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.659+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01b08b0 con 0x561090b01da0 2024-03-26T19:14:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.659+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01b01e0 con 0x561090b02790 2024-03-26T19:14:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.659+0000 7f3ba9bf6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01ad3e0 con 0x561090b07900 2024-03-26T19:14:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.659+0000 7f3ba9bf6740 10 client.0 _ll_register_callbacks cb 0x7f3b8c09bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3ba5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b11d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3b9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b10830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3ba5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b11d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:40560/0 (socket says 172.21.15.97:40560) 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3ba5000640 1 -- 172.21.15.97:0/3901547917 learned_addr learned my addr 172.21.15.97:0/3901547917 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3b9f400640 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b10d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3ba5000640 1 -- 172.21.15.97:0/3901547917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 msgr2=0x561090b10d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3ba5000640 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b10d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3ba5000640 1 -- 172.21.15.97:0/3901547917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 msgr2=0x561090b10830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3ba5000640 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b10830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3ba5000640 1 -- 172.21.15.97:0/3901547917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561090b12370 con 0x561090b07900 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3b9fe00640 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b10830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3ba5000640 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b11d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009f90 tx=0x7f3b98004e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1a76afa8626e5ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.660+0000 7f3b8a000640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T19:14:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.661+0000 7f3b8a000640 1 -- 172.21.15.97:0/3901547917 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98040060 con 0x561090b07900 2024-03-26T19:14:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.661+0000 7f3b8a000640 1 -- 172.21.15.97:0/3901547917 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3b98003b80 con 0x561090b07900 2024-03-26T19:14:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.661+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3901547917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b98012d10 con 0x561090b07900 2024-03-26T19:14:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.662+0000 7f3b8a000640 1 -- 172.21.15.97:0/3901547917 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3b98037030 con 0x561090b07900 2024-03-26T19:14:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.662+0000 7f3ba9bf6740 10 client.26386 fetch_fsmap learned FSMap version 29 2024-03-26T19:14:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.662+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3901547917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561090b7e0b0 con 0x561090b07900 2024-03-26T19:14:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.662+0000 7f3b8a000640 1 -- 172.21.15.97:0/3901547917 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b98012d10 con 0x561090b07900 2024-03-26T19:14:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.663+0000 7f3b8a000640 1 -- 172.21.15.97:0/3901547917 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3b98004260 con 0x561090b07900 2024-03-26T19:14:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.663+0000 7f3ba9bf6740 10 client.26386 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T19:14:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.663+0000 7f3ba9bf6740 10 client.26386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:14:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.663+0000 7f3ba9bf6740 20 client.26386 populate_metadata read hostname 'smithi097' 2024-03-26T19:14:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.663+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561090b7e7b0 0x561090b80bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.663+0000 7f3ba9bf6740 4 client.26386 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:14:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.663+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3901547917 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561090a6e710 con 0x561090b7e7b0 2024-03-26T19:14:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.663+0000 7f3b9fe00640 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561090b7e7b0 0x561090b80bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.665+0000 7f3b9fe00640 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561090b7e7b0 0x561090b80bd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c23ba242aac517 server_cookie=4e98a8b656777f79 in_seq=0 out_seq=0 2024-03-26T19:14:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.665+0000 7f3b8a000640 10 client.26386 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.695+0000 7f3b8a000640 1 -- 172.21.15.97:0/3901547917 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561090a6e710 con 0x561090b7e7b0 2024-03-26T19:14:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.695+0000 7f3b8a000640 10 client.26386 handle_command_reply: tid=0 2024-03-26T19:14:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.695+0000 7f3ba9bf6740 1 client.26386 shutdown 2024-03-26T19:14:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3901547917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 msgr2=0x561090b11d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b11d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009f90 tx=0x7f3b98004e90 comp rx=0 tx=0).stop 2024-03-26T19:14:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3901547917 shutdown_connections 2024-03-26T19:14:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x561090b7e7b0 0x561090b80bd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3b8a000640 0 client.26386 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561090b07900 0x561090b11d80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561090b02790 0x561090b10d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3ba9bf6740 1 --2- 172.21.15.97:0/3901547917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561090b01da0 0x561090b10830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3901547917 >> 172.21.15.97:0/3901547917 conn(0x561090afd310 msgr2=0x561090b00e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3901547917 shutdown_connections 2024-03-26T19:14:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.696+0000 7f3ba9bf6740 1 -- 172.21.15.97:0/3901547917 wait complete. 2024-03-26T19:14:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.697+0000 7f3ba9bf6740 20 client.26386 trim_cache size 0 max 16384 2024-03-26T19:14:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:14:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:14:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:14:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:14:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 -- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b840a3040 msgr2=0x7f3b840a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b840a3040 0x7f3b840a5500 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b98005fd0 tx=0x7f3b98000f40 comp rx=0 tx=0).stop 2024-03-26T19:14:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 -- 172.21.15.97:0/2628909497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ba00774c0 msgr2=0x7f3ba019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ba00774c0 0x7f3ba019e700 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00e9f0 tx=0x7f3b8c00eeb0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 -- 172.21.15.97:0/2628909497 shutdown_connections 2024-03-26T19:14:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f3b840a3040 0x7f3b840a5500 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0077e60 0x7f3ba01a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ba00774c0 0x7f3ba019e700 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 --2- 172.21.15.97:0/2628909497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba0076280 0x7f3ba019e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 -- 172.21.15.97:0/2628909497 >> 172.21.15.97:0/2628909497 conn(0x7f3ba00ff0b0 msgr2=0x7f3ba0100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 -- 172.21.15.97:0/2628909497 shutdown_connections 2024-03-26T19:14:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:20.698+0000 7f3ba7000640 1 -- 172.21.15.97:0/2628909497 wait complete. 2024-03-26T19:14:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:20 smithi097 ceph-mon[31955]: pgmap v853: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-03-26T19:14:20.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:20 smithi144 ceph-mon[36053]: pgmap v853: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-03-26T19:14:20.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:20 smithi105 ceph-mon[36270]: pgmap v853: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-03-26T19:14:22.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:22 smithi097 ceph-mon[31955]: pgmap v854: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 673 op/s 2024-03-26T19:14:22.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:22 smithi144 ceph-mon[36053]: pgmap v854: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 673 op/s 2024-03-26T19:14:22.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:22 smithi105 ceph-mon[36270]: pgmap v854: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 673 op/s 2024-03-26T19:14:24.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:24 smithi097 ceph-mon[31955]: pgmap v855: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 625 op/s 2024-03-26T19:14:24.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:24 smithi144 ceph-mon[36053]: pgmap v855: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 625 op/s 2024-03-26T19:14:24.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:24 smithi105 ceph-mon[36270]: pgmap v855: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 625 op/s 2024-03-26T19:14:25.030 INFO:teuthology.orchestra.run.smithi097.stderr:progress: 900.0 s, 1162.8 tps, lat 88.004 ms stddev 111.222 2024-03-26T19:14:25.364 INFO:teuthology.orchestra.run.smithi097.stdout:transaction type: 2024-03-26T19:14:25.364 INFO:teuthology.orchestra.run.smithi097.stdout:scaling factor: 32 2024-03-26T19:14:25.364 INFO:teuthology.orchestra.run.smithi097.stdout:query mode: simple 2024-03-26T19:14:25.364 INFO:teuthology.orchestra.run.smithi097.stdout:number of clients: 100 2024-03-26T19:14:25.364 INFO:teuthology.orchestra.run.smithi097.stdout:number of threads: 4 2024-03-26T19:14:25.364 INFO:teuthology.orchestra.run.smithi097.stdout:duration: 900 s 2024-03-26T19:14:25.364 INFO:teuthology.orchestra.run.smithi097.stdout:number of transactions actually processed: 1649027 2024-03-26T19:14:25.365 INFO:teuthology.orchestra.run.smithi097.stdout:latency average = 54.561 ms 2024-03-26T19:14:25.365 INFO:teuthology.orchestra.run.smithi097.stdout:latency stddev = 72.343 ms 2024-03-26T19:14:25.365 INFO:teuthology.orchestra.run.smithi097.stdout:tps = 1831.573243 (including connections establishing) 2024-03-26T19:14:25.365 INFO:teuthology.orchestra.run.smithi097.stdout:tps = 1831.588808 (excluding connections establishing) 2024-03-26T19:14:25.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-03-26T19:14:26.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:26 smithi097 ceph-mon[31955]: pgmap v856: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 644 op/s 2024-03-26T19:14:26.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:26 smithi144 ceph-mon[36053]: pgmap v856: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 644 op/s 2024-03-26T19:14:26.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:26 smithi105 ceph-mon[36270]: pgmap v856: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 644 op/s 2024-03-26T19:14:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:28 smithi097 ceph-mon[31955]: pgmap v857: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 685 op/s 2024-03-26T19:14:28.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:28 smithi144 ceph-mon[36053]: pgmap v857: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 685 op/s 2024-03-26T19:14:28.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:28 smithi105 ceph-mon[36270]: pgmap v857: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 685 op/s 2024-03-26T19:14:30.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:30 smithi097 ceph-mon[31955]: pgmap v858: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 656 op/s 2024-03-26T19:14:30.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:30 smithi144 ceph-mon[36053]: pgmap v858: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 656 op/s 2024-03-26T19:14:30.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:30 smithi105 ceph-mon[36270]: pgmap v858: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 656 op/s 2024-03-26T19:14:32.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:32 smithi097 ceph-mon[31955]: pgmap v859: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 739 op/s 2024-03-26T19:14:32.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:32 smithi144 ceph-mon[36053]: pgmap v859: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 739 op/s 2024-03-26T19:14:32.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:32 smithi105 ceph-mon[36270]: pgmap v859: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 739 op/s 2024-03-26T19:14:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.891+0000 7f5b40e00640 1 -- 172.21.15.97:0/1615164032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3c109070 msgr2=0x7f5b3c1094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.891+0000 7f5b40e00640 1 --2- 172.21.15.97:0/1615164032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3c109070 0x7f5b3c1094f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009900 tx=0x7f5b2802bc40 comp rx=0 tx=0).stop 2024-03-26T19:14:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.892+0000 7f5b40e00640 1 -- 172.21.15.97:0/1615164032 shutdown_connections 2024-03-26T19:14:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.892+0000 7f5b40e00640 1 --2- 172.21.15.97:0/1615164032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b3c109a30 0x7f5b3c110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.892+0000 7f5b40e00640 1 --2- 172.21.15.97:0/1615164032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3c109070 0x7f5b3c1094f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.892+0000 7f5b40e00640 1 --2- 172.21.15.97:0/1615164032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c107f20 0x7f5b3c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.892+0000 7f5b40e00640 1 -- 172.21.15.97:0/1615164032 >> 172.21.15.97:0/1615164032 conn(0x7f5b3c076ed0 msgr2=0x7f5b3c0792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.894+0000 7f5b40e00640 1 -- 172.21.15.97:0/1615164032 shutdown_connections 2024-03-26T19:14:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.896+0000 7f5b40e00640 1 -- 172.21.15.97:0/1615164032 wait complete. 2024-03-26T19:14:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.897+0000 7f5b40e00640 1 Processor -- start 2024-03-26T19:14:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.897+0000 7f5b40e00640 1 -- start start 2024-03-26T19:14:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.898+0000 7f5b40e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b3c107f20 0x7f5b3c06b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.898+0000 7f5b40e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3c109070 0x7f5b3c06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.898+0000 7f5b40e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c109a30 0x7f5b3c072c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.898+0000 7f5b40e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c113c00 con 0x7f5b3c107f20 2024-03-26T19:14:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.898+0000 7f5b40e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c113a80 con 0x7f5b3c109070 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.898+0000 7f5b40e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c113d80 con 0x7f5b3c109a30 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.898+0000 7f5b3a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c109a30 0x7f5b3c072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.898+0000 7f5b3a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c109a30 0x7f5b3c072c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50098/0 (socket says 172.21.15.97:50098) 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.898+0000 7f5b3a800640 1 -- 172.21.15.97:0/2665318707 learned_addr learned my addr 172.21.15.97:0/2665318707 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.898+0000 7f5b39400640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3c109070 0x7f5b3c06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.899+0000 7f5b3a800640 1 -- 172.21.15.97:0/2665318707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3c109070 msgr2=0x7f5b3c06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.899+0000 7f5b3a800640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3c109070 0x7f5b3c06bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.899+0000 7f5b3a800640 1 -- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b3c107f20 msgr2=0x7f5b3c06b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.899+0000 7f5b39e00640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b3c107f20 0x7f5b3c06b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.899+0000 7f5b3a800640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b3c107f20 0x7f5b3c06b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.899+0000 7f5b3a800640 1 -- 172.21.15.97:0/2665318707 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c073300 con 0x7f5b3c109a30 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.899+0000 7f5b39e00640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b3c107f20 0x7f5b3c06b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T19:14:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.900+0000 7f5b3a800640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c109a30 0x7f5b3c072c90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400eaa0 tx=0x7f5b3400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.900+0000 7f5b32a00640 1 -- 172.21.15.97:0/2665318707 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400cdb0 con 0x7f5b3c109a30 2024-03-26T19:14:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.900+0000 7f5b32a00640 1 -- 172.21.15.97:0/2665318707 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b34005020 con 0x7f5b3c109a30 2024-03-26T19:14:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.900+0000 7f5b32a00640 1 -- 172.21.15.97:0/2665318707 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b340116b0 con 0x7f5b3c109a30 2024-03-26T19:14:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.900+0000 7f5b40e00640 1 -- 172.21.15.97:0/2665318707 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c0735f0 con 0x7f5b3c109a30 2024-03-26T19:14:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.901+0000 7f5b40e00640 1 -- 172.21.15.97:0/2665318707 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c1aef40 con 0x7f5b3c109a30 2024-03-26T19:14:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.905+0000 7f5b43b95740 1 Processor -- start 2024-03-26T19:14:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.908+0000 7f5b32a00640 1 -- 172.21.15.97:0/2665318707 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f5b34005560 con 0x7f5b3c109a30 2024-03-26T19:14:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.908+0000 7f5b43b95740 1 -- start start 2024-03-26T19:14:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.909+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba5192da0 0x55dba51931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.909+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5193790 0x55dba5198290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.909+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5198900 0x55dba519ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.909+0000 7f5b43b95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dba50f7e80 con 0x55dba5198900 2024-03-26T19:14:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.909+0000 7f5b43b95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dba50a0660 con 0x55dba5192da0 2024-03-26T19:14:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.909+0000 7f5b43b95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dba50f3e30 con 0x55dba5193790 2024-03-26T19:14:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.909+0000 7f5b39400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5198900 0x55dba519ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.909+0000 7f5b39400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5198900 0x55dba519ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49750/0 (socket says 172.21.15.97:49750) 2024-03-26T19:14:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.909+0000 7f5b39400640 1 -- 172.21.15.97:0/3980954257 learned_addr learned my addr 172.21.15.97:0/3980954257 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.910+0000 7f5b39e00640 1 --2- 172.21.15.97:0/3980954257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5193790 0x55dba5198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.910+0000 7f5b39400640 1 --2- 172.21.15.97:0/3980954257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba5192da0 0x55dba51931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.910+0000 7f5b32a00640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5b200a3040 0x7f5b200a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.910+0000 7f5b39e00640 1 -- 172.21.15.97:0/3980954257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba5192da0 msgr2=0x55dba51931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.910+0000 7f5b39e00640 1 --2- 172.21.15.97:0/3980954257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba5192da0 0x55dba51931c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.910+0000 7f5b39e00640 1 -- 172.21.15.97:0/3980954257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5198900 msgr2=0x55dba519ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.910+0000 7f5b39e00640 1 --2- 172.21.15.97:0/3980954257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5198900 0x55dba519ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.910+0000 7f5b39e00640 1 -- 172.21.15.97:0/3980954257 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba4e0c210 con 0x55dba5193790 2024-03-26T19:14:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.910+0000 7f5b32a00640 1 -- 172.21.15.97:0/2665318707 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5b340cf830 con 0x7f5b3c109a30 2024-03-26T19:14:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.911+0000 7f5b39e00640 1 --2- 172.21.15.97:0/3980954257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5193790 0x55dba5198290 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009b90 tx=0x7f5b2c00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8573591d99f42f75 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.911+0000 7f5b3a800640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5b200a3040 0x7f5b200a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.912+0000 7f5b27400640 1 -- 172.21.15.97:0/3980954257 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c016070 con 0x55dba5193790 2024-03-26T19:14:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.913+0000 7f5b27400640 1 -- 172.21.15.97:0/3980954257 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2c00ee90 con 0x55dba5193790 2024-03-26T19:14:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.913+0000 7f5b27400640 1 -- 172.21.15.97:0/3980954257 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c011470 con 0x55dba5193790 2024-03-26T19:14:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.913+0000 7f5b3a800640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5b200a3040 0x7f5b200a5500 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5b3409bf20 tx=0x7f5b34005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.914+0000 7f5b43b95740 1 -- 172.21.15.97:0/3980954257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5193790 msgr2=0x55dba5198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.914+0000 7f5b43b95740 1 --2- 172.21.15.97:0/3980954257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5193790 0x55dba5198290 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009b90 tx=0x7f5b2c00e6c0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.915+0000 7f5b43b95740 1 -- 172.21.15.97:0/3980954257 shutdown_connections 2024-03-26T19:14:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.915+0000 7f5b43b95740 1 --2- 172.21.15.97:0/3980954257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5198900 0x55dba519ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.915+0000 7f5b43b95740 1 --2- 172.21.15.97:0/3980954257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5193790 0x55dba5198290 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.915+0000 7f5b43b95740 1 --2- 172.21.15.97:0/3980954257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba5192da0 0x55dba51931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.915+0000 7f5b43b95740 1 -- 172.21.15.97:0/3980954257 >> 172.21.15.97:0/3980954257 conn(0x55dba518e310 msgr2=0x55dba518e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.915+0000 7f5b43b95740 1 -- 172.21.15.97:0/3980954257 shutdown_connections 2024-03-26T19:14:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.915+0000 7f5b43b95740 1 -- 172.21.15.97:0/3980954257 wait complete. 2024-03-26T19:14:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.915+0000 7f5b43b95740 1 -- start start 2024-03-26T19:14:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.916+0000 7f5b43b95740 1 Processor -- start 2024-03-26T19:14:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.920+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5192da0 0x55dba519cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.920+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5198900 0x55dba519d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.920+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 0x55dba51a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.921+0000 7f5b39e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5198900 0x55dba519d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.921+0000 7f5b43b95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dba519b8b0 con 0x55dba5192da0 2024-03-26T19:14:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.921+0000 7f5b43b95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dba50b3ac0 con 0x55dba51a24d0 2024-03-26T19:14:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.921+0000 7f5b43b95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dba519ba30 con 0x55dba5198900 2024-03-26T19:14:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.921+0000 7f5b43b95740 10 client.0 _ll_register_callbacks cb 0x55dba50b3490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:14:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.921+0000 7f5b39400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5192da0 0x55dba519cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.921+0000 7f5b39400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5192da0 0x55dba519cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49758/0 (socket says 172.21.15.97:49758) 2024-03-26T19:14:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.921+0000 7f5b39400640 1 -- 172.21.15.97:0/4207632437 learned_addr learned my addr 172.21.15.97:0/4207632437 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.922+0000 7f5b39400640 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 0x55dba51a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.923+0000 7f5b39400640 1 -- 172.21.15.97:0/4207632437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5198900 msgr2=0x55dba519d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.923+0000 7f5b39400640 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5198900 0x55dba519d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.923+0000 7f5b39400640 1 -- 172.21.15.97:0/4207632437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 msgr2=0x55dba51a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.923+0000 7f5b39400640 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 0x55dba51a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.923+0000 7f5b39400640 1 -- 172.21.15.97:0/4207632437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba4ed8620 con 0x55dba5192da0 2024-03-26T19:14:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.923+0000 7f5b39e00640 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5198900 0x55dba519d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:14:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.924+0000 7f5b39400640 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5192da0 0x55dba519cf60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b28036040 tx=0x7f5b2802bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33df8491df73c0b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.924+0000 7f5b1be00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-26T19:14:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.925+0000 7f5b1be00640 1 -- 172.21.15.97:0/4207632437 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28005230 con 0x55dba5192da0 2024-03-26T19:14:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.925+0000 7f5b1be00640 1 -- 172.21.15.97:0/4207632437 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5b280053d0 con 0x55dba5192da0 2024-03-26T19:14:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.925+0000 7f5b43b95740 1 -- 172.21.15.97:0/4207632437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b2c00ee90 con 0x55dba5192da0 2024-03-26T19:14:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.926+0000 7f5b1be00640 1 -- 172.21.15.97:0/4207632437 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b28003880 con 0x55dba5192da0 2024-03-26T19:14:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.926+0000 7f5b43b95740 10 client.15924 fetch_fsmap learned FSMap version 29 2024-03-26T19:14:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.926+0000 7f5b43b95740 1 -- 172.21.15.97:0/4207632437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dba51a7ca0 con 0x55dba5192da0 2024-03-26T19:14:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.927+0000 7f5b1be00640 1 -- 172.21.15.97:0/4207632437 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2c00ee90 con 0x55dba5192da0 2024-03-26T19:14:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.927+0000 7f5b1be00640 1 -- 172.21.15.97:0/4207632437 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5b28034440 con 0x55dba5192da0 2024-03-26T19:14:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.928+0000 7f5b43b95740 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T19:14:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.928+0000 7f5b43b95740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:14:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.928+0000 7f5b43b95740 20 client.15924 populate_metadata read hostname 'smithi097' 2024-03-26T19:14:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.929+0000 7f5b43b95740 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dba51a9370 0x55dba51ab780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.929+0000 7f5b43b95740 4 client.15924 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T19:14:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.929+0000 7f5b43b95740 1 -- 172.21.15.97:0/4207632437 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dba51053c0 con 0x55dba51a9370 2024-03-26T19:14:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.929+0000 7f5b39e00640 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dba51a9370 0x55dba51ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.931+0000 7f5b1be00640 10 client.15924 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.931+0000 7f5b39e00640 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dba51a9370 0x55dba51ab780 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c021cc60d56f7cc server_cookie=f809c14b30bf52e3 in_seq=0 out_seq=0 2024-03-26T19:14:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.935+0000 7f5b1be00640 1 -- 172.21.15.97:0/4207632437 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dba51053c0 con 0x55dba51a9370 2024-03-26T19:14:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.935+0000 7f5b1be00640 10 client.15924 handle_command_reply: tid=0 2024-03-26T19:14:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.935+0000 7f5b43b95740 1 client.15924 shutdown 2024-03-26T19:14:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.936+0000 7f5b43b95740 1 -- 172.21.15.97:0/4207632437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5192da0 msgr2=0x55dba519cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.936+0000 7f5b43b95740 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5192da0 0x55dba519cf60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b28036040 tx=0x7f5b2802bc40 comp rx=0 tx=0).stop 2024-03-26T19:14:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.936+0000 7f5b43b95740 1 -- 172.21.15.97:0/4207632437 shutdown_connections 2024-03-26T19:14:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.936+0000 7f5b43b95740 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dba51a9370 0x55dba51ab780 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.936+0000 7f5b43b95740 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 0x55dba51a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.936+0000 7f5b1be00640 0 client.15924 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.936+0000 7f5b43b95740 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5198900 0x55dba519d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.936+0000 7f5b43b95740 1 --2- 172.21.15.97:0/4207632437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba5192da0 0x55dba519cf60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.936+0000 7f5b43b95740 1 -- 172.21.15.97:0/4207632437 >> 172.21.15.97:0/4207632437 conn(0x55dba518e310 msgr2=0x55dba518d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.937+0000 7f5b43b95740 1 -- 172.21.15.97:0/4207632437 shutdown_connections 2024-03-26T19:14:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.937+0000 7f5b43b95740 1 -- 172.21.15.97:0/4207632437 wait complete. 2024-03-26T19:14:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.937+0000 7f5b43b95740 20 client.15924 trim_cache size 0 max 16384 2024-03-26T19:14:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.967+0000 7f5b43b95740 1 Processor -- start 2024-03-26T19:14:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.967+0000 7f5b43b95740 1 -- start start 2024-03-26T19:14:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.968+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5192da0 0x55dba519d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.968+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba5198900 0x55dba5191c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.968+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba51a24d0 0x55dba5194230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.968+0000 7f5b43b95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b7b20 con 0x55dba51a24d0 2024-03-26T19:14:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.968+0000 7f5b43b95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b9660 con 0x55dba5198900 2024-03-26T19:14:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.968+0000 7f5b43b95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b81f0 con 0x55dba5192da0 2024-03-26T19:14:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.968+0000 7f5b39400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5192da0 0x55dba519d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.968+0000 7f5b39400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5192da0 0x55dba519d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50122/0 (socket says 172.21.15.97:50122) 2024-03-26T19:14:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.968+0000 7f5b39400640 1 -- 172.21.15.97:0/2513401937 learned_addr learned my addr 172.21.15.97:0/2513401937 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.969+0000 7f5b39e00640 1 --2- 172.21.15.97:0/2513401937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba5198900 0x55dba5191c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.969+0000 7f5b39400640 1 --2- 172.21.15.97:0/2513401937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba51a24d0 0x55dba5194230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.969+0000 7f5b39400640 1 -- 172.21.15.97:0/2513401937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba5198900 msgr2=0x55dba5191c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.969+0000 7f5b39400640 1 --2- 172.21.15.97:0/2513401937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba5198900 0x55dba5191c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.969+0000 7f5b39400640 1 -- 172.21.15.97:0/2513401937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba51a24d0 msgr2=0x55dba5194230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T19:14:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.969+0000 7f5b39400640 1 --2- 172.21.15.97:0/2513401937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba51a24d0 0x55dba5194230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.969+0000 7f5b39400640 1 -- 172.21.15.97:0/2513401937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba519c030 con 0x55dba5192da0 2024-03-26T19:14:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.970+0000 7f5b39400640 1 --2- 172.21.15.97:0/2513401937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5192da0 0x55dba519d400 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5b2802c180 tx=0x7f5b28005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8526cf535988dd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.971+0000 7f5b26a00640 1 -- 172.21.15.97:0/2513401937 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28034640 con 0x55dba5192da0 2024-03-26T19:14:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.971+0000 7f5b26a00640 1 -- 172.21.15.97:0/2513401937 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b28005030 con 0x55dba5192da0 2024-03-26T19:14:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.971+0000 7f5b26a00640 1 -- 172.21.15.97:0/2513401937 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28004ba0 con 0x55dba5192da0 2024-03-26T19:14:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.972+0000 7f5b43b95740 1 -- 172.21.15.97:0/2513401937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5192da0 msgr2=0x55dba519d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.972+0000 7f5b43b95740 1 --2- 172.21.15.97:0/2513401937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5192da0 0x55dba519d400 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5b2802c180 tx=0x7f5b28005fd0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.972+0000 7f5b43b95740 1 -- 172.21.15.97:0/2513401937 shutdown_connections 2024-03-26T19:14:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.972+0000 7f5b43b95740 1 --2- 172.21.15.97:0/2513401937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba51a24d0 0x55dba5194230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.972+0000 7f5b43b95740 1 --2- 172.21.15.97:0/2513401937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba5198900 0x55dba5191c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.972+0000 7f5b43b95740 1 --2- 172.21.15.97:0/2513401937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5192da0 0x55dba519d400 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5b2802c180 tx=0x7f5b28005fd0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.972+0000 7f5b43b95740 1 -- 172.21.15.97:0/2513401937 >> 172.21.15.97:0/2513401937 conn(0x55dba518e310 msgr2=0x55dba5191570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.972+0000 7f5b43b95740 1 -- 172.21.15.97:0/2513401937 shutdown_connections 2024-03-26T19:14:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.973+0000 7f5b43b95740 1 -- 172.21.15.97:0/2513401937 wait complete. 2024-03-26T19:14:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.973+0000 7f5b43b95740 1 -- start start 2024-03-26T19:14:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.974+0000 7f5b43b95740 1 Processor -- start 2024-03-26T19:14:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.974+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5198900 0x55dba51ba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.974+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 0x55dba51bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.975+0000 7f5b43b95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba51bbd50 0x55dba51bc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.975+0000 7f5b43b95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b8f90 con 0x55dba51bbd50 2024-03-26T19:14:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.975+0000 7f5b43b95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b88c0 con 0x55dba51a24d0 2024-03-26T19:14:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.975+0000 7f5b43b95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b5b00 con 0x55dba5198900 2024-03-26T19:14:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.975+0000 7f5b43b95740 10 client.0 _ll_register_callbacks cb 0x7f5b28004130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T19:14:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.975+0000 7f5b39e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 0x55dba51bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.975+0000 7f5b39e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 0x55dba51bad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34302/0 (socket says 172.21.15.97:34302) 2024-03-26T19:14:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.975+0000 7f5b39e00640 1 -- 172.21.15.97:0/2460349770 learned_addr learned my addr 172.21.15.97:0/2460349770 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.976+0000 7f5b39e00640 1 -- 172.21.15.97:0/2460349770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5198900 msgr2=0x55dba51ba850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T19:14:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.976+0000 7f5b39e00640 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5198900 0x55dba51ba850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.976+0000 7f5b39e00640 1 -- 172.21.15.97:0/2460349770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba51bbd50 msgr2=0x55dba51bc1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T19:14:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.976+0000 7f5b39400640 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba51bbd50 0x55dba51bc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.976+0000 7f5b39e00640 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba51bbd50 0x55dba51bc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.976+0000 7f5b39e00640 1 -- 172.21.15.97:0/2460349770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba51bc710 con 0x55dba51a24d0 2024-03-26T19:14:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.977+0000 7f5b39e00640 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 0x55dba51bad90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009b60 tx=0x7f5b2c005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb7d5488f47febd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.977+0000 7f5b1be00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T19:14:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.977+0000 7f5b1be00640 1 -- 172.21.15.97:0/2460349770 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c016040 con 0x55dba51a24d0 2024-03-26T19:14:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.977+0000 7f5b43b95740 1 -- 172.21.15.97:0/2460349770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b28005030 con 0x55dba51a24d0 2024-03-26T19:14:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.978+0000 7f5b1be00640 1 -- 172.21.15.97:0/2460349770 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5b2c026da0 con 0x55dba51a24d0 2024-03-26T19:14:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.978+0000 7f5b1be00640 1 -- 172.21.15.97:0/2460349770 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b2c00ee90 con 0x55dba51a24d0 2024-03-26T19:14:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.978+0000 7f5b43b95740 10 client.26414 fetch_fsmap learned FSMap version 29 2024-03-26T19:14:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.978+0000 7f5b43b95740 1 -- 172.21.15.97:0/2460349770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dba520fd90 con 0x55dba51a24d0 2024-03-26T19:14:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.979+0000 7f5b1be00640 1 -- 172.21.15.97:0/2460349770 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b28005030 con 0x55dba51a24d0 2024-03-26T19:14:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.979+0000 7f5b1be00640 1 -- 172.21.15.97:0/2460349770 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5b2802cbd0 con 0x55dba51a24d0 2024-03-26T19:14:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.979+0000 7f5b43b95740 10 client.26414 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T19:14:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.979+0000 7f5b43b95740 10 client.26414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T19:14:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.979+0000 7f5b43b95740 20 client.26414 populate_metadata read hostname 'smithi097' 2024-03-26T19:14:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.979+0000 7f5b43b95740 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dba5210370 0x55dba5212760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.979+0000 7f5b43b95740 4 client.26414 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T19:14:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.979+0000 7f5b43b95740 1 -- 172.21.15.97:0/2460349770 --> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dba50ff710 con 0x55dba5210370 2024-03-26T19:14:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.980+0000 7f5b39400640 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dba5210370 0x55dba5212760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.981+0000 7f5b39400640 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dba5210370 0x55dba5212760 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99c6a0131793b1c server_cookie=1a79878a577d6a58 in_seq=0 out_seq=0 2024-03-26T19:14:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:33.981+0000 7f5b1be00640 10 client.26414 ms_handle_connect on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.478+0000 7f5b1be00640 1 -- 172.21.15.97:0/2460349770 <== mds.0 v2:172.21.15.105:6838/2391073232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dba50ff710 con 0x55dba5210370 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.478+0000 7f5b1be00640 10 client.26414 handle_command_reply: tid=0 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.478+0000 7f5b43b95740 1 client.26414 shutdown 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 1 -- 172.21.15.97:0/2460349770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 msgr2=0x55dba51bad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 0x55dba51bad90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009b60 tx=0x7f5b2c005780 comp rx=0 tx=0).stop 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 1 -- 172.21.15.97:0/2460349770 shutdown_connections 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.105:6838/2391073232,v1:172.21.15.105:6839/2391073232] conn(0x55dba5210370 0x55dba5212760 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dba51bbd50 0x55dba51bc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba51a24d0 0x55dba51bad90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 1 --2- 172.21.15.97:0/2460349770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba5198900 0x55dba51ba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b1be00640 0 client.26414 ms_handle_reset on v2:172.21.15.105:6838/2391073232 2024-03-26T19:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 1 -- 172.21.15.97:0/2460349770 >> 172.21.15.97:0/2460349770 conn(0x55dba518e310 msgr2=0x55dba519d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 1 -- 172.21.15.97:0/2460349770 shutdown_connections 2024-03-26T19:14:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 1 -- 172.21.15.97:0/2460349770 wait complete. 2024-03-26T19:14:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.479+0000 7f5b43b95740 20 client.26414 trim_cache size 0 max 16384 2024-03-26T19:14:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-26T19:14:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-26T19:14:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-26T19:14:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-26T19:14:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 -- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5b200a3040 msgr2=0x7f5b200a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5b200a3040 0x7f5b200a5500 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5b3409bf20 tx=0x7f5b34005850 comp rx=0 tx=0).stop 2024-03-26T19:14:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 -- 172.21.15.97:0/2665318707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c109a30 msgr2=0x7f5b3c072c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c109a30 0x7f5b3c072c90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400eaa0 tx=0x7f5b3400ef60 comp rx=0 tx=0).stop 2024-03-26T19:14:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 -- 172.21.15.97:0/2665318707 shutdown_connections 2024-03-26T19:14:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f5b200a3040 0x7f5b200a5500 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c109a30 0x7f5b3c072c90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3c109070 0x7f5b3c06bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 --2- 172.21.15.97:0/2665318707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b3c107f20 0x7f5b3c06b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 -- 172.21.15.97:0/2665318707 >> 172.21.15.97:0/2665318707 conn(0x7f5b3c076ed0 msgr2=0x7f5b3c10c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 -- 172.21.15.97:0/2665318707 shutdown_connections 2024-03-26T19:14:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-26T19:14:34.480+0000 7f5b40e00640 1 -- 172.21.15.97:0/2665318707 wait complete. 2024-03-26T19:14:34.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:34 smithi097 ceph-mon[31955]: pgmap v860: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.1 MiB/s wr, 849 op/s 2024-03-26T19:14:34.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:34 smithi144 ceph-mon[36053]: pgmap v860: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.1 MiB/s wr, 849 op/s 2024-03-26T19:14:34.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:34 smithi105 ceph-mon[36270]: pgmap v860: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.1 MiB/s wr, 849 op/s 2024-03-26T19:14:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:36 smithi097 ceph-mon[31955]: pgmap v861: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.4 MiB/s wr, 854 op/s 2024-03-26T19:14:36.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:36 smithi144 ceph-mon[36053]: pgmap v861: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.4 MiB/s wr, 854 op/s 2024-03-26T19:14:36.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:36 smithi105 ceph-mon[36270]: pgmap v861: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.4 MiB/s wr, 854 op/s 2024-03-26T19:14:37.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-26T19:14:37.818 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:37 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:14:37.868 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/: 2024-03-26T19:14:37.868 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.868 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:58 . 2024-03-26T19:14:37.868 INFO:teuthology.orchestra.run.smithi097.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:58 .. 2024-03-26T19:14:37.868 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 data 2024-03-26T19:14:37.868 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.868 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data: 2024-03-26T19:14:37.868 INFO:teuthology.orchestra.run.smithi097.stdout:total 36 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 . 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:58 .. 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:59 base 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Mar 26 18:59 current_logfiles 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Mar 26 18:59 global 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 log 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_commit_ts 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_dynshmem 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Mar 26 18:59 pg_hba.conf 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Mar 26 18:59 pg_ident.conf 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 19:14 pg_logical 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 pg_multixact 2024-03-26T19:14:37.869 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_notify 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_replslot 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_serial 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_snapshots 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 19:14 pg_stat 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 19:14 pg_stat_tmp 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 19:14 pg_subtrans 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_tblspc 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pg_twophase 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 PG_VERSION 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Mar 26 19:14 pg_wal 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 19:07 pg_xact 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Mar 26 18:59 postgresql.auto.conf 2024-03-26T19:14:37.870 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Mar 26 18:59 postgresql.conf 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Mar 26 18:59 postmaster.opts 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/base: 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:59 . 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:59 1 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:59 13493 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Mar 26 19:03 13494 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 pgsql_tmp 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/base/1: 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:total 7777 2024-03-26T19:14:37.871 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:59 . 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:59 .. 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 112 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 113 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 1247 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1247_fsm 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1247_vm 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:59 1249 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1249_fsm 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1249_vm 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:59 1255 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1255_fsm 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1255_vm 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:59 1259 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1259_fsm 2024-03-26T19:14:37.872 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1259_vm 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:59 13344 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13344_fsm 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13344_vm 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13346 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13348 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13349_fsm 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349_vm 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13351 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13353 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13354_fsm 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354_vm 2024-03-26T19:14:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13356 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13358 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13359_fsm 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359_vm 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13361 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13363 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1417 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1418 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 174 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 175 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2187 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2224 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2328 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2336 2024-03-26T19:14:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2337 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2579 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2600 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2600_fsm 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2600_vm 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2601_fsm 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601_vm 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2602 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2602_fsm 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2602_vm 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2603 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2603_fsm 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2603_vm 2024-03-26T19:14:37.875 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2604 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2605 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2605_fsm 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2605_vm 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2606_fsm 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606_vm 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2607 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2607_fsm 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2607_vm 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:59 2608 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2608_fsm 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2608_vm 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:59 2609 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2609_fsm 2024-03-26T19:14:37.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2609_vm 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2610 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2610_fsm 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2610_vm 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2611 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2612_fsm 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612_vm 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2613 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2615_fsm 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615_vm 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2616 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2616_fsm 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2616_vm 2024-03-26T19:14:37.877 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:59 2617 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2617_fsm 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2617_vm 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:59 2618 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2618_fsm 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2618_vm 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 18:59 2619 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2619_fsm 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2619_vm 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2620 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2650 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2651 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2652 2024-03-26T19:14:37.878 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2653 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2654 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2655 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2656 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2657 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:59 2658 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 2659 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2660 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2661 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2662 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2663 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2664 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2665 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2666 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2667 2024-03-26T19:14:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2668 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2669 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2670 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:59 2673 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:59 2674 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:59 2675 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2678 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2679 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2680 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2681 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2682 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2683 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2684 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2685 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2686 2024-03-26T19:14:37.880 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2687 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2688 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2689 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:59 2690 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:59 2691 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2692 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2693 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2696 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2699 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2701 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2702 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2703 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2704 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2753 2024-03-26T19:14:37.881 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2753_fsm 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2753_vm 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2754 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2755 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2756 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2757 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2830 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2831 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2832 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2833 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2834 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2835 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2836 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2837 2024-03-26T19:14:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:59 2838 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2838_fsm 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2838_vm 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2839 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840_fsm 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2840_vm 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2841 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2995 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2996 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3079_fsm 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079_vm 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3080 2024-03-26T19:14:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3081 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3085 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3118 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3119 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:59 3164 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3256 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3257 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3258 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3350 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3351 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3379 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3380 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3381 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3394 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3394_fsm 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3394_vm 2024-03-26T19:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3395 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3429 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3430 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3431 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3433 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3439 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3440 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3455 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:59 3456 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3456_fsm 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3456_vm 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3466 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3467 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3468 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3501 2024-03-26T19:14:37.885 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3502 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3503 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3534 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3541_fsm 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541_vm 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3542 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3574 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3575 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3576 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3596 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3597 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3598 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3599 2024-03-26T19:14:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3600_fsm 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600_vm 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3601_fsm 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601_vm 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3602_fsm 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602_vm 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603_fsm 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3603_vm 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3604 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3605 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3606 2024-03-26T19:14:37.887 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3607 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3608 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 3609 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3712 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3764_fsm 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764_vm 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3766 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3767 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3997 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4143 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4144 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4145 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4146 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4147 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4148 2024-03-26T19:14:37.888 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4149 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4150 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4151 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4152 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4153 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4154 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4155 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4156 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4157 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4158 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4159 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4160 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4161 2024-03-26T19:14:37.889 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4162 2024-03-26T19:14:37.890 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:37 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:14:37.891 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4163 2024-03-26T19:14:37.891 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4164 2024-03-26T19:14:37.891 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4165 2024-03-26T19:14:37.891 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4166 2024-03-26T19:14:37.891 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4167 2024-03-26T19:14:37.891 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4168 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4169 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4170 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4171 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4172 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4173 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4174 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 5002 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 548 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 549 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6102 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6104 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6106 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6110 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6111 2024-03-26T19:14:37.892 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6112 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6113 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6117 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 826 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 827 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 828 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:59 pg_filenode.map 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 PG_VERSION 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:total 7777 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:59 . 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:59 .. 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 112 2024-03-26T19:14:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 113 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 1247 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1247_fsm 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1247_vm 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:59 1249 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1249_fsm 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1249_vm 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:59 1255 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1255_fsm 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1255_vm 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:59 1259 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1259_fsm 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1259_vm 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:59 13344 2024-03-26T19:14:37.894 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13344_fsm 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13344_vm 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13346 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13348 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13349_fsm 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349_vm 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13351 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13353 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13354_fsm 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354_vm 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13356 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13358 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359 2024-03-26T19:14:37.895 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13359_fsm 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359_vm 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13361 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13363 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1417 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1418 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 174 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 175 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2187 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2224 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2328 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2336 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2337 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2579 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2600 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2600_fsm 2024-03-26T19:14:37.896 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2600_vm 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2601_fsm 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601_vm 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2602 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2602_fsm 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2602_vm 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2603 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2603_fsm 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2603_vm 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2604 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2605 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2605_fsm 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2605_vm 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606 2024-03-26T19:14:37.897 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2606_fsm 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606_vm 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2607 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2607_fsm 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2607_vm 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:59 2608 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2608_fsm 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2608_vm 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:59 2609 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2609_fsm 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2609_vm 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2610 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2610_fsm 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2610_vm 2024-03-26T19:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2611 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2612_fsm 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612_vm 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2613 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2615_fsm 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615_vm 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2616 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2616_fsm 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2616_vm 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:59 2617 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2617_fsm 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2617_vm 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:59 2618 2024-03-26T19:14:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2618_fsm 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2618_vm 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 18:59 2619 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2619_fsm 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2619_vm 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2620 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2650 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2651 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2652 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2653 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2654 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2655 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2656 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2657 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:59 2658 2024-03-26T19:14:37.900 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 2659 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2660 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2661 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2662 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2663 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2664 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2665 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2666 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2667 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2668 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2669 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2670 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:59 2673 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:59 2674 2024-03-26T19:14:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:59 2675 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2678 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2679 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2680 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2681 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2682 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2683 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2684 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2685 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2686 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2687 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2688 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2689 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:59 2690 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:59 2691 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2692 2024-03-26T19:14:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2693 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2696 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2699 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2701 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2702 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2703 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2704 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2753 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2753_fsm 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2753_vm 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2754 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2755 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2756 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2757 2024-03-26T19:14:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2830 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2831 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2832 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2833 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2834 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2835 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2836 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2837 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:59 2838 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2838_fsm 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2838_vm 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2839 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840_fsm 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2840_vm 2024-03-26T19:14:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2841 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2995 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2996 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3079_fsm 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079_vm 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3080 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3081 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3085 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3118 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3119 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:59 3164 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3256 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3257 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3258 2024-03-26T19:14:37.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3350 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3351 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3379 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3380 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3381 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3394 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3394_fsm 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3394_vm 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3395 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3429 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3430 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3431 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3433 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3439 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3440 2024-03-26T19:14:37.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3455 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:59 3456 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3456_fsm 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3456_vm 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3466 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3467 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3468 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3501 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3502 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3503 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3534 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3541_fsm 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541_vm 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3542 2024-03-26T19:14:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3574 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3575 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3576 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3596 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3597 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3598 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3599 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3600_fsm 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600_vm 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3601_fsm 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601_vm 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3602_fsm 2024-03-26T19:14:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602_vm 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603_fsm 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3603_vm 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3604 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3605 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3606 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3607 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3608 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 3609 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3712 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3764_fsm 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764_vm 2024-03-26T19:14:37.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3766 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3767 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3997 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4143 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4144 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4145 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4146 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4147 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4148 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4149 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4150 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4151 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4152 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4153 2024-03-26T19:14:37.910 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4154 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4155 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4156 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4157 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4158 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4159 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4160 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4161 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4162 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4163 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4164 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4165 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4166 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4167 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4168 2024-03-26T19:14:37.911 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4169 2024-03-26T19:14:37.912 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4170 2024-03-26T19:14:37.912 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4171 2024-03-26T19:14:37.912 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4172 2024-03-26T19:14:37.914 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4173 2024-03-26T19:14:37.914 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4174 2024-03-26T19:14:37.914 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 5002 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 548 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 549 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6102 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6104 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6106 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6110 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6111 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6112 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6113 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6117 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 826 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 827 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 828 2024-03-26T19:14:37.915 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:59 pg_filenode.map 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 PG_VERSION 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:total 598982 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Mar 26 19:03 . 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:59 .. 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 112 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 113 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 1247 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1247_fsm 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1247_vm 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:59 1249 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1249_fsm 2024-03-26T19:14:37.916 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1249_vm 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:59 1255 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1255_fsm 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1255_vm 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 19:14 1259 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1259_fsm 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1259_vm 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:59 13344 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13344_fsm 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13344_vm 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13346 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13348 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13349_fsm 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13349_vm 2024-03-26T19:14:37.917 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13351 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13353 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13354_fsm 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13354_vm 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13356 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13358 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 13359_fsm 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13359_vm 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 13361 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 13363 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1417 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 1418 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 443097088 Mar 26 19:14 16396 2024-03-26T19:14:37.918 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Mar 26 19:14 16396_fsm 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 19:03 16396_vm 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 704512 Mar 26 19:14 16397 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 19:14 16397_fsm 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 19:14 16397_vm 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1187840 Mar 26 19:14 16399 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 19:14 16399_fsm 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 19:14 16399_vm 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 19:14 16400 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 19:14 16402 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Mar 26 19:14 16404 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 87973888 Mar 26 19:14 16406 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 19:14 16406_fsm 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 19:10 16406_vm 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 174 2024-03-26T19:14:37.919 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 175 2024-03-26T19:14:37.920 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2187 2024-03-26T19:14:37.920 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2224 2024-03-26T19:14:37.920 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2328 2024-03-26T19:14:37.920 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2336 2024-03-26T19:14:37.920 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2337 2024-03-26T19:14:37.921 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2579 2024-03-26T19:14:37.921 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2600 2024-03-26T19:14:37.921 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2600_fsm 2024-03-26T19:14:37.921 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2600_vm 2024-03-26T19:14:37.921 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601 2024-03-26T19:14:37.921 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2601_fsm 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2601_vm 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2602 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2602_fsm 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2602_vm 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2603 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2603_fsm 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2603_vm 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2604 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2605 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2605_fsm 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2605_vm 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2606_fsm 2024-03-26T19:14:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2606_vm 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2607 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2607_fsm 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2607_vm 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:59 2608 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2608_fsm 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2608_vm 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:59 2609 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2609_fsm 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2609_vm 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2610 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2610_fsm 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2610_vm 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2611 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612 2024-03-26T19:14:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2612_fsm 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2612_vm 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2613 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2615_fsm 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2615_vm 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2616 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2616_fsm 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2616_vm 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:59 2617 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2617_fsm 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2617_vm 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:59 2618 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2618_fsm 2024-03-26T19:14:37.924 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2618_vm 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Mar 26 19:14 2619 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 19:02 2619_fsm 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2619_vm 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2620 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2650 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2651 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2652 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2653 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2654 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2655 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2656 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2657 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:59 2658 2024-03-26T19:14:37.925 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:59 2659 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2660 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2661 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2662 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2663 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2664 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2665 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2666 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2667 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2668 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2669 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2670 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:59 2673 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:59 2674 2024-03-26T19:14:37.926 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:59 2675 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2678 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2679 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2680 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2681 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2682 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2683 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2684 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2685 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2686 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2687 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2688 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:59 2689 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:59 2690 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:59 2691 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2692 2024-03-26T19:14:37.927 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2693 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 19:03 2696 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2699 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2701 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2702 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2703 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 2704 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2753 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2753_fsm 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2753_vm 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2754 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2755 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2756 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 2757 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2830 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2831 2024-03-26T19:14:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2832 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2833 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2834 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2835 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2836 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2837 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:59 2838 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2838_fsm 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2838_vm 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2839 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2840_fsm 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2840_vm 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2841 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2995 2024-03-26T19:14:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2996 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3079_fsm 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3079_vm 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3080 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3081 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3085 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3118 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3119 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:59 3164 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3256 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3257 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3258 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3350 2024-03-26T19:14:37.930 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3351 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3379 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3380 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3381 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3394 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3394_fsm 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3394_vm 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3395 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3429 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3430 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3431 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3433 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3439 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3440 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:59 3455 2024-03-26T19:14:37.931 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:59 3456 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3456_fsm 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3456_vm 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3466 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3467 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3468 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3501 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3502 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3503 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3534 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3541_fsm 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3541_vm 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3542 2024-03-26T19:14:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3574 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3575 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3576 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3596 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3597 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3598 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3599 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3600_fsm 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3600_vm 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3601_fsm 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3601_vm 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3602_fsm 2024-03-26T19:14:37.933 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3602_vm 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3603_fsm 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3603_vm 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3604 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3605 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3606 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3607 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3608 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:59 3609 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3712 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 3764_fsm 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3764_vm 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3766 2024-03-26T19:14:37.934 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 3767 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3997 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4143 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4144 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4145 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4146 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4147 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4148 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4149 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4150 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4151 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4152 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4153 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4154 2024-03-26T19:14:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4155 2024-03-26T19:14:37.936 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4156 2024-03-26T19:14:37.936 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4157 2024-03-26T19:14:37.936 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4158 2024-03-26T19:14:37.936 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4159 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4160 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4161 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4162 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4163 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4164 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4165 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4166 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4167 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4168 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4169 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4170 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4171 2024-03-26T19:14:37.937 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4172 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4173 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4174 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 5002 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 548 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 549 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6102 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6104 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6106 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6110 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6111 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6112 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6113 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6117 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 826 2024-03-26T19:14:37.938 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 827 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 828 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:59 pg_filenode.map 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Mar 26 18:59 pg_internal.init 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:59 PG_VERSION 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:59 .. 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/global: 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:total 560 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Mar 26 18:59 . 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.939 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1213 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1213_fsm 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1213_vm 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1214 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1214_fsm 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1214_vm 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 1232 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 1233 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1260 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1260_fsm 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1260_vm 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1261 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1261_fsm 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1261_vm 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1262 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 1262_fsm 2024-03-26T19:14:37.940 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 1262_vm 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2396 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:59 2396_fsm 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2396_vm 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2397 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2671 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2672 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2676 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2677 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2694 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2695 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2697 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:59 2698 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2846 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2847 2024-03-26T19:14:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2964 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2965 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 2966 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 2967 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 3592 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 3593 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4060 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4061 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4175 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4176 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4177 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4178 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4181 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4182 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4183 2024-03-26T19:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4184 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 4185 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 4186 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6000 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6001 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6002 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 6100 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6114 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 6115 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 19:14 pg_control 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:59 pg_filenode.map 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Mar 26 18:59 pg_internal.init 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/log: 2024-03-26T19:14:37.943 INFO:teuthology.orchestra.run.smithi097.stdout:total 2 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 . 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1222 Mar 26 19:14 postgresql-Tue.log 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.944 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:total 1 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 19:14 . 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 mappings 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Mar 26 19:14 replorigin_checkpoint 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 snapshots 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 19:14 .. 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.945 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 19:14 .. 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 . 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 members 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 offsets 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:total 8 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 . 2024-03-26T19:14:37.946 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 .. 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:59 0000 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:total 8 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:59 . 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:59 .. 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 19:03 0000 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-03-26T19:14:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:total 15 2024-03-26T19:14:37.948 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 19:14 . 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Mar 26 19:14 db_0.stat 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Mar 26 19:14 db_13494.stat 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Mar 26 19:14 global.stat 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 19:14 . 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:total 48 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 19:14 . 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 19:14 0019 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:total 1048576 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Mar 26 19:14 . 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.950 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:14 000000010000000000000099 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:10 00000001000000000000009A 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:09 00000001000000000000009B 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 00000001000000000000009C 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 00000001000000000000009D 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 00000001000000000000009E 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:10 00000001000000000000009F 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:08 0000000100000000000000A0 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000A1 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:08 0000000100000000000000A2 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:08 0000000100000000000000A3 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000A4 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:08 0000000100000000000000A5 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000A6 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000A7 2024-03-26T19:14:37.951 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:08 0000000100000000000000A8 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:09 0000000100000000000000A9 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:08 0000000100000000000000AA 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000AB 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:08 0000000100000000000000AC 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000AD 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000AE 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:08 0000000100000000000000AF 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000B0 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:08 0000000100000000000000B1 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000B2 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000B3 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:08 0000000100000000000000B4 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000B5 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:11 0000000100000000000000B6 2024-03-26T19:14:37.952 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:09 0000000100000000000000B7 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000B8 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:11 0000000100000000000000B9 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:09 0000000100000000000000BA 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:07 0000000100000000000000BB 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000BC 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000BD 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000BE 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000BF 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000C0 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:14 0000000100000000000000C1 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000C2 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000C3 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:14 0000000100000000000000C4 2024-03-26T19:14:37.953 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000C5 2024-03-26T19:14:37.954 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000C6 2024-03-26T19:14:37.954 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000C7 2024-03-26T19:14:37.954 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000C8 2024-03-26T19:14:37.954 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:14 0000000100000000000000C9 2024-03-26T19:14:37.954 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000CA 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000CB 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000CC 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000CD 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000CE 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000CF 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000D0 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000D1 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000D2 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000D3 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:12 0000000100000000000000D4 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000D5 2024-03-26T19:14:37.955 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000D6 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000D7 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 19:13 0000000100000000000000D8 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 archive_status 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:total 0 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:59 . 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Mar 26 19:14 .. 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:total 408 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 19:07 . 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 19:14 .. 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Mar 26 19:07 0000 2024-03-26T19:14:37.956 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Mar 26 19:14 0001 2024-03-26T19:14:37.957 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-26T19:14:37.967 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi097.front.sepia.ceph.com 2024-03-26T19:14:37.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:37 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T19:14:37.982 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-26T19:14:37.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T19:14:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.224+0000 7f1221a00640 1 -- 172.21.15.97:0/3456599314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f121c0fff80 msgr2=0x7f121c1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.224+0000 7f1221a00640 1 --2- 172.21.15.97:0/3456599314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f121c0fff80 0x7f121c1023b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f120c009860 tx=0x7f120c02bb70 comp rx=0 tx=0).stop 2024-03-26T19:14:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.224+0000 7f1221a00640 1 -- 172.21.15.97:0/3456599314 shutdown_connections 2024-03-26T19:14:38.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.224+0000 7f1221a00640 1 --2- 172.21.15.97:0/3456599314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f121c1052a0 0x7f121c105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.224+0000 7f1221a00640 1 --2- 172.21.15.97:0/3456599314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f121c1028f0 0x7f121c104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.224+0000 7f1221a00640 1 --2- 172.21.15.97:0/3456599314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f121c0fff80 0x7f121c1023b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.224+0000 7f1221a00640 1 -- 172.21.15.97:0/3456599314 >> 172.21.15.97:0/3456599314 conn(0x7f121c0fbba0 msgr2=0x7f121c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:38.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.224+0000 7f1221a00640 1 -- 172.21.15.97:0/3456599314 shutdown_connections 2024-03-26T19:14:38.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.224+0000 7f1221a00640 1 -- 172.21.15.97:0/3456599314 wait complete. 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.225+0000 7f1221a00640 1 Processor -- start 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1221a00640 1 -- start start 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1221a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f121c0fff80 0x7f121c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1221a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f121c1028f0 0x7f121c19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1221a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f121c1052a0 0x7f121c1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1221a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f121c113960 con 0x7f121c0fff80 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1221a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1137e0 con 0x7f121c1052a0 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1221a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f121c113ae0 con 0x7f121c1028f0 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1213e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f121c1028f0 0x7f121c19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f121be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f121c0fff80 0x7f121c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:38.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1213e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f121c1028f0 0x7f121c19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:50156/0 (socket says 172.21.15.97:50156) 2024-03-26T19:14:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1213e00640 1 -- 172.21.15.97:0/664127048 learned_addr learned my addr 172.21.15.97:0/664127048 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.226+0000 7f1221000640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f121c1052a0 0x7f121c1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.227+0000 7f1213e00640 1 -- 172.21.15.97:0/664127048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f121c1052a0 msgr2=0x7f121c1a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.227+0000 7f1213e00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f121c1052a0 0x7f121c1a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.227+0000 7f1213e00640 1 -- 172.21.15.97:0/664127048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f121c0fff80 msgr2=0x7f121c19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.227+0000 7f1213e00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f121c0fff80 0x7f121c19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.227+0000 7f1213e00640 1 -- 172.21.15.97:0/664127048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c1a5be0 con 0x7f121c1028f0 2024-03-26T19:14:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.227+0000 7f1221000640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f121c1052a0 0x7f121c1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:14:38.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.227+0000 7f1213e00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f121c1028f0 0x7f121c19e500 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f120400e9f0 tx=0x7f120400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.227+0000 7f1219600640 1 -- 172.21.15.97:0/664127048 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120400cd80 con 0x7f121c1028f0 2024-03-26T19:14:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.227+0000 7f1219600640 1 -- 172.21.15.97:0/664127048 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1204010910 con 0x7f121c1028f0 2024-03-26T19:14:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.227+0000 7f1219600640 1 -- 172.21.15.97:0/664127048 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204018990 con 0x7f121c1028f0 2024-03-26T19:14:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.228+0000 7f1221a00640 1 -- 172.21.15.97:0/664127048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121c1a5ed0 con 0x7f121c1028f0 2024-03-26T19:14:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.228+0000 7f1221a00640 1 -- 172.21.15.97:0/664127048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121c1a6340 con 0x7f121c1028f0 2024-03-26T19:14:38.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.230+0000 7f1221a00640 1 -- 172.21.15.97:0/664127048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f121c0fd690 con 0x7f121c1028f0 2024-03-26T19:14:38.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.233+0000 7f1219600640 1 -- 172.21.15.97:0/664127048 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f1204010ac0 con 0x7f121c1028f0 2024-03-26T19:14:38.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.234+0000 7f1219600640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f12000a2f20 0x7f12000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:38.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.234+0000 7f1219600640 1 -- 172.21.15.97:0/664127048 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f12040cf8b0 con 0x7f121c1028f0 2024-03-26T19:14:38.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.234+0000 7f121be00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f12000a2f20 0x7f12000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:38.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.235+0000 7f121be00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f12000a2f20 0x7f12000a53e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f120c009830 tx=0x7f120c034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:38.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.238+0000 7f1219600640 1 -- 172.21.15.97:0/664127048 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f120409c6b0 con 0x7f121c1028f0 2024-03-26T19:14:38.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.437+0000 7f1221a00640 1 -- 172.21.15.97:0/664127048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f121c0624c0 con 0x7f121c1028f0 2024-03-26T19:14:38.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.439+0000 7f1219600640 1 -- 172.21.15.97:0/664127048 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22581 (secure 0 0 0) 0x7f120409bfb0 con 0x7f121c1028f0 2024-03-26T19:14:38.440 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T19:13:27.698010+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T19:14:38.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 -- 172.21.15.97:0/664127048 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f12000a2f20 msgr2=0x7f12000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:38.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f12000a2f20 0x7f12000a53e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f120c009830 tx=0x7f120c034ab0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 -- 172.21.15.97:0/664127048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f121c1028f0 msgr2=0x7f121c19e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:38.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f121c1028f0 0x7f121c19e500 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f120400e9f0 tx=0x7f120400eeb0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 -- 172.21.15.97:0/664127048 shutdown_connections 2024-03-26T19:14:38.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f12000a2f20 0x7f12000a53e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f121c1052a0 0x7f121c1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f121c1028f0 0x7f121c19e500 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 --2- 172.21.15.97:0/664127048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f121c0fff80 0x7f121c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 -- 172.21.15.97:0/664127048 >> 172.21.15.97:0/664127048 conn(0x7f121c0fbba0 msgr2=0x7f121c103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 -- 172.21.15.97:0/664127048 shutdown_connections 2024-03-26T19:14:38.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.441+0000 7f1221a00640 1 -- 172.21.15.97:0/664127048 wait complete. 2024-03-26T19:14:38.460 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T18:51:11.459500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T18:56:22.069048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T18:56:23.116030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T19:14:38.461 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T19:14:38.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.685+0000 7fb754c00640 1 -- 172.21.15.97:0/4254446667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb750074320 msgr2=0x7fb7500747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:38.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.685+0000 7fb754c00640 1 --2- 172.21.15.97:0/4254446667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb750074320 0x7fb7500747a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009860 tx=0x7fb73c02bb70 comp rx=0 tx=0).stop 2024-03-26T19:14:38.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.686+0000 7fb754c00640 1 -- 172.21.15.97:0/4254446667 shutdown_connections 2024-03-26T19:14:38.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.686+0000 7fb754c00640 1 --2- 172.21.15.97:0/4254446667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb750109680 0x7fb75010bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.686+0000 7fb754c00640 1 --2- 172.21.15.97:0/4254446667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb750074320 0x7fb7500747a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.686+0000 7fb754c00640 1 --2- 172.21.15.97:0/4254446667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb750075900 0x7fb750073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.686+0000 7fb754c00640 1 -- 172.21.15.97:0/4254446667 >> 172.21.15.97:0/4254446667 conn(0x7fb7500fcd30 msgr2=0x7fb7500ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:38.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.686+0000 7fb754c00640 1 -- 172.21.15.97:0/4254446667 shutdown_connections 2024-03-26T19:14:38.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.686+0000 7fb754c00640 1 -- 172.21.15.97:0/4254446667 wait complete. 2024-03-26T19:14:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb754c00640 1 Processor -- start 2024-03-26T19:14:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb754c00640 1 -- start start 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb754c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb750074320 0x7fb750199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb754c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb750075900 0x7fb75019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb754c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb750109680 0x7fb7501a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb754c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010f620 con 0x7fb750075900 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb754c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010f4a0 con 0x7fb750109680 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb754c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010f7a0 con 0x7fb750074320 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb74fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb750109680 0x7fb7501a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb74ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb750075900 0x7fb75019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb74fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb750109680 0x7fb7501a1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34342/0 (socket says 172.21.15.97:34342) 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb74f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb750074320 0x7fb750199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.687+0000 7fb74fe00640 1 -- 172.21.15.97:0/3345133023 learned_addr learned my addr 172.21.15.97:0/3345133023 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb74f400640 1 -- 172.21.15.97:0/3345133023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb750109680 msgr2=0x7fb7501a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb74f400640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb750109680 0x7fb7501a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb74f400640 1 -- 172.21.15.97:0/3345133023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb750075900 msgr2=0x7fb75019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb74f400640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb750075900 0x7fb75019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb74f400640 1 -- 172.21.15.97:0/3345133023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7501a1930 con 0x7fb750074320 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb74fe00640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb750109680 0x7fb7501a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb74f400640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb750074320 0x7fb750199d20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb740009fd0 tx=0x7fb74000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb747e00640 1 -- 172.21.15.97:0/3345133023 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74000ccc0 con 0x7fb750074320 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb747e00640 1 -- 172.21.15.97:0/3345133023 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74000ee70 con 0x7fb750074320 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb754c00640 1 -- 172.21.15.97:0/3345133023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7501a1c20 con 0x7fb750074320 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.688+0000 7fb747e00640 1 -- 172.21.15.97:0/3345133023 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740004a00 con 0x7fb750074320 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.689+0000 7fb754c00640 1 -- 172.21.15.97:0/3345133023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7501a2170 con 0x7fb750074320 2024-03-26T19:14:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.690+0000 7fb754c00640 1 -- 172.21.15.97:0/3345133023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7501083b0 con 0x7fb750074320 2024-03-26T19:14:38.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.691+0000 7fb747e00640 1 -- 172.21.15.97:0/3345133023 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fb740004ba0 con 0x7fb750074320 2024-03-26T19:14:38.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.692+0000 7fb747e00640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7340a2fa0 0x7fb7340a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:38.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.693+0000 7fb747e00640 1 -- 172.21.15.97:0/3345133023 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fb7400cf320 con 0x7fb750074320 2024-03-26T19:14:38.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.693+0000 7fb74ea00640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7340a2fa0 0x7fb7340a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:38.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.694+0000 7fb74ea00640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7340a2fa0 0x7fb7340a5460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009830 tx=0x7fb73c034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:38.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.697+0000 7fb747e00640 1 -- 172.21.15.97:0/3345133023 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb74009c180 con 0x7fb750074320 2024-03-26T19:14:38.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:38 smithi097 ceph-mon[31955]: pgmap v862: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 20 MiB/s wr, 1.50k op/s 2024-03-26T19:14:38.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:38 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:14:38.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:38 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:14:38.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:38 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:14:38.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:38 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/664127048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T19:14:38.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:38 smithi144 ceph-mon[36053]: pgmap v862: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 20 MiB/s wr, 1.50k op/s 2024-03-26T19:14:38.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:38 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:14:38.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:38 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:14:38.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:38 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:14:38.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:38 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/664127048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T19:14:38.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.896+0000 7fb754c00640 1 -- 172.21.15.97:0/3345133023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb7500624c0 con 0x7fb750074320 2024-03-26T19:14:38.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.896+0000 7fb747e00640 1 -- 172.21.15.97:0/3345133023 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fb74009ba80 con 0x7fb750074320 2024-03-26T19:14:38.897 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-26T19:14:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 -- 172.21.15.97:0/3345133023 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7340a2fa0 msgr2=0x7fb7340a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7340a2fa0 0x7fb7340a5460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009830 tx=0x7fb73c034ab0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 -- 172.21.15.97:0/3345133023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb750074320 msgr2=0x7fb750199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb750074320 0x7fb750199d20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb740009fd0 tx=0x7fb74000ec20 comp rx=0 tx=0).stop 2024-03-26T19:14:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 -- 172.21.15.97:0/3345133023 shutdown_connections 2024-03-26T19:14:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7340a2fa0 0x7fb7340a5460 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb750109680 0x7fb7501a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb750075900 0x7fb75019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 --2- 172.21.15.97:0/3345133023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb750074320 0x7fb750199d20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 -- 172.21.15.97:0/3345133023 >> 172.21.15.97:0/3345133023 conn(0x7fb7500fcd30 msgr2=0x7fb750105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:38.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 -- 172.21.15.97:0/3345133023 shutdown_connections 2024-03-26T19:14:38.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:38.899+0000 7fb754c00640 1 -- 172.21.15.97:0/3345133023 wait complete. 2024-03-26T19:14:38.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T19:14:38.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:38 smithi105 ceph-mon[36270]: pgmap v862: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 20 MiB/s wr, 1.50k op/s 2024-03-26T19:14:38.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T19:14:38.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T19:14:38.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:38 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' 2024-03-26T19:14:38.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:38 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/664127048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T19:14:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.121+0000 7fdccf000640 1 -- 172.21.15.97:0/3792133756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcc8104c80 msgr2=0x7fdcc8107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.121+0000 7fdccf000640 1 --2- 172.21.15.97:0/3792133756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcc8104c80 0x7fdcc8107070 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009900 tx=0x7fdcb002bc30 comp rx=0 tx=0).stop 2024-03-26T19:14:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.121+0000 7fdccf000640 1 -- 172.21.15.97:0/3792133756 shutdown_connections 2024-03-26T19:14:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.121+0000 7fdccf000640 1 --2- 172.21.15.97:0/3792133756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc81075b0 0x7fdcc8109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.121+0000 7fdccf000640 1 --2- 172.21.15.97:0/3792133756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcc8104c80 0x7fdcc8107070 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.121+0000 7fdccf000640 1 --2- 172.21.15.97:0/3792133756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc8102350 0x7fdcc8104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.121+0000 7fdccf000640 1 -- 172.21.15.97:0/3792133756 >> 172.21.15.97:0/3792133756 conn(0x7fdcc80fbd60 msgr2=0x7fdcc80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.121+0000 7fdccf000640 1 -- 172.21.15.97:0/3792133756 shutdown_connections 2024-03-26T19:14:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.121+0000 7fdccf000640 1 -- 172.21.15.97:0/3792133756 wait complete. 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.122+0000 7fdccf000640 1 Processor -- start 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdccf000640 1 -- start start 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdccf000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc8102350 0x7fdcc8199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdccf000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcc8104c80 0x7fdcc819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdccf000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc81075b0 0x7fdcc81a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdccf000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc810d590 con 0x7fdcc8102350 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdccf000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc810d410 con 0x7fdcc8104c80 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdccf000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc810d710 con 0x7fdcc81075b0 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdcbfe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcc8104c80 0x7fdcc819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdcc7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc8102350 0x7fdcc8199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdcbfe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcc8104c80 0x7fdcc819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34362/0 (socket says 172.21.15.97:34362) 2024-03-26T19:14:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdcbfe00640 1 -- 172.21.15.97:0/3048044586 learned_addr learned my addr 172.21.15.97:0/3048044586 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.123+0000 7fdccd000640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc81075b0 0x7fdcc81a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdcbfe00640 1 -- 172.21.15.97:0/3048044586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc81075b0 msgr2=0x7fdcc81a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdcbfe00640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc81075b0 0x7fdcc81a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdcbfe00640 1 -- 172.21.15.97:0/3048044586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc8102350 msgr2=0x7fdcc8199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdcbfe00640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc8102350 0x7fdcc8199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdcbfe00640 1 -- 172.21.15.97:0/3048044586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc81a1a80 con 0x7fdcc8104c80 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdccd000640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc81075b0 0x7fdcc81a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdcbfe00640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcc8104c80 0x7fdcc819a3a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0000c00 tx=0x7fdcb002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdcc5600640 1 -- 172.21.15.97:0/3048044586 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0043020 con 0x7fdcc8104c80 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdcc5600640 1 -- 172.21.15.97:0/3048044586 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb00054e0 con 0x7fdcc8104c80 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdccf000640 1 -- 172.21.15.97:0/3048044586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc81a1d10 con 0x7fdcc8104c80 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.124+0000 7fdcc5600640 1 -- 172.21.15.97:0/3048044586 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0003c40 con 0x7fdcc8104c80 2024-03-26T19:14:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.125+0000 7fdccf000640 1 -- 172.21.15.97:0/3048044586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc81a2250 con 0x7fdcc8104c80 2024-03-26T19:14:39.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.126+0000 7fdccf000640 1 -- 172.21.15.97:0/3048044586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc8109100 con 0x7fdcc8104c80 2024-03-26T19:14:39.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.127+0000 7fdcc5600640 1 -- 172.21.15.97:0/3048044586 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fdcb0046020 con 0x7fdcc8104c80 2024-03-26T19:14:39.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.129+0000 7fdcc5600640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcac0a2fa0 0x7fdcac0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:39.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.129+0000 7fdcc5600640 1 -- 172.21.15.97:0/3048044586 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fdcb00f1440 con 0x7fdcc8104c80 2024-03-26T19:14:39.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.129+0000 7fdcc7e00640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcac0a2fa0 0x7fdcac0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:39.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.130+0000 7fdcc7e00640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcac0a2fa0 0x7fdcac0a5460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009bd0 tx=0x7fdcb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.137+0000 7fdcc5600640 1 -- 172.21.15.97:0/3048044586 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdcb004d020 con 0x7fdcc8104c80 2024-03-26T19:14:39.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.337+0000 7fdccf000640 1 -- 172.21.15.97:0/3048044586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdcc8062560 con 0x7fdcc8104c80 2024-03-26T19:14:39.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.338+0000 7fdcc5600640 1 -- 172.21.15.97:0/3048044586 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fdcb00be240 con 0x7fdcc8104c80 2024-03-26T19:14:39.339 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-26T19:14:39.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 -- 172.21.15.97:0/3048044586 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcac0a2fa0 msgr2=0x7fdcac0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:39.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcac0a2fa0 0x7fdcac0a5460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009bd0 tx=0x7fdcb8005f90 comp rx=0 tx=0).stop 2024-03-26T19:14:39.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 -- 172.21.15.97:0/3048044586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcc8104c80 msgr2=0x7fdcc819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:39.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcc8104c80 0x7fdcc819a3a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0000c00 tx=0x7fdcb002c650 comp rx=0 tx=0).stop 2024-03-26T19:14:39.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 -- 172.21.15.97:0/3048044586 shutdown_connections 2024-03-26T19:14:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fdcac0a2fa0 0x7fdcac0a5460 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc81075b0 0x7fdcc81a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcc8104c80 0x7fdcc819a3a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 --2- 172.21.15.97:0/3048044586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc8102350 0x7fdcc8199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 -- 172.21.15.97:0/3048044586 >> 172.21.15.97:0/3048044586 conn(0x7fdcc80fbd60 msgr2=0x7fdcc8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 -- 172.21.15.97:0/3048044586 shutdown_connections 2024-03-26T19:14:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.341+0000 7fdccf000640 1 -- 172.21.15.97:0/3048044586 wait complete. 2024-03-26T19:14:39.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T19:14:39.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.561+0000 7f2024c00640 1 -- 172.21.15.97:0/4231339847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20201017c0 msgr2=0x7f2020101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:39.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.561+0000 7f2024c00640 1 --2- 172.21.15.97:0/4231339847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20201017c0 0x7f2020101be0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2010009ca0 tx=0x7f201002c0f0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.562+0000 7f2024c00640 1 -- 172.21.15.97:0/4231339847 shutdown_connections 2024-03-26T19:14:39.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.562+0000 7f2024c00640 1 --2- 172.21.15.97:0/4231339847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2020103370 0x7f2020109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.562+0000 7f2024c00640 1 --2- 172.21.15.97:0/4231339847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20201029b0 0x7f2020102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.562+0000 7f2024c00640 1 --2- 172.21.15.97:0/4231339847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20201017c0 0x7f2020101be0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.562+0000 7f2024c00640 1 -- 172.21.15.97:0/4231339847 >> 172.21.15.97:0/4231339847 conn(0x7f20200fcd30 msgr2=0x7f20200ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:39.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.562+0000 7f2024c00640 1 -- 172.21.15.97:0/4231339847 shutdown_connections 2024-03-26T19:14:39.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.562+0000 7f2024c00640 1 -- 172.21.15.97:0/4231339847 wait complete. 2024-03-26T19:14:39.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f2024c00640 1 Processor -- start 2024-03-26T19:14:39.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f2024c00640 1 -- start start 2024-03-26T19:14:39.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f2024c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20201017c0 0x7f2020075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:39.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f2024c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20201029b0 0x7f2020073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:39.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f2024c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2020103370 0x7f20200741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:39.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f2024c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f202010d410 con 0x7f20201029b0 2024-03-26T19:14:39.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f2024c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f202010d290 con 0x7f20201017c0 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f2024c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f202010d590 con 0x7f2020103370 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f201f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20201017c0 0x7f2020075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f201f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20201017c0 0x7f2020075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34380/0 (socket says 172.21.15.97:34380) 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.563+0000 7f201f400640 1 -- 172.21.15.97:0/830424015 learned_addr learned my addr 172.21.15.97:0/830424015 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f2016a00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20201029b0 0x7f2020073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f201fe00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2020103370 0x7f20200741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f201f400640 1 -- 172.21.15.97:0/830424015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2020103370 msgr2=0x7f20200741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f201f400640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2020103370 0x7f20200741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f201f400640 1 -- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20201029b0 msgr2=0x7f2020073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f201f400640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20201029b0 0x7f2020073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f201f400640 1 -- 172.21.15.97:0/830424015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2020074a80 con 0x7f20201017c0 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f2016a00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20201029b0 0x7f2020073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:14:39.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f201f400640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20201017c0 0x7f2020075670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2010000f80 tx=0x7f2010032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:39.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f201cc00640 1 -- 172.21.15.97:0/830424015 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010041020 con 0x7f20201017c0 2024-03-26T19:14:39.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f201cc00640 1 -- 172.21.15.97:0/830424015 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2010034cd0 con 0x7f20201017c0 2024-03-26T19:14:39.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f2024c00640 1 -- 172.21.15.97:0/830424015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20201a1b40 con 0x7f20201017c0 2024-03-26T19:14:39.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.564+0000 7f201cc00640 1 -- 172.21.15.97:0/830424015 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010039040 con 0x7f20201017c0 2024-03-26T19:14:39.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.565+0000 7f2024c00640 1 -- 172.21.15.97:0/830424015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20201a2060 con 0x7f20201017c0 2024-03-26T19:14:39.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.566+0000 7f2024c00640 1 -- 172.21.15.97:0/830424015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2020108b30 con 0x7f20201017c0 2024-03-26T19:14:39.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.567+0000 7f201cc00640 1 -- 172.21.15.97:0/830424015 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f2010044020 con 0x7f20201017c0 2024-03-26T19:14:39.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.568+0000 7f201cc00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20040a2f20 0x7f20040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:39.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.569+0000 7f201cc00640 1 -- 172.21.15.97:0/830424015 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f20100f1250 con 0x7f20201017c0 2024-03-26T19:14:39.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.569+0000 7f2016a00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20040a2f20 0x7f20040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:39.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.570+0000 7f2016a00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20040a2f20 0x7f20040a53e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f20200753e0 tx=0x7f2008005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:39.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.577+0000 7f201cc00640 1 -- 172.21.15.97:0/830424015 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f201004b020 con 0x7f20201017c0 2024-03-26T19:14:39.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.817+0000 7f2024c00640 1 -- 172.21.15.97:0/830424015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f20200624c0 con 0x7f20201017c0 2024-03-26T19:14:39.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.818+0000 7f201cc00640 1 -- 172.21.15.97:0/830424015 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f20100348f0 con 0x7f20201017c0 2024-03-26T19:14:39.819 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-26T19:14:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.820+0000 7f2024c00640 1 -- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20040a2f20 msgr2=0x7f20040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.820+0000 7f2024c00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20040a2f20 0x7f20040a53e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f20200753e0 tx=0x7f2008005f90 comp rx=0 tx=0).stop 2024-03-26T19:14:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.820+0000 7f2024c00640 1 -- 172.21.15.97:0/830424015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20201017c0 msgr2=0x7f2020075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.820+0000 7f2024c00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20201017c0 0x7f2020075670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2010000f80 tx=0x7f2010032ec0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.820+0000 7f2024c00640 1 -- 172.21.15.97:0/830424015 shutdown_connections 2024-03-26T19:14:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.820+0000 7f2024c00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f20040a2f20 0x7f20040a53e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.820+0000 7f2024c00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2020103370 0x7f20200741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.820+0000 7f2024c00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20201029b0 0x7f2020073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.820+0000 7f2024c00640 1 --2- 172.21.15.97:0/830424015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20201017c0 0x7f2020075670 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.820+0000 7f2024c00640 1 -- 172.21.15.97:0/830424015 >> 172.21.15.97:0/830424015 conn(0x7f20200fcd30 msgr2=0x7f2020105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:39.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.821+0000 7f2024c00640 1 -- 172.21.15.97:0/830424015 shutdown_connections 2024-03-26T19:14:39.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:39.821+0000 7f2024c00640 1 -- 172.21.15.97:0/830424015 wait complete. 2024-03-26T19:14:39.844 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:39 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3345133023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T19:14:39.844 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:39 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3048044586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T19:14:39.846 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-26T19:14:39.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:39 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3345133023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T19:14:39.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:39 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3048044586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T19:14:39.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:39 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3345133023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T19:14:39.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:39 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3048044586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T19:14:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.061+0000 7fa3d1200640 1 -- 172.21.15.97:0/3687131723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3cc102190 msgr2=0x7fa3cc104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.061+0000 7fa3d1200640 1 --2- 172.21.15.97:0/3687131723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3cc102190 0x7fa3cc104580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009900 tx=0x7fa3bc02bc50 comp rx=0 tx=0).stop 2024-03-26T19:14:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.061+0000 7fa3d1200640 1 -- 172.21.15.97:0/3687131723 shutdown_connections 2024-03-26T19:14:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.061+0000 7fa3d1200640 1 --2- 172.21.15.97:0/3687131723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3cc1073f0 0x7fa3cc109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.061+0000 7fa3d1200640 1 --2- 172.21.15.97:0/3687131723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc104ac0 0x7fa3cc106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.061+0000 7fa3d1200640 1 --2- 172.21.15.97:0/3687131723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3cc102190 0x7fa3cc104580 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.061+0000 7fa3d1200640 1 -- 172.21.15.97:0/3687131723 >> 172.21.15.97:0/3687131723 conn(0x7fa3cc0fbba0 msgr2=0x7fa3cc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.062+0000 7fa3d1200640 1 -- 172.21.15.97:0/3687131723 shutdown_connections 2024-03-26T19:14:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.062+0000 7fa3d1200640 1 -- 172.21.15.97:0/3687131723 wait complete. 2024-03-26T19:14:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.062+0000 7fa3d1200640 1 Processor -- start 2024-03-26T19:14:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3d1200640 1 -- start start 2024-03-26T19:14:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3d1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc102190 0x7fa3cc101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3d1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3cc104ac0 0x7fa3cc100140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3d1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3cc1073f0 0x7fa3cc100680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3d1200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc10d400 con 0x7fa3cc102190 2024-03-26T19:14:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3d1200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc10d280 con 0x7fa3cc104ac0 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3d1200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc10d580 con 0x7fa3cc1073f0 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3cb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc102190 0x7fa3cc101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3caa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3cc104ac0 0x7fa3cc100140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3caa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3cc104ac0 0x7fa3cc100140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:34404/0 (socket says 172.21.15.97:34404) 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3caa00640 1 -- 172.21.15.97:0/4293487847 learned_addr learned my addr 172.21.15.97:0/4293487847 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.063+0000 7fa3cbe00640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3cc1073f0 0x7fa3cc100680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3caa00640 1 -- 172.21.15.97:0/4293487847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3cc1073f0 msgr2=0x7fa3cc100680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3caa00640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3cc1073f0 0x7fa3cc100680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3caa00640 1 -- 172.21.15.97:0/4293487847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc102190 msgr2=0x7fa3cc101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3caa00640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc102190 0x7fa3cc101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3caa00640 1 -- 172.21.15.97:0/4293487847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc100f40 con 0x7fa3cc104ac0 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3caa00640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3cc104ac0 0x7fa3cc100140 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005fd0 tx=0x7fa3b800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3c3e00640 1 -- 172.21.15.97:0/4293487847 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8017070 con 0x7fa3cc104ac0 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3c3e00640 1 -- 172.21.15.97:0/4293487847 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b8009c30 con 0x7fa3cc104ac0 2024-03-26T19:14:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3c3e00640 1 -- 172.21.15.97:0/4293487847 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8010460 con 0x7fa3cc104ac0 2024-03-26T19:14:40.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3d1200640 1 -- 172.21.15.97:0/4293487847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3cc1a1b90 con 0x7fa3cc104ac0 2024-03-26T19:14:40.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.064+0000 7fa3d1200640 1 -- 172.21.15.97:0/4293487847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3cc1a2100 con 0x7fa3cc104ac0 2024-03-26T19:14:40.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.066+0000 7fa3d1200640 1 -- 172.21.15.97:0/4293487847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3cc0fd690 con 0x7fa3cc104ac0 2024-03-26T19:14:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.068+0000 7fa3c3e00640 1 -- 172.21.15.97:0/4293487847 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fa3b8003e70 con 0x7fa3cc104ac0 2024-03-26T19:14:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.070+0000 7fa3c3e00640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3b00a2fa0 0x7fa3b00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:14:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.070+0000 7fa3c3e00640 1 -- 172.21.15.97:0/4293487847 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa3b80cf150 con 0x7fa3cc104ac0 2024-03-26T19:14:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.070+0000 7fa3cb400640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3b00a2fa0 0x7fa3b00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:14:40.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.071+0000 7fa3cb400640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3b00a2fa0 0x7fa3b00a5460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc02c260 tx=0x7fa3bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:14:40.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.071+0000 7fa3c3e00640 1 -- 172.21.15.97:0/4293487847 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa3b8044e20 con 0x7fa3cc104ac0 2024-03-26T19:14:40.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.349+0000 7fa3d1200640 1 -- 172.21.15.97:0/4293487847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa3cc02d440 con 0x7fa3cc104ac0 2024-03-26T19:14:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.358+0000 7fa3c3e00640 1 -- 172.21.15.97:0/4293487847 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+741842 (secure 0 0 0) 0x7fa3b8020030 con 0x7fa3cc104ac0 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 20, 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "active_gid": 14152, 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "active_name": "x", 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "active_addrs": { 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-03-26T19:14:40.373 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6800", 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3680188982 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6801", 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3680188982 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "active_addr": "172.21.15.97:6801/3680188982", 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "active_change": "2024-03-26T18:48:28.768673+0000", 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "active_mgr_features": 4540701547738038271, 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "available": true, 2024-03-26T19:14:40.374 INFO:teuthology.orchestra.run.smithi097.stdout: "standbys": [ 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "gid": 24109, 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "z", 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_features": 4540701547738038271, 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.375 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2024-03-26T19:14:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2024-03-26T19:14:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2024-03-26T19:14:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2024-03-26T19:14:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.390 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.391 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T19:14:40.392 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.394 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T19:14:40.395 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T19:14:40.396 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.397 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.398 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.399 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.400 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.402 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "read", 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap", 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap-read" 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.403 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T19:14:40.404 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.408 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T19:14:40.409 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T19:14:40.410 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T19:14:40.411 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.412 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T19:14:40.414 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.415 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.417 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2024-03-26T19:14:40.418 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2024-03-26T19:14:40.419 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.420 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.421 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T19:14:40.422 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.423 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.424 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2024-03-26T19:14:40.425 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2024-03-26T19:14:40.426 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.427 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2024-03-26T19:14:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2024-03-26T19:14:40.430 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.431 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.432 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2024-03-26T19:14:40.433 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.434 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.436 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "grafana_dashboards_path": { 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "grafana_dashboards_path", 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.438 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "hw_monitoring": { 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hw_monitoring", 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.441 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.442 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.443 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.444 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.445 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.446 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.448 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2024-03-26T19:14:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T19:14:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T19:14:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "oob_default_addr": { 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "oob_default_addr", 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "169.254.1.1", 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.453 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.455 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2024-03-26T19:14:40.456 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2024-03-26T19:14:40.457 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.458 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.459 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.460 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.461 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.462 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.463 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.464 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.465 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.466 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.467 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.468 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.469 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.470 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.471 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.473 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.474 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.475 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.476 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.478 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.479 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T19:14:40.480 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.481 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.483 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.484 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.485 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.486 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.487 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.488 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T19:14:40.490 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T19:14:40.491 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.492 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.493 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "{}", 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.494 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.496 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.497 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.498 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.499 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.500 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T19:14:40.502 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T19:14:40.504 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T19:14:40.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.506 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.507 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-26T19:14:40.508 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.509 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.510 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.511 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.512 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.512 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.512 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T19:14:40.512 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T19:14:40.512 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.512 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.512 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:40.512 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.514 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.515 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.516 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.517 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.518 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.519 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.521 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.522 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.523 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.524 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.526 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.527 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.528 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.529 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.530 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.531 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.532 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.533 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.534 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.535 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.536 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.537 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.538 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.539 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2024-03-26T19:14:40.541 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.542 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2024-03-26T19:14:40.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.544 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.546 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.547 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.548 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.549 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.550 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.551 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:40.553 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2024-03-26T19:14:40.554 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.555 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2024-03-26T19:14:40.556 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.557 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.558 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.559 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.560 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.562 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.563 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.564 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.565 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.566 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.567 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.569 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.570 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.571 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.572 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.573 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.574 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.577 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.578 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.579 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.580 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.581 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.583 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.584 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.585 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.587 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.588 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.589 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.590 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2024-03-26T19:14:40.592 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.593 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.600 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.602 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.603 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.605 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.606 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.609 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.610 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.611 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.612 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.615 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.616 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.617 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2024-03-26T19:14:40.618 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.619 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.620 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.621 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.622 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.623 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2024-03-26T19:14:40.624 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.625 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.626 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.629 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.630 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.631 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.632 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.633 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2024-03-26T19:14:40.635 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T19:14:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.638 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.639 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.640 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.641 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.643 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.644 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.645 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.647 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.648 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.649 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.650 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.651 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.652 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.653 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.654 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.655 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.657 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.658 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.659 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.660 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.662 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.663 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.664 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-26T19:14:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-26T19:14:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.669 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.670 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.672 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.673 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.674 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.677 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.679 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.682 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.683 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.684 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.685 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.686 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.687 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.688 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.690 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.691 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.693 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.694 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.695 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2024-03-26T19:14:40.696 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.697 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.699 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.700 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.701 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.702 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.703 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T19:14:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2024-03-26T19:14:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2024-03-26T19:14:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_no_wait": { 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T19:14:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "gid": 24110, 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "y", 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_features": 4540701547738038271, 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2024-03-26T19:14:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2024-03-26T19:14:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2024-03-26T19:14:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2024-03-26T19:14:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T19:14:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2024-03-26T19:14:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T19:14:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2024-03-26T19:14:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "read", 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap", 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap-read" 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2024-03-26T19:14:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2024-03-26T19:14:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2024-03-26T19:14:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2024-03-26T19:14:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T19:14:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2024-03-26T19:14:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2024-03-26T19:14:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2024-03-26T19:14:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2024-03-26T19:14:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "grafana_dashboards_path": { 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "grafana_dashboards_path", 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "hw_monitoring": { 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hw_monitoring", 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T19:14:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T19:14:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "oob_default_addr": { 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "oob_default_addr", 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "169.254.1.1", 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2024-03-26T19:14:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2024-03-26T19:14:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2024-03-26T19:14:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T19:14:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T19:14:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T19:14:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T19:14:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "{}", 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T19:14:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T19:14:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T19:14:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.903 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.904 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.905 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.906 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.907 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.908 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.909 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.910 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.911 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-26T19:14:40.916 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.917 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.918 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.919 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.920 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.922 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.923 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.924 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.925 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.926 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.928 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.931 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.932 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.933 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.934 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2024-03-26T19:14:40.936 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.937 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.938 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.939 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.940 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:40.941 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.942 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.943 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.944 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.945 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.946 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.947 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.948 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.949 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:40.950 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2024-03-26T19:14:40.951 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2024-03-26T19:14:40.952 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2024-03-26T19:14:40.953 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:40.954 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.955 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.957 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.958 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.959 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.961 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.962 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.965 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.966 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.967 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.969 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.975 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.976 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.977 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.978 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.980 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.981 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.982 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.983 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.985 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.986 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.987 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2024-03-26T19:14:40.989 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2024-03-26T19:14:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.990 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.991 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.994 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:40.995 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.996 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.997 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:40.998 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:40.999 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.000 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.001 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.002 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.003 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.004 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.005 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.006 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.007 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.008 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.009 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2024-03-26T19:14:41.010 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.011 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.012 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.013 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.014 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.015 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2024-03-26T19:14:41.016 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.017 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.018 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.019 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.020 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.021 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.022 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.023 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.024 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.025 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.026 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.027 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.028 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.029 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.030 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.031 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:41.032 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.033 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2024-03-26T19:14:41.034 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-26T19:14:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.036 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.037 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.039 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.040 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.041 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.042 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.043 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.044 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.045 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.046 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.047 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.049 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.050 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.051 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.052 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.053 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.054 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.055 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.057 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.058 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.059 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.060 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.061 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2024-03-26T19:14:41.063 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.064 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.065 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.066 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.067 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.068 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.069 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.070 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.072 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.073 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.074 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.075 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.076 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.077 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2024-03-26T19:14:41.078 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.080 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.081 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.082 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2024-03-26T19:14:41.083 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2024-03-26T19:14:41.084 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.085 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.086 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.087 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.089 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.090 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.091 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.092 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.093 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.094 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.095 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.096 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.097 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.099 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T19:14:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.102 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.103 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.104 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.105 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.106 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.107 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.108 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.109 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.110 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.111 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.113 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.115 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.116 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.117 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.118 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.119 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.120 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.121 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.122 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.123 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.124 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.125 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2024-03-26T19:14:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.129 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.130 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.131 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2024-03-26T19:14:41.135 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.136 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.137 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.138 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.139 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.140 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.144 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.145 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.146 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.147 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.149 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.150 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.152 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.153 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.155 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_no_wait": { 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T19:14:41.156 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2024-03-26T19:14:41.157 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.158 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.159 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.161 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.162 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.163 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.164 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.165 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.166 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.167 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: "modules": [ 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: "dashboard", 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: "iostat", 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: "nfs", 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: "restful" 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2024-03-26T19:14:41.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T19:14:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.170 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.171 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.173 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2024-03-26T19:14:41.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.175 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.176 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.177 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.178 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.179 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T19:14:41.180 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.182 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2024-03-26T19:14:41.183 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-26T19:14:41.184 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.185 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T19:14:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T19:14:41.187 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.188 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.188 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2024-03-26T19:14:41.188 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2024-03-26T19:14:41.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.188 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.189 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.189 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2024-03-26T19:14:41.189 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:41.190 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.191 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.192 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.194 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "read", 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap", 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap-read" 2024-03-26T19:14:41.197 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.198 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.199 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T19:14:41.200 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.201 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.202 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2024-03-26T19:14:41.203 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.204 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.205 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.206 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.207 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2024-03-26T19:14:41.208 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.209 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.210 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T19:14:41.211 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.212 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.213 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.214 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.215 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.217 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2024-03-26T19:14:41.219 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2024-03-26T19:14:41.220 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.221 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.222 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.223 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.224 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.225 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.226 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.227 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.228 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T19:14:41.229 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.230 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.232 INFO:teuthology.orchestra.run.smithi097.stdout: "grafana_dashboards_path": { 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "grafana_dashboards_path", 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2024-03-26T19:14:41.233 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "hw_monitoring": { 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hw_monitoring", 2024-03-26T19:14:41.234 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.235 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.236 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.238 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.240 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.241 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.242 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T19:14:41.243 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2024-03-26T19:14:41.244 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T19:14:41.245 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2024-03-26T19:14:41.246 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2024-03-26T19:14:41.247 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "oob_default_addr": { 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "oob_default_addr", 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "169.254.1.1", 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T19:14:41.249 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T19:14:41.250 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T19:14:41.251 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.252 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.254 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.255 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2024-03-26T19:14:41.256 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.257 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.258 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.259 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.260 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.261 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.262 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.263 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.264 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.265 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2024-03-26T19:14:41.266 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.267 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T19:14:41.268 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T19:14:41.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T19:14:41.270 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.271 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.272 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.273 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.274 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.275 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.276 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.277 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.278 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.279 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T19:14:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T19:14:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2024-03-26T19:14:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "{}", 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T19:14:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2024-03-26T19:14:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2024-03-26T19:14:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-26T19:14:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2024-03-26T19:14:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2024-03-26T19:14:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2024-03-26T19:14:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-26T19:14:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2024-03-26T19:14:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2024-03-26T19:14:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2024-03-26T19:14:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2024-03-26T19:14:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2024-03-26T19:14:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T19:14:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.385 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.386 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.387 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.388 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.389 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.393 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.393 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.393 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-26T19:14:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T19:14:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.418 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2024-03-26T19:14:41.418 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.418 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.418 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.418 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.418 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.419 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.420 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.420 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.420 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.420 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.420 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.420 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.420 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.420 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.420 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.421 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.421 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.421 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.421 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.421 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.421 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.421 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.421 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.422 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.422 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.422 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.422 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.422 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.422 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.422 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.422 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.423 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.423 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.423 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.423 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.423 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.423 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.423 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.423 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.423 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.424 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.424 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.424 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.424 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.424 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.424 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.424 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.424 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.425 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.425 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.425 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.425 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.425 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.425 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.425 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.425 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.425 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.426 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.426 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2024-03-26T19:14:41.426 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.426 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.427 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.427 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.427 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.427 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.427 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.427 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.427 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.427 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.428 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.428 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.428 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.428 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.428 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.428 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.428 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.428 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.428 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.429 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.429 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.429 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.429 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.429 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.429 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.429 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.430 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.430 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.430 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.430 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.430 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.430 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.430 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.430 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.430 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.431 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.431 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.431 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.431 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.431 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.431 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.431 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.431 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.432 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.432 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.432 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.432 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.432 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.432 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.432 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.432 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.432 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.433 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.433 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.433 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.433 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.433 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.433 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.433 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.433 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.434 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.435 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.436 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.437 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.437 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.437 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.437 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.437 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.437 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.438 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.438 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.438 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.438 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.438 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.439 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.439 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.439 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.439 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.439 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.439 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.440 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.440 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.440 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.440 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.440 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.440 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.440 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.441 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.441 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.441 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.441 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.441 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.442 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.442 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.442 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.442 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.442 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.442 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.443 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.443 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.443 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.443 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.443 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.443 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.444 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.444 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.444 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.444 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:41.445 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:41.445 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.445 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.445 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.445 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:41.445 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.446 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.446 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.446 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.446 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.446 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.447 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.447 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2024-03-26T19:14:41.447 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2024-03-26T19:14:41.447 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:41.447 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.447 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.447 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-26T19:14:41.447 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2024-03-26T19:14:41.448 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.448 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.448 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2024-03-26T19:14:41.448 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T19:14:41.448 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.448 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.449 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2024-03-26T19:14:41.449 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.450 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.450 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.450 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2024-03-26T19:14:41.450 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2024-03-26T19:14:41.450 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.450 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.451 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.451 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.451 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.451 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.451 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.452 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T19:14:41.452 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.452 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.452 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.452 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-26T19:14:41.453 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-26T19:14:41.453 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.453 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.453 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.453 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.453 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.454 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.454 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.454 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.454 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.454 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.454 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.455 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.455 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.455 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.455 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.456 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.456 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.456 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.456 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.456 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.456 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.456 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.457 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.457 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.457 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.457 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.457 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.457 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.458 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.458 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.458 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.458 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.458 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.459 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.459 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.459 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.459 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.459 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.459 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.460 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.460 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.460 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.460 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.460 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.461 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.461 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.461 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.461 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.461 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.462 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.462 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.462 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.462 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.462 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.462 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.463 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.463 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.463 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.463 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.463 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.463 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.464 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.464 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.464 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.464 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.464 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.465 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.465 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.465 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.465 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.465 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.465 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.466 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.466 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.466 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.466 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.467 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2024-03-26T19:14:41.467 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2024-03-26T19:14:41.467 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.467 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.467 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.467 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2024-03-26T19:14:41.468 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.468 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.468 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.468 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2024-03-26T19:14:41.468 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.468 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.468 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.469 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-26T19:14:41.469 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-26T19:14:41.469 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.469 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.469 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.469 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-26T19:14:41.469 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.469 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.469 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.470 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2024-03-26T19:14:41.470 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.470 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.470 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.470 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2024-03-26T19:14:41.471 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.471 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.471 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.471 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2024-03-26T19:14:41.471 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2024-03-26T19:14:41.471 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.471 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.471 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.471 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.472 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.472 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.472 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.472 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.472 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.472 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.472 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.473 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2024-03-26T19:14:41.473 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2024-03-26T19:14:41.473 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.473 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.473 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.473 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2024-03-26T19:14:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-26T19:14:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-26T19:14:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-26T19:14:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T19:14:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-26T19:14:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-26T19:14:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2024-03-26T19:14:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2024-03-26T19:14:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2024-03-26T19:14:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2024-03-26T19:14:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2024-03-26T19:14:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2024-03-26T19:14:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2024-03-26T19:14:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2024-03-26T19:14:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T19:14:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2024-03-26T19:14:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2024-03-26T19:14:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2024-03-26T19:14:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2024-03-26T19:14:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_no_wait": { 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-26T19:14:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-26T19:14:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-26T19:14:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-26T19:14:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-26T19:14:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-26T19:14:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "services": { 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "dashboard": "https://172.21.15.97:8443/" 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "always_on_modules": { 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "octopus": [ 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-03-26T19:14:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-03-26T19:14:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-03-26T19:14:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-03-26T19:14:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-03-26T19:14:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-03-26T19:14:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-03-26T19:14:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-03-26T19:14:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "pacific": [ 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-03-26T19:14:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "quincy": [ 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "reef": [ 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-03-26T19:14:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "squid": [ 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-03-26T19:14:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "last_failure_osd_epoch": 3, 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "active_clients": [ 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-03-26T19:14:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2642516573 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "libcephsqlite", 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 302710042 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-03-26T19:14:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1788391090 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2675612236 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-26T19:14:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-26T19:14:41.926 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T19:14:41.927 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:40 smithi105 ceph-mon[36270]: pgmap v863: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 1.60k op/s 2024-03-26T19:14:41.927 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:40 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/830424015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T19:14:41.927 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:40 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/4293487847' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T19:14:41.927 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:40 smithi144 ceph-mon[36053]: pgmap v863: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 1.60k op/s 2024-03-26T19:14:41.928 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:40 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/830424015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T19:14:41.928 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:40 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/4293487847' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T19:14:41.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 -- 172.21.15.97:0/4293487847 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3b00a2fa0 msgr2=0x7fa3b00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3b00a2fa0 0x7fa3b00a5460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc02c260 tx=0x7fa3bc000bc0 comp rx=0 tx=0).stop 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 -- 172.21.15.97:0/4293487847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3cc104ac0 msgr2=0x7fa3cc100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3cc104ac0 0x7fa3cc100140 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005fd0 tx=0x7fa3b800dde0 comp rx=0 tx=0).stop 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 -- 172.21.15.97:0/4293487847 shutdown_connections 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fa3b00a2fa0 0x7fa3b00a5460 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3cc1073f0 0x7fa3cc100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3cc104ac0 0x7fa3cc100140 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 --2- 172.21.15.97:0/4293487847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc102190 0x7fa3cc101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 -- 172.21.15.97:0/4293487847 >> 172.21.15.97:0/4293487847 conn(0x7fa3cc0fbba0 msgr2=0x7fa3cc102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 -- 172.21.15.97:0/4293487847 shutdown_connections 2024-03-26T19:14:41.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:14:40.367+0000 7fa3d1200640 1 -- 172.21.15.97:0/4293487847 wait complete. 2024-03-26T19:14:41.936 DEBUG:tasks.check_counter:Getting stats from a 2024-03-26T19:14:41.936 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-26T19:14:41.936 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:40 smithi097 ceph-mon[31955]: pgmap v863: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 1.60k op/s 2024-03-26T19:14:41.937 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:40 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/830424015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T19:14:41.937 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:40 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/4293487847' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T19:14:42.392 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T19:14:42.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:42 smithi097 ceph-mon[31955]: pgmap v864: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 1.76k op/s 2024-03-26T19:14:42.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:42 smithi144 ceph-mon[36053]: pgmap v864: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 1.76k op/s 2024-03-26T19:14:42.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:42 smithi105 ceph-mon[36270]: pgmap v864: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 1.76k op/s 2024-03-26T19:14:43.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:43 smithi144 ceph-mon[36053]: pgmap v865: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 25 MiB/s wr, 1.57k op/s 2024-03-26T19:14:43.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:43 smithi105 ceph-mon[36270]: pgmap v865: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 25 MiB/s wr, 1.57k op/s 2024-03-26T19:14:44.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:43 smithi097 ceph-mon[31955]: pgmap v865: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 25 MiB/s wr, 1.57k op/s 2024-03-26T19:14:45.679 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T19:14:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:14:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 466, 2024-03-26T19:14:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 686, 2024-03-26T19:14:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 106116, 2024-03-26T19:14:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 289516, 2024-03-26T19:14:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 8, 2024-03-26T19:14:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-03-26T19:14:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.120367326, 2024-03-26T19:14:45.679 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.045705609, 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.055225502, 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.009342800, 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 684, 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.217917264, 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000318592 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1093, 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000291697, 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000266 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.680 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 20, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 245, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 4616, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 622695, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 16, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 7, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.053262375, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.035259952, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.013665387, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000294030, 2024-03-26T19:14:45.681 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 243, 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.071171506, 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000292886 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 11, 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000001794, 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000163 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 570704 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 372, 2024-03-26T19:14:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 354, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 859632, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 200139, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.125083870, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.060410809, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.050304100, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.011334107, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 354, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.102845314, 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000290523 2024-03-26T19:14:45.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 15, 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000011031, 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000735 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 852992, 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 189088 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-03-26T19:14:45.684 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 47, 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.002556567, 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000054395 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000008417, 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000004208 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.685 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 2, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 1, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000164969, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000164969 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 1, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 11, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-03-26T19:14:45.686 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 10, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 10, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 10, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 1, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 1, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-03-26T19:14:45.687 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 1, 2024-03-26T19:14:45.688 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 1028, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 1702093227, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T19:14:45.689 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 1, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-03-26T19:14:45.690 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-03-26T19:14:45.691 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 5, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 5, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 2, 2024-03-26T19:14:45.692 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4195730, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.021487610, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005371902 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-03-26T19:14:45.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 13, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 13, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 0, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 11, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 12, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 1, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 10, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 10, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 0, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 1, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 1, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 51456, 2024-03-26T19:14:45.694 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 223612 2024-03-26T19:14:45.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.695 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-03-26T19:14:45.695 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 2, 2024-03-26T19:14:45.695 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-03-26T19:14:45.695 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 2, 2024-03-26T19:14:45.697 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 17, 2024-03-26T19:14:45.697 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-03-26T19:14:45.697 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-03-26T19:14:45.697 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.697 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.697 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000164969, 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000164969 2024-03-26T19:14:45.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-03-26T19:14:45.701 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.701 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.701 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.701 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-03-26T19:14:45.701 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.702 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.703 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.704 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-03-26T19:14:45.707 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 2, 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 1, 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 947, 2024-03-26T19:14:45.709 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:14:45.710 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:14:45.711 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 248469, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 87, 2024-03-26T19:14:45.712 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7432, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 156, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 51032, 2024-03-26T19:14:45.713 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 200, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 40, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 11728, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 27, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 40, 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-03-26T19:14:45.714 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 40, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.283995202, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.007099880 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 40, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 62 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 40, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 0, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 40, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-03-26T19:14:45.715 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 10, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 0, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 4, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 7, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 6, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 20, 2024-03-26T19:14:45.716 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-03-26T19:14:45.717 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 88, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-03-26T19:14:45.718 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 3, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 9, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 13, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 4, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 14, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 1 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-03-26T19:14:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-03-26T19:14:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-03-26T19:14:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-03-26T19:14:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T19:14:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-03-26T19:14:45.720 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 854, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 872977, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 854, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 854, 2024-03-26T19:14:45.721 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 872977, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-26T19:14:45.722 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 40, 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 11728, 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 40, 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 11728, 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:45.723 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 40, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 40, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 40, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 40, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-03-26T19:14:45.724 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.725 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x556a849c6500": { 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 5, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1426, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 5, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 4, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1426, 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:45.726 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:45.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:45.727 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T19:14:46.647 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-26T19:14:46.647 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-26T19:14:46.647 DEBUG:tasks.check_counter:Getting stats from d 2024-03-26T19:14:46.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-26T19:14:46.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:46 smithi097 ceph-mon[31955]: pgmap v866: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 MiB/s wr, 1.37k op/s 2024-03-26T19:14:46.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:46 smithi144 ceph-mon[36053]: pgmap v866: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 MiB/s wr, 1.37k op/s 2024-03-26T19:14:46.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:46 smithi105 ceph-mon[36270]: pgmap v866: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 MiB/s wr, 1.37k op/s 2024-03-26T19:14:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T19:14:48.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:48 smithi097 ceph-mon[31955]: pgmap v867: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 22 MiB/s wr, 1.28k op/s 2024-03-26T19:14:48.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:48 smithi144 ceph-mon[36053]: pgmap v867: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 22 MiB/s wr, 1.28k op/s 2024-03-26T19:14:48.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:48 smithi105 ceph-mon[36270]: pgmap v867: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 22 MiB/s wr, 1.28k op/s 2024-03-26T19:14:49.464 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T19:14:49.464 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:14:49.464 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1398, 2024-03-26T19:14:49.464 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 977, 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 332895, 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 363480, 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 8, 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.182584640, 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.072212199, 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.135857966, 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.009253227, 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 975, 2024-03-26T19:14:49.465 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.283590935, 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000290862 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2197, 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000725002, 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000329 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:14:49.466 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 384, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 374, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 842700, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 253703, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 11, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.136006830, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.073714842, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.053942048, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.012783382, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 374, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.134308205, 2024-03-26T19:14:49.467 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000359112 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 22, 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000016240, 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000738 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 834768, 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 193888 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 29, 2024-03-26T19:14:49.468 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 259, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 7745, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 583696, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 9, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.061238659, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.049945150, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.007148826, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000664179, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 257, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.095565554, 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000371850 2024-03-26T19:14:49.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 15, 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000002619, 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000174 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 2496, 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 574176 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-03-26T19:14:49.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 55, 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001955630, 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000035556 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-03-26T19:14:49.471 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000022961, 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000011480 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 6, 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 4, 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.003465034, 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000866258 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-03-26T19:14:49.472 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 2, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 11, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 14, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 14, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 14, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 1, 2024-03-26T19:14:49.473 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 1, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 2, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 9, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 7, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 1, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-03-26T19:14:49.474 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 1028, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 1702093227, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T19:14:49.475 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 1, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 2, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 1, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-03-26T19:14:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 1 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 2, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-03-26T19:14:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-03-26T19:14:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-03-26T19:14:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-03-26T19:14:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-03-26T19:14:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-03-26T19:14:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-03-26T19:14:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-03-26T19:14:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-03-26T19:14:49.478 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-03-26T19:14:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-03-26T19:14:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T19:14:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-03-26T19:14:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-03-26T19:14:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-03-26T19:14:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-03-26T19:14:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-03-26T19:14:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-03-26T19:14:49.479 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-03-26T19:14:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 5, 2024-03-26T19:14:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-03-26T19:14:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-03-26T19:14:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 5, 2024-03-26T19:14:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-03-26T19:14:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-03-26T19:14:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 2, 2024-03-26T19:14:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-03-26T19:14:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4195730, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.020593996, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005148499 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 17, 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 17, 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 0, 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 15, 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 16, 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 1, 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 14, 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 14, 2024-03-26T19:14:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 0, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 1, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 4, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 3, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 51200, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 223612 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 10, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 6, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 17, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 1, 2024-03-26T19:14:49.483 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.002806722, 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001403361 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000658312, 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000329156 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-03-26T19:14:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-03-26T19:14:49.488 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-03-26T19:14:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-03-26T19:14:49.489 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.490 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-03-26T19:14:49.491 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.492 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.493 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.494 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.495 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.496 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 2, 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 3, 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 1, 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 953, 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:14:49.497 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:14:49.498 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:14:49.499 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 257317, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 94, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 88, 2024-03-26T19:14:49.500 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 1, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7432, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 156, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-03-26T19:14:49.501 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 67853, 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 299, 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 44, 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 12443, 2024-03-26T19:14:49.502 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 33, 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 44, 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 44, 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.346994111, 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.007886229 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 44, 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 70 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.503 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 44, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 0, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 44, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 10, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 0, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 4, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 7, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-03-26T19:14:49.504 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 6, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 20, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-03-26T19:14:49.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-03-26T19:14:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 88, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 3, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 9, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 10, 2024-03-26T19:14:49.507 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 1, 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 22, 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 1 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-03-26T19:14:49.508 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.509 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 1807, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 999701, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 1807, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1807, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 999701, 2024-03-26T19:14:49.510 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:49.511 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 44, 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 12443, 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 44, 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 12443, 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:49.512 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 44, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 44, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 44, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 44, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.513 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-26T19:14:49.514 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55e6994be500": { 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 5, 2024-03-26T19:14:49.515 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1426, 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 5, 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 4, 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1426, 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:49.516 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T19:14:50.477 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-03-26T19:14:50.477 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-03-26T19:14:50.477 DEBUG:tasks.check_counter:Getting stats from g 2024-03-26T19:14:50.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-26T19:14:50.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:50 smithi097 ceph-mon[31955]: pgmap v868: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.3 MiB/s wr, 520 op/s 2024-03-26T19:14:50.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:50 smithi144 ceph-mon[36053]: pgmap v868: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.3 MiB/s wr, 520 op/s 2024-03-26T19:14:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T19:14:50.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:50 smithi105 ceph-mon[36270]: pgmap v868: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.3 MiB/s wr, 520 op/s 2024-03-26T19:14:52.699 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:52 smithi097 ceph-mon[31955]: pgmap v869: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.8 MiB/s wr, 274 op/s 2024-03-26T19:14:52.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:52 smithi144 ceph-mon[36053]: pgmap v869: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.8 MiB/s wr, 274 op/s 2024-03-26T19:14:52.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:52 smithi105 ceph-mon[36270]: pgmap v869: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.8 MiB/s wr, 274 op/s 2024-03-26T19:14:53.278 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T19:14:53.279 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:14:53.279 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 308, 2024-03-26T19:14:53.279 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 301, 2024-03-26T19:14:53.279 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 781312, 2024-03-26T19:14:53.279 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 185152, 2024-03-26T19:14:53.279 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-03-26T19:14:53.280 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-03-26T19:14:53.280 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.092211410, 2024-03-26T19:14:53.280 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.045880840, 2024-03-26T19:14:53.280 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.032453104, 2024-03-26T19:14:53.280 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.009988003, 2024-03-26T19:14:53.280 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:53.280 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 301, 2024-03-26T19:14:53.280 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.056057918, 2024-03-26T19:14:53.280 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000186238 2024-03-26T19:14:53.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 781312, 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 185152 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2, 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 229, 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 320, 2024-03-26T19:14:53.281 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 196240, 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.041589194, 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.034614082, 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.003862848, 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 227, 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.038317945, 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000168801 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:53.282 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:53.283 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:53.283 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:53.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.283 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T19:14:53.283 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 196240 2024-03-26T19:14:53.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.283 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:14:53.283 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2024-03-26T19:14:53.283 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2024-03-26T19:14:53.283 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.000975625, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000788259, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.284 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-03-26T19:14:53.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:14:53.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:14:53.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:14:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148704, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 29, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-03-26T19:14:53.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:14:53.290 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:14:53.290 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-03-26T19:14:53.290 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-03-26T19:14:53.290 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-03-26T19:14:53.290 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-03-26T19:14:53.290 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 306, 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 735351, 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:53.291 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 306, 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 306, 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 735351, 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:53.292 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T19:14:54.144 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-26T19:14:54.144 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-26T19:14:54.144 DEBUG:tasks.check_counter:Getting stats from j 2024-03-26T19:14:54.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-26T19:14:54.605 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T19:14:54.755 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:54 smithi097 ceph-mon[31955]: pgmap v870: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 67 KiB/s wr, 21 op/s 2024-03-26T19:14:54.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:54 smithi144 ceph-mon[36053]: pgmap v870: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 67 KiB/s wr, 21 op/s 2024-03-26T19:14:54.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:54 smithi105 ceph-mon[36270]: pgmap v870: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 67 KiB/s wr, 21 op/s 2024-03-26T19:14:56.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:56 smithi097 ceph-mon[31955]: pgmap v871: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-03-26T19:14:56.882 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-26T19:14:56.882 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:14:56.882 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 239, 2024-03-26T19:14:56.882 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 233, 2024-03-26T19:14:56.882 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 761136, 2024-03-26T19:14:56.882 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 142432, 2024-03-26T19:14:56.882 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.079470384, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.029839361, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.044296774, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.008913611, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 232, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.040187956, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000173223 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 761136, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 142432 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 79, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 310, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 162000, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 168256, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.056777337, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.038071683, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.014223723, 2024-03-26T19:14:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.002680102, 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 308, 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.063404931, 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000205860 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 162000, 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 168256 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:14:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 3, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 3, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.001728331, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.001467645, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:56.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:14:56.887 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-03-26T19:14:56.888 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-03-26T19:14:56.889 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148951, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 28, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-03-26T19:14:56.890 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-26T19:14:56.891 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 314, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 875740, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 314, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 314, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 875740, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:56.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-26T19:14:56.893 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-26T19:14:56.893 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:56 smithi144 ceph-mon[36053]: pgmap v871: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-03-26T19:14:56.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:56 smithi105 ceph-mon[36270]: pgmap v871: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-03-26T19:14:57.701 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-26T19:14:57.701 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-26T19:14:57.702 DEBUG:tasks.check_counter:Getting stats from b 2024-03-26T19:14:57.702 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-26T19:14:57.949 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T19:14:58.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:14:58 smithi105 ceph-mon[36270]: pgmap v872: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-03-26T19:14:58.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:14:58 smithi097 ceph-mon[31955]: pgmap v872: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-03-26T19:14:58.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:14:58 smithi144 ceph-mon[36053]: pgmap v872: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-03-26T19:14:59.745 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T19:14:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:14:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 5058, 2024-03-26T19:14:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 5259, 2024-03-26T19:14:59.745 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1563443, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 12884829, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 19, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.428890182, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.146845983, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.118713371, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.131740107, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 5261, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.543052538, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000293300 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.746 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3047, 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000597308, 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000196 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 2977, 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 2989, 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 843728, 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 5758922, 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 11, 2024-03-26T19:14:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.241372016, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.091448533, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.054815310, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.076453474, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2987, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.068027802, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000692342 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1085, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000215265, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000198 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 578768 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 13736, 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 14995, 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 4951121, 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 14826884, 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 147, 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 9, 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.876204617, 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.334453017, 2024-03-26T19:14:59.749 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.776814093, 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.140296816, 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 14991, 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.267558581, 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000084554 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 12282, 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002907099, 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000236 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 948256, 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 223088 2024-03-26T19:14:59.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6340, 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 5.045233143, 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000795778 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-26T19:14:59.751 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 934, 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.033429832, 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000035792 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 4707, 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 4707, 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4707, 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 15.437815032, 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.003279756 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.752 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 1, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 1271, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 3, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 1164, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 9, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 1155, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 1151, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 1059, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 1059, 2024-03-26T19:14:59.753 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 3, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 8236, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 6875, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 1, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 415, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-26T19:14:59.754 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 1026, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 1702083777, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T19:14:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 1, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 558, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 2999, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 1, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 4377, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 2007, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 198, 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 1637 2024-03-26T19:14:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 5, 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 6, 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 488, 2024-03-26T19:14:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 266, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 216, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 3, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-26T19:14:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 7365, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 7142, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 7142, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 223, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 119, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 77, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 76, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 76, 2024-03-26T19:14:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 1, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 1, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 1, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 22121549, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 22721499, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3023, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 23.852847711, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.007890455 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 1083, 2024-03-26T19:14:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 1346, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 263, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 67, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 74, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 7, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 1163, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 2077, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 914, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 1059, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 1586, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 527, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 71712, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 231804 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-26T19:14:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 5651, 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 1, 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 4707, 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 457, 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 2, 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1298, 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 8.283152824, 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.006381473 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 18, 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003897039, 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000216502 2024-03-26T19:14:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 216, 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.627019536, 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002902868 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1700, 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.310063256, 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001358860 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000156451, 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000156451 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-26T19:14:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-26T19:14:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 37, 2024-03-26T19:14:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.651015693, 2024-03-26T19:14:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.017595018 2024-03-26T19:14:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 644, 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.351878600, 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000546395 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 65, 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.138989087, 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002138293 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 456, 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.483115729, 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.005445429 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.012086650, 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.006043325 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-26T19:14:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.008079882, 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.004039941 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 8, 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.022323808, 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002790476 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-26T19:14:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 260, 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.546036477, 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002100140 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 3, 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 2, 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 1156, 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 1156, 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 963, 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-26T19:14:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:14:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-26T19:14:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 383705, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 1204, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 880, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 10, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-26T19:14:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7432, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 156, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 3872292, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 53423, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-26T19:14:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 7669, 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 22721571, 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 32, 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 7669, 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 7669, 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 138.816377396, 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.018100975 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.778 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 7669, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 13658 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 7669, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 107, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 7562, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 1198, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 1515, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 106, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 3129, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 372, 2024-03-26T19:14:59.779 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 3, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 800, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 1, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 3030, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-26T19:14:59.780 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 1, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-26T19:14:59.781 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 88, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 4, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 12, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 12, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 3209, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 292 2024-03-26T19:14:59.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 13, 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 6, 2024-03-26T19:14:59.783 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 280, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 266, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 21767, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 5535758, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 21767, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:14:59.784 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 21767, 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 5535758, 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:14:59.785 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 7669, 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 22748421, 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 7669, 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 22748421, 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.786 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 7669, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 7669, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 7669, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 7669, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.787 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 266, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 26866, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 266, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 106, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 26866, 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:14:59.788 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x5604484c0500": { 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 9875, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 7365, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 18527195, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 7365, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 3023, 2024-03-26T19:14:59.789 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 18517320, 2024-03-26T19:14:59.790 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:14:59.790 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:14:59.790 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:14:59.790 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:14:59.790 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:14:59.790 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:14:59.790 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T19:15:00.448 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-26T19:15:00.448 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-26T19:15:00.448 DEBUG:tasks.check_counter:Getting stats from e 2024-03-26T19:15:00.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-26T19:15:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T19:15:00.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:00 smithi105 ceph-mon[36270]: pgmap v873: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-03-26T19:15:00.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:00 smithi097 ceph-mon[31955]: pgmap v873: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-03-26T19:15:00.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:00 smithi144 ceph-mon[36053]: pgmap v873: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-03-26T19:15:02.496 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 372, 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 360, 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 846287, 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 248218, 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 11, 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.044147768, 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.014845516, 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.026296094, 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.006714604, 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:02.497 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 359, 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.039301414, 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000109474 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000293, 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000073 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 842464, 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 195360 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 19, 2024-03-26T19:15:02.498 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 250, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 6795, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 660663, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 10, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.013346726, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.008349450, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.003151956, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000208294, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 249, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.035483208, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000142502 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:02.499 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 654992 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 449, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 666, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 100063, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 123826, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 9, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-26T19:15:02.500 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.061206142, 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.031439684, 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.024099376, 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.004414838, 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 664, 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.048678296, 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000073310 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1082, 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000237567, 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000219 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.501 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 47, 2024-03-26T19:15:02.502 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001047015, 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000022276 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000006187, 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000003093 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-03-26T19:15:02.503 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 11, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 10, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 10, 2024-03-26T19:15:02.504 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 10, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 1, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-26T19:15:02.505 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 1028, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 1702093227, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-26T19:15:02.506 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-03-26T19:15:02.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 1, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-26T19:15:02.508 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 2, 2024-03-26T19:15:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 2, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4194892, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-26T19:15:02.510 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001982224, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001982224 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 13, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 13, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 11, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 12, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 1, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 10, 2024-03-26T19:15:02.511 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 10, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 49516, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 223612 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.512 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.513 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.514 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.515 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.516 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-26T19:15:02.516 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.516 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.516 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.516 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-26T19:15:02.516 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.517 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.517 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.517 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.517 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-26T19:15:02.517 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.517 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.517 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.517 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.517 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-26T19:15:02.517 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.518 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.521 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.522 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 0, 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 0, 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 0, 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 0, 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-26T19:15:02.524 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-26T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 230743, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 73, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7432, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 156, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 50664, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 198, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-26T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 35, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 10710, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 33, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 35, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 35, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.280994604, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.008028417 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 35, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 57 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 35, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 0, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 35, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 10, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 0, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 1, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 5, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-26T19:15:02.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 6, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 20, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-26T19:15:02.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-26T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 88, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 3, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 8, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 13, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 5, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 14, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 1 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-26T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 836, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 856710, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 836, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 836, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 856710, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.535 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 35, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 10710, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 35, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 10710, 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:02.536 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 35, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 35, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 35, 2024-03-26T19:15:02.537 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 35, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:02.538 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55976d6e8500": { 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 2, 2024-03-26T19:15:02.539 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 588, 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 2, 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1, 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 588, 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:02.540 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T19:15:02.796 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:02 smithi105 ceph-mon[36270]: pgmap v874: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-03-26T19:15:02.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:02 smithi097 ceph-mon[31955]: pgmap v874: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-03-26T19:15:02.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:02 smithi144 ceph-mon[36053]: pgmap v874: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-03-26T19:15:03.356 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-26T19:15:03.356 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-26T19:15:03.356 DEBUG:tasks.check_counter:Getting stats from h 2024-03-26T19:15:03.356 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-26T19:15:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T19:15:04.778 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:04 smithi105 ceph-mon[36270]: pgmap v875: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-03-26T19:15:04.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:04 smithi097 ceph-mon[31955]: pgmap v875: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-03-26T19:15:04.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:04 smithi144 ceph-mon[36053]: pgmap v875: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-03-26T19:15:05.734 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 220, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 222, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1785302, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 69970, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.020346479, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.005716417, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.006019659, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.005972632, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 220, 2024-03-26T19:15:05.735 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.019423082, 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000088286 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 137, 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 134, 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 191652, 2024-03-26T19:15:05.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 49323, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.016239185, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.004792111, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.008081625, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.003926551, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 134, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.037361690, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000278818 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 167856, 2024-03-26T19:15:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 19056 2024-03-26T19:15:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:15:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 4, 2024-03-26T19:15:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 26, 2024-03-26T19:15:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1555, 2024-03-26T19:15:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 122788, 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 5, 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.002416544, 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000915867, 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.001193230, 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000087257, 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 24, 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.011303519, 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000470979 2024-03-26T19:15:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 121824 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-26T19:15:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 532, 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.017125319, 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000032190 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-26T19:15:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 292, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 98, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 194, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 158, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 2, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-03-26T19:15:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-26T19:15:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 1026, 2024-03-26T19:15:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 1704066747, 2024-03-26T19:15:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-26T19:15:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T19:15:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-26T19:15:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T19:15:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-26T19:15:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-26T19:15:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T19:15:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T19:15:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-26T19:15:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-26T19:15:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 635, 2024-03-26T19:15:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 6, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 22121549, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 22749040, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 22747902, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 635 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 141, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 173, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 32, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 31, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 31, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 0, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 292, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 292, 2024-03-26T19:15:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 48988, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215396 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-26T19:15:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-26T19:15:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.758 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.759 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-26T19:15:05.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.760 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.760 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-26T19:15:05.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.760 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.761 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-26T19:15:05.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-26T19:15:05.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-26T19:15:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.765 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-26T19:15:05.766 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 1, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 0, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-03-26T19:15:05.767 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 2529, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:15:05.768 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-26T19:15:05.769 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 201255, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 177, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-26T19:15:05.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7432, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 156, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 579699, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 14236, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-26T19:15:05.771 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 318, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 26, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 4, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 318, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 318, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 5.102906677, 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.016046876 2024-03-26T19:15:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 318, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 319 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 318, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 318, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 186, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 129, 2024-03-26T19:15:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 2, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-26T19:15:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-26T19:15:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 6, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 6, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-03-26T19:15:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-26T19:15:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 357, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1941430, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 357, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 357, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1941430, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-26T19:15:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 318, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 1720713, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 318, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1720713, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 318, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 318, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 318, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 318, 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:15:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55c3067946e0": { 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:05.784 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T19:15:06.537 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-26T19:15:06.537 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-26T19:15:06.537 DEBUG:tasks.check_counter:Getting stats from k 2024-03-26T19:15:06.537 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-26T19:15:06.800 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T19:15:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:06 smithi097 ceph-mon[31955]: pgmap v876: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-03-26T19:15:06.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:06 smithi144 ceph-mon[36053]: pgmap v876: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-03-26T19:15:06.921 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:06 smithi105 ceph-mon[36270]: pgmap v876: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-03-26T19:15:08.400 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T19:15:08.400 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:15:08.400 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 2230, 2024-03-26T19:15:08.400 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 2232, 2024-03-26T19:15:08.400 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 667287, 2024-03-26T19:15:08.400 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 709141, 2024-03-26T19:15:08.400 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.155382583, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.051207793, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.040136242, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.051291336, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2230, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.087651363, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000039305 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 5, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 240, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1710, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 577598, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 8, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.011469974, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.008314714, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.001448803, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000057636, 2024-03-26T19:15:08.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 238, 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.015565894, 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000065402 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 576576 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 331, 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 321, 2024-03-26T19:15:08.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 821004, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 193886, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.040388348, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.013842582, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.022379608, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.006471392, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 321, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.010567166, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000032919 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.404 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 820736, 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 193568 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4461, 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.099486755, 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000022301 2024-03-26T19:15:08.405 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-26T19:15:08.407 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 1, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-26T19:15:08.408 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-26T19:15:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-26T19:15:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-26T19:15:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 2, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4194892, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 4194352, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.413 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 2 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 3, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 3, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 1, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 1, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 0, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 0, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 0, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-03-26T19:15:08.414 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 48168, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215420 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.415 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-26T19:15:08.416 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.417 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.418 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-26T19:15:08.418 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.418 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.418 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.418 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-26T19:15:08.418 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.418 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.418 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-26T19:15:08.419 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.420 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.420 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.421 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-26T19:15:08.422 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-26T19:15:08.423 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.424 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.425 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 0, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 0, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 0, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-26T19:15:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 0, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:15:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:15:08.429 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:15:08.429 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:15:08.429 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:15:08.429 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:15:08.429 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:15:08.429 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-26T19:15:08.430 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 190145, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 52, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-26T19:15:08.431 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7432, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 156, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 8464, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 13, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-26T19:15:08.432 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 2234, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 21, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 3, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 2234, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2234, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.744949447, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001228715 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.433 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2234, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2235 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 2234, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 2234, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 1114, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 1118, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-03-26T19:15:08.434 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 1, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-26T19:15:08.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-26T19:15:08.436 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 88, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 3, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 5, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 8, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 3, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-03-26T19:15:08.437 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T19:15:08.438 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-26T19:15:08.439 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-26T19:15:08.439 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-26T19:15:08.439 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-26T19:15:08.439 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-26T19:15:08.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.439 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:15:08.439 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:08.439 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T19:15:08.439 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:08.439 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 2562, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1239623, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 2562, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2562, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1239623, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:08.440 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 2234, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 593, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2234, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 593, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.441 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 2234, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 2234, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2234, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 2234, 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:08.442 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:15:08.443 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x5654b95206e0": { 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:08.444 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T19:15:08.445 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T19:15:08.445 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-26T19:15:08.445 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-26T19:15:08.445 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T19:15:08.445 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T19:15:08.445 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T19:15:08.445 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T19:15:08.445 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:08.445 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T19:15:08.445 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T19:15:08.717 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:08 smithi105 ceph-mon[36270]: pgmap v877: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-26T19:15:08.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:08 smithi097 ceph-mon[31955]: pgmap v877: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-26T19:15:08.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:08 smithi144 ceph-mon[36053]: pgmap v877: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-26T19:15:09.445 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-26T19:15:09.445 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-26T19:15:09.445 DEBUG:tasks.check_counter:Getting stats from c 2024-03-26T19:15:09.446 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-26T19:15:09.703 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T19:15:10.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:10 smithi144 ceph-mon[36053]: pgmap v878: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-03-26T19:15:10.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:10 smithi097 ceph-mon[31955]: pgmap v878: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-03-26T19:15:10.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:10 smithi105 ceph-mon[36270]: pgmap v878: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-03-26T19:15:11.460 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1445, 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1432, 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1217664, 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 546084, 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 1, 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.133865443, 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.040718947, 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.048319927, 2024-03-26T19:15:11.461 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.039365460, 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1431, 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.061173571, 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000042748 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 842016, 2024-03-26T19:15:11.462 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 190560 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 5, 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 238, 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1036, 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 656682, 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.013284548, 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.008310440, 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.003255927, 2024-03-26T19:15:11.463 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000054099, 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 235, 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.013633460, 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000058014 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 655984 2024-03-26T19:15:11.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 2099, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 2099, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 538353, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 666665, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.115166067, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.032619200, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.029234762, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.046726550, 2024-03-26T19:15:11.465 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2098, 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.092546459, 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000044111 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T19:15:11.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5366, 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.118103222, 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000022009 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:11.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-26T19:15:11.468 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-26T19:15:11.469 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 1, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-26T19:15:11.470 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-26T19:15:11.471 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-26T19:15:11.472 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-26T19:15:11.473 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-26T19:15:11.474 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-26T19:15:11.475 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 5, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-26T19:15:11.476 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4195730, 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 4195589, 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 5 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 3, 2024-03-26T19:15:11.477 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 3, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 0, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 1, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 1, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 0, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 0, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 0, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 0, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 49964, 2024-03-26T19:15:11.478 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215420 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.479 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.480 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.481 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.482 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.483 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-26T19:15:11.484 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.484 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.485 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.486 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-26T19:15:11.487 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.487 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.487 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.489 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-26T19:15:11.489 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.489 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.489 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.489 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.491 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.492 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.493 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 2, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 1, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 2, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 4094, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:15:11.494 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-26T19:15:11.495 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-26T19:15:11.496 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 182105, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 49, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7432, 2024-03-26T19:15:11.497 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 156, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 8464, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 13, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-26T19:15:11.498 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 3215, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 21, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 3, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 3215, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3215, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 4.404935557, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001370119 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3215, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 3216 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 3215, 2024-03-26T19:15:11.499 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 3215, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 2089, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 1124, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-26T19:15:11.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-26T19:15:11.501 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-26T19:15:11.502 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 88, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 3, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 4, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 5, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 1, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-26T19:15:11.503 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:11.504 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 3545, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1418135, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 3545, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3545, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1418135, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.505 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 3215, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 1431, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3215, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1431, 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:11.506 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 3215, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 3215, 2024-03-26T19:15:11.507 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3215, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 3215, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:11.508 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x56354a7526e0": { 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:11.509 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:11.510 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-26T19:15:12.224 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-26T19:15:12.224 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-26T19:15:12.225 DEBUG:tasks.check_counter:Getting stats from f 2024-03-26T19:15:12.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-26T19:15:12.487 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T19:15:12.776 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:12 smithi144 ceph-mon[36053]: pgmap v879: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-03-26T19:15:12.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:12 smithi097 ceph-mon[31955]: pgmap v879: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-03-26T19:15:12.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:12 smithi105 ceph-mon[36270]: pgmap v879: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-03-26T19:15:14.112 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-26T19:15:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:15:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1121, 2024-03-26T19:15:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1121, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 297389, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 355718, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 2, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.074056777, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.020863745, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.020955365, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.024960263, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1120, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.033148893, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000029597 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1295, 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1282, 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1080218, 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 499033, 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-26T19:15:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.107942511, 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.030246918, 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.039404896, 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.036264893, 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1281, 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.042582092, 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000033241 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 842560, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 191840 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1112, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1346, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 373456, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 845330, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.103329079, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.035066062, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.026604017, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.032481294, 2024-03-26T19:15:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1344, 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.062446080, 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000046462 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 492032 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-26T19:15:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5380, 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.122535084, 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000022776 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-26T19:15:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 1, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-26T19:15:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 0, 2024-03-26T19:15:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T19:15:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-26T19:15:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-26T19:15:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 4, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-03-26T19:15:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4195424, 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 4195283, 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 4 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-26T19:15:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 3, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 3, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 0, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 1, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 1, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 0, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 0, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 0, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 0, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 49688, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215420 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-26T19:15:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-26T19:15:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 0, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 1, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 1, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 1621, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-26T19:15:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:15:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:15:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 190145, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 52, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7432, 2024-03-26T19:15:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 156, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 8464, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 13, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-26T19:15:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 3194, 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 21, 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 3, 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 3194, 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3194, 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 15.022780724, 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.004703437 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3194, 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 3195 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 3194, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 3194, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 2075, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 1117, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-26T19:15:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-26T19:15:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-26T19:15:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 88, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 3, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 5, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 6, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 1, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-26T19:15:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 3524, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1414017, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 3524, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3524, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1414017, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 3194, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 1125, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3194, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1125, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 3194, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 3194, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3194, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 3194, 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x563e32b026e0": { 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:14.155 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-26T19:15:14.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:14 smithi097 ceph-mon[31955]: pgmap v880: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-03-26T19:15:14.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:14 smithi144 ceph-mon[36053]: pgmap v880: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-03-26T19:15:14.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:14 smithi105 ceph-mon[36270]: pgmap v880: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-03-26T19:15:15.035 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-26T19:15:15.036 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-26T19:15:15.036 DEBUG:tasks.check_counter:Getting stats from i 2024-03-26T19:15:15.036 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-26T19:15:15.287 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T19:15:16.672 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:16 smithi144 ceph-mon[36053]: pgmap v881: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-03-26T19:15:16.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:16 smithi097 ceph-mon[31955]: pgmap v881: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-03-26T19:15:16.935 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 467, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 689, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 108210, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 133872, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 8, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.049732644, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.017912175, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.024663556, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.005206029, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 687, 2024-03-26T19:15:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.065800435, 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000095779 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1098, 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000256188, 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000233 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 371, 2024-03-26T19:15:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 355, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 855534, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 207592, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 6, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.041401304, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.013505561, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.022934501, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.006267833, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 355, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.028212586, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000079472 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 851744, 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 195488 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 15, 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 247, 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 3218, 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 629492, 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 14, 2024-03-26T19:15:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.013802300, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.008189064, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.003749372, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000143368, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 245, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.018140420, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000074042 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 11, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000001231, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000111 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 580016 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 48, 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.001600205, 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000033337 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-26T19:15:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T19:15:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T19:15:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-03-26T19:15:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000008641, 2024-03-26T19:15:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000004320 2024-03-26T19:15:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-26T19:15:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 11, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-26T19:15:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 10, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 10, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 10, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 1, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-26T19:15:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 1028, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 1702093227, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-26T19:15:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-26T19:15:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 1, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-26T19:15:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-26T19:15:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-26T19:15:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 4, 2024-03-26T19:15:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-26T19:15:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-26T19:15:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 4, 2024-03-26T19:15:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-26T19:15:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-26T19:15:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 2, 2024-03-26T19:15:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-26T19:15:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4195424, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.012717246, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.004239082 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-26T19:15:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 13, 2024-03-26T19:15:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 13, 2024-03-26T19:15:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 0, 2024-03-26T19:15:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 11, 2024-03-26T19:15:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 12, 2024-03-26T19:15:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 1, 2024-03-26T19:15:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 10, 2024-03-26T19:15:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 10, 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 0, 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 50028, 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 223612 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-26T19:15:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-03-26T19:15:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 4, 2024-03-26T19:15:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-26T19:15:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-26T19:15:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-26T19:15:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-26T19:15:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-26T19:15:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-26T19:15:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-26T19:15:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-26T19:15:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-26T19:15:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 0, 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 1, 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 1, 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 0, 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-26T19:15:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 59, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:15:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:15:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:15:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 230951, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 75, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7432, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 156, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:15:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 50664, 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 198, 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-26T19:15:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 39, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 11422, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 27, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 39, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 39, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.328994820, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.008435764 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 39, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 61 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.971 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 39, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 0, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 39, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 0, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 10, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 0, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 3, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 7, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 6, 2024-03-26T19:15:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-26T19:15:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-26T19:15:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-26T19:15:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 0, 2024-03-26T19:15:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 20, 2024-03-26T19:15:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-26T19:15:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-26T19:15:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-26T19:15:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-26T19:15:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-26T19:15:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-26T19:15:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 88, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 3, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 8, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 11, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 3, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 14, 2024-03-26T19:15:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 1 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T19:15:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 849, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 869355, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 849, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 849, 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 869355, 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 39, 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 11422, 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 39, 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 11422, 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-26T19:15:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-26T19:15:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 39, 2024-03-26T19:15:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 39, 2024-03-26T19:15:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 39, 2024-03-26T19:15:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 39, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5570bdaa4500": { 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 4, 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1120, 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 4, 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3, 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1120, 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:16.984 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-26T19:15:16.984 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:16 smithi105 ceph-mon[36270]: pgmap v881: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-03-26T19:15:17.601 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-26T19:15:17.601 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-26T19:15:17.601 DEBUG:tasks.check_counter:Getting stats from l 2024-03-26T19:15:17.602 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-26T19:15:17.869 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T19:15:18.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:18 smithi097 ceph-mon[31955]: pgmap v882: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 13 op/s 2024-03-26T19:15:18.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:18 smithi144 ceph-mon[36053]: pgmap v882: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 13 op/s 2024-03-26T19:15:18.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:18 smithi105 ceph-mon[36270]: pgmap v882: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 13 op/s 2024-03-26T19:15:19.527 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-26T19:15:19.527 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T19:15:19.527 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1122, 2024-03-26T19:15:19.527 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1124, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 376937, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 356797, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 2, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.095362378, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.026913548, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.026348852, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.033278990, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1122, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.063183431, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000056313 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 2437, 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 2428, 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1358472, 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 864935, 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-26T19:15:19.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.165452541, 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.048398956, 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.054254189, 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.057220301, 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2428, 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.111735205, 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000046019 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 821216, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 195488 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 5, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 244, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1668, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 661237, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 8, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.011269101, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.007884366, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.001629719, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000053001, 2024-03-26T19:15:19.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 242, 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.012412473, 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000051291 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 660272 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-26T19:15:19.532 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5465, 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.119967852, 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000021952 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.533 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-26T19:15:19.534 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 1, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-26T19:15:19.535 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 0, 2024-03-26T19:15:19.536 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T19:15:19.537 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-26T19:15:19.538 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-26T19:15:19.539 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 5, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-26T19:15:19.540 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4195730, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 4195589, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 5 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 3, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 3, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 0, 2024-03-26T19:15:19.541 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 1, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 1, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 0, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 0, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 0, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 0, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 49284, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215420 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-26T19:15:19.542 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.543 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-26T19:15:19.546 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.547 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.547 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.547 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-26T19:15:19.547 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.547 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.547 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.547 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.548 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.549 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.551 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.552 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-03-26T19:15:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 2, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 1, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 2, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 4099, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T19:15:19.554 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T19:15:19.555 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-26T19:15:19.556 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 198225, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 56, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7432, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 156, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-26T19:15:19.557 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 8464, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 13, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 3230, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 21, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 3, 2024-03-26T19:15:19.558 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 3230, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3230, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 8.330878402, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002579219 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3230, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 3231 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 3230, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 3230, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-26T19:15:19.559 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 2100, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 1128, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-26T19:15:19.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-26T19:15:19.561 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 88, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 3, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 6, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 8, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 2, 2024-03-26T19:15:19.562 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-26T19:15:19.563 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 3561, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1396969, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 3561, 2024-03-26T19:15:19.564 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3561, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1396969, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:19.565 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 3230, 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 1431, 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3230, 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1431, 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.566 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 3230, 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 3230, 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3230, 2024-03-26T19:15:19.567 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 3230, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:19.568 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5574d4bdc6e0": { 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T19:15:19.569 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T19:15:19.570 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-26T19:15:20.325 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-26T19:15:20.325 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-26T19:15:20.326 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_ljflores_ceph_72e2437271dcafb9826f2f49bb1001426fa6de81/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} 2024-03-26T19:15:20.326 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-26T19:15:20.339 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-26T19:15:20.340 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T19:15:20.340 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T19:15:20.400 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T19:15:20.400 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T19:15:20.449 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T19:15:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.693+0000 7fb72b200640 1 -- 172.21.15.97:0/4076740158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb724109a50 msgr2=0x7fb724110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.693+0000 7fb72b200640 1 --2- 172.21.15.97:0/4076740158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb724109a50 0x7fb724110370 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009ca0 tx=0x7fb71c02c0f0 comp rx=0 tx=0).stop 2024-03-26T19:15:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.694+0000 7fb72b200640 1 -- 172.21.15.97:0/4076740158 shutdown_connections 2024-03-26T19:15:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.694+0000 7fb72b200640 1 --2- 172.21.15.97:0/4076740158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb724109a50 0x7fb724110370 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.694+0000 7fb72b200640 1 --2- 172.21.15.97:0/4076740158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb724109090 0x7fb724109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.694+0000 7fb72b200640 1 --2- 172.21.15.97:0/4076740158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb724107eb0 0x7fb7241082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.694+0000 7fb72b200640 1 -- 172.21.15.97:0/4076740158 >> 172.21.15.97:0/4076740158 conn(0x7fb724076ed0 msgr2=0x7fb7240792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.694+0000 7fb72b200640 1 -- 172.21.15.97:0/4076740158 shutdown_connections 2024-03-26T19:15:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.694+0000 7fb72b200640 1 -- 172.21.15.97:0/4076740158 wait complete. 2024-03-26T19:15:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.695+0000 7fb72b200640 1 Processor -- start 2024-03-26T19:15:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.695+0000 7fb72b200640 1 -- start start 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb72b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb724107eb0 0x7fb72419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb72b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb724109090 0x7fb72419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb72b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb724109a50 0x7fb7241a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb72b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb724113b80 con 0x7fb724109a50 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb72b200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb724113a00 con 0x7fb724107eb0 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb72b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb724113d00 con 0x7fb724109090 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb723e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb724107eb0 0x7fb72419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb723e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb724107eb0 0x7fb72419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:45746/0 (socket says 172.21.15.97:45746) 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb723e00640 1 -- 172.21.15.97:0/648074952 learned_addr learned my addr 172.21.15.97:0/648074952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb723400640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb724109090 0x7fb72419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb729200640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb724109a50 0x7fb7241a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb723e00640 1 -- 172.21.15.97:0/648074952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb724109090 msgr2=0x7fb72419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:20.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb723e00640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb724109090 0x7fb72419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:20.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb723e00640 1 -- 172.21.15.97:0/648074952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb724109a50 msgr2=0x7fb7241a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:20.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.696+0000 7fb723e00640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb724109a50 0x7fb7241a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:20.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.697+0000 7fb723e00640 1 -- 172.21.15.97:0/648074952 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7241a5db0 con 0x7fb724107eb0 2024-03-26T19:15:20.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.697+0000 7fb723e00640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb724107eb0 0x7fb72419e190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e9c0 tx=0x7fb71400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:20.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.697+0000 7fb720c00640 1 -- 172.21.15.97:0/648074952 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71400cd60 con 0x7fb724107eb0 2024-03-26T19:15:20.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.697+0000 7fb720c00640 1 -- 172.21.15.97:0/648074952 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb714010910 con 0x7fb724107eb0 2024-03-26T19:15:20.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.697+0000 7fb72b200640 1 -- 172.21.15.97:0/648074952 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7241a60a0 con 0x7fb724107eb0 2024-03-26T19:15:20.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.697+0000 7fb720c00640 1 -- 172.21.15.97:0/648074952 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714018970 con 0x7fb724107eb0 2024-03-26T19:15:20.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.697+0000 7fb72b200640 1 -- 172.21.15.97:0/648074952 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7241a65f0 con 0x7fb724107eb0 2024-03-26T19:15:20.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.698+0000 7fb72b200640 1 -- 172.21.15.97:0/648074952 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72410f1f0 con 0x7fb724107eb0 2024-03-26T19:15:20.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.700+0000 7fb720c00640 1 -- 172.21.15.97:0/648074952 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fb71401f020 con 0x7fb724107eb0 2024-03-26T19:15:20.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.702+0000 7fb720c00640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7080a2f70 0x7fb7080a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:20.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.702+0000 7fb720c00640 1 -- 172.21.15.97:0/648074952 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fb7140cf660 con 0x7fb724107eb0 2024-03-26T19:15:20.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.702+0000 7fb723400640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7080a2f70 0x7fb7080a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:20.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.703+0000 7fb723400640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7080a2f70 0x7fb7080a5430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb72419f6a0 tx=0x7fb710005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:20.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.708+0000 7fb720c00640 1 -- 172.21.15.97:0/648074952 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb71409c460 con 0x7fb724107eb0 2024-03-26T19:15:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:20 smithi097 ceph-mon[31955]: pgmap v883: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-03-26T19:15:20.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:20 smithi144 ceph-mon[36053]: pgmap v883: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-03-26T19:15:20.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.910+0000 7fb72b200640 1 -- 172.21.15.97:0/648074952 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7240624f0 con 0x7fb724107eb0 2024-03-26T19:15:20.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.912+0000 7fb720c00640 1 -- 172.21.15.97:0/648074952 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22581 (secure 0 0 0) 0x7fb71409bd60 con 0x7fb724107eb0 2024-03-26T19:15:20.913 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:15:20.914 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T19:13:27.698010+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T19:15:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 -- 172.21.15.97:0/648074952 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7080a2f70 msgr2=0x7fb7080a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7080a2f70 0x7fb7080a5430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb72419f6a0 tx=0x7fb710005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 -- 172.21.15.97:0/648074952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb724107eb0 msgr2=0x7fb72419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb724107eb0 0x7fb72419e190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e9c0 tx=0x7fb71400ee80 comp rx=0 tx=0).stop 2024-03-26T19:15:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 -- 172.21.15.97:0/648074952 shutdown_connections 2024-03-26T19:15:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fb7080a2f70 0x7fb7080a5430 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb724109a50 0x7fb7241a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb724109090 0x7fb72419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 --2- 172.21.15.97:0/648074952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb724107eb0 0x7fb72419e190 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 -- 172.21.15.97:0/648074952 >> 172.21.15.97:0/648074952 conn(0x7fb724076ed0 msgr2=0x7fb72410c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:20.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 -- 172.21.15.97:0/648074952 shutdown_connections 2024-03-26T19:15:20.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:20.915+0000 7fb72b200640 1 -- 172.21.15.97:0/648074952 wait complete. 2024-03-26T19:15:20.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T18:51:11.459500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T18:56:22.069048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T18:56:23.116030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T19:15:20.936 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T19:15:20.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:20 smithi105 ceph-mon[36270]: pgmap v883: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-03-26T19:15:21.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.162+0000 7f38db200640 1 -- 172.21.15.97:0/3030376903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38dc103370 msgr2=0x7f38dc109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:21.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.162+0000 7f38db200640 1 --2- 172.21.15.97:0/3030376903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38dc103370 0x7f38dc109c00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009ca0 tx=0x7f38d402c100 comp rx=0 tx=0).stop 2024-03-26T19:15:21.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.162+0000 7f38db200640 1 -- 172.21.15.97:0/3030376903 shutdown_connections 2024-03-26T19:15:21.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.162+0000 7f38db200640 1 --2- 172.21.15.97:0/3030376903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38dc103370 0x7f38dc109c00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.162+0000 7f38db200640 1 --2- 172.21.15.97:0/3030376903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1029b0 0x7f38dc102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.162+0000 7f38db200640 1 --2- 172.21.15.97:0/3030376903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38dc1017c0 0x7f38dc101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.162+0000 7f38db200640 1 -- 172.21.15.97:0/3030376903 >> 172.21.15.97:0/3030376903 conn(0x7f38dc0fcd30 msgr2=0x7f38dc0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:21.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.163+0000 7f38db200640 1 -- 172.21.15.97:0/3030376903 shutdown_connections 2024-03-26T19:15:21.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.163+0000 7f38db200640 1 -- 172.21.15.97:0/3030376903 wait complete. 2024-03-26T19:15:21.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38db200640 1 Processor -- start 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38db200640 1 -- start start 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38db200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38dc1017c0 0x7f38dc075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38db200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1029b0 0x7f38dc073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38db200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38dc103370 0x7f38dc0741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38db200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc10d420 con 0x7f38dc1017c0 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38db200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc10d2a0 con 0x7f38dc103370 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38db200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc10d5a0 con 0x7f38dc1029b0 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38da800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38dc103370 0x7f38dc0741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38d9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1029b0 0x7f38dc073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38d9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38dc1017c0 0x7f38dc075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38da800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38dc103370 0x7f38dc0741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38262/0 (socket says 172.21.15.97:38262) 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.164+0000 7f38da800640 1 -- 172.21.15.97:0/3720196900 learned_addr learned my addr 172.21.15.97:0/3720196900 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:21.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.165+0000 7f38d9400640 1 -- 172.21.15.97:0/3720196900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38dc103370 msgr2=0x7f38dc0741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.165+0000 7f38d9400640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38dc103370 0x7f38dc0741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.165+0000 7f38d9400640 1 -- 172.21.15.97:0/3720196900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38dc1017c0 msgr2=0x7f38dc075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.165+0000 7f38d9400640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38dc1017c0 0x7f38dc075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.165+0000 7f38d9400640 1 -- 172.21.15.97:0/3720196900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc074a80 con 0x7f38dc1029b0 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.165+0000 7f38da800640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38dc103370 0x7f38dc0741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.165+0000 7f38d9400640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1029b0 0x7f38dc073cb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f38c800e9f0 tx=0x7f38c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.165+0000 7f38d2a00640 1 -- 172.21.15.97:0/3720196900 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c800ccc0 con 0x7f38dc1029b0 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.166+0000 7f38d2a00640 1 -- 172.21.15.97:0/3720196900 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c8010910 con 0x7f38dc1029b0 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.166+0000 7f38db200640 1 -- 172.21.15.97:0/3720196900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc1a1b50 con 0x7f38dc1029b0 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.166+0000 7f38d2a00640 1 -- 172.21.15.97:0/3720196900 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8018960 con 0x7f38dc1029b0 2024-03-26T19:15:21.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.166+0000 7f38db200640 1 -- 172.21.15.97:0/3720196900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc1a20a0 con 0x7f38dc1029b0 2024-03-26T19:15:21.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.167+0000 7f38db200640 1 -- 172.21.15.97:0/3720196900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc108ae0 con 0x7f38dc1029b0 2024-03-26T19:15:21.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.169+0000 7f38d2a00640 1 -- 172.21.15.97:0/3720196900 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f38c8010ac0 con 0x7f38dc1029b0 2024-03-26T19:15:21.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.170+0000 7f38d2a00640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f38c00a2f20 0x7f38c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:21.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.171+0000 7f38d2a00640 1 -- 172.21.15.97:0/3720196900 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f38c80cf780 con 0x7f38dc1029b0 2024-03-26T19:15:21.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.171+0000 7f38d9e00640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f38c00a2f20 0x7f38c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:21.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.171+0000 7f38d9e00640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f38c00a2f20 0x7f38c00a53e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f38dc102810 tx=0x7f38cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:21.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.175+0000 7f38d2a00640 1 -- 172.21.15.97:0/3720196900 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f38c809bf30 con 0x7f38dc1029b0 2024-03-26T19:15:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.416+0000 7f38db200640 1 -- 172.21.15.97:0/3720196900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f38dc0624c0 con 0x7f38dc1029b0 2024-03-26T19:15:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.416+0000 7f38d2a00640 1 -- 172.21.15.97:0/3720196900 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f38c809ffd0 con 0x7f38dc1029b0 2024-03-26T19:15:21.417 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-26T19:15:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 -- 172.21.15.97:0/3720196900 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f38c00a2f20 msgr2=0x7f38c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f38c00a2f20 0x7f38c00a53e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f38dc102810 tx=0x7f38cc005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 -- 172.21.15.97:0/3720196900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1029b0 msgr2=0x7f38dc073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1029b0 0x7f38dc073cb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f38c800e9f0 tx=0x7f38c800eeb0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 -- 172.21.15.97:0/3720196900 shutdown_connections 2024-03-26T19:15:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f38c00a2f20 0x7f38c00a53e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38dc103370 0x7f38dc0741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1029b0 0x7f38dc073cb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 --2- 172.21.15.97:0/3720196900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38dc1017c0 0x7f38dc075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 -- 172.21.15.97:0/3720196900 >> 172.21.15.97:0/3720196900 conn(0x7f38dc0fcd30 msgr2=0x7f38dc105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 -- 172.21.15.97:0/3720196900 shutdown_connections 2024-03-26T19:15:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.419+0000 7f38db200640 1 -- 172.21.15.97:0/3720196900 wait complete. 2024-03-26T19:15:21.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T19:15:21.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.655+0000 7f2b32c00640 1 -- 172.21.15.97:0/2136965699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b2c1029b0 msgr2=0x7f2b2c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:21.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.655+0000 7f2b32c00640 1 --2- 172.21.15.97:0/2136965699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b2c1029b0 0x7f2b2c102e30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009900 tx=0x7f2b1802bc30 comp rx=0 tx=0).stop 2024-03-26T19:15:21.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.655+0000 7f2b32c00640 1 -- 172.21.15.97:0/2136965699 shutdown_connections 2024-03-26T19:15:21.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.655+0000 7f2b32c00640 1 --2- 172.21.15.97:0/2136965699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b2c103370 0x7f2b2c109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.655+0000 7f2b32c00640 1 --2- 172.21.15.97:0/2136965699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b2c1029b0 0x7f2b2c102e30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.655+0000 7f2b32c00640 1 --2- 172.21.15.97:0/2136965699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c1017c0 0x7f2b2c101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.655+0000 7f2b32c00640 1 -- 172.21.15.97:0/2136965699 >> 172.21.15.97:0/2136965699 conn(0x7f2b2c0fcd30 msgr2=0x7f2b2c0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.655+0000 7f2b32c00640 1 -- 172.21.15.97:0/2136965699 shutdown_connections 2024-03-26T19:15:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.655+0000 7f2b32c00640 1 -- 172.21.15.97:0/2136965699 wait complete. 2024-03-26T19:15:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.656+0000 7f2b32c00640 1 Processor -- start 2024-03-26T19:15:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.656+0000 7f2b32c00640 1 -- start start 2024-03-26T19:15:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.656+0000 7f2b32c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c1017c0 0x7f2b2c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.656+0000 7f2b32c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b2c1029b0 0x7f2b2c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.656+0000 7f2b32c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b2c103370 0x7f2b2c1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.656+0000 7f2b32c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c10d420 con 0x7f2b2c1017c0 2024-03-26T19:15:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.656+0000 7f2b32c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c10d2a0 con 0x7f2b2c1029b0 2024-03-26T19:15:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.656+0000 7f2b32c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c10d5a0 con 0x7f2b2c103370 2024-03-26T19:15:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b30e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b2c1029b0 0x7f2b2c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b30e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b2c1029b0 0x7f2b2c19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38274/0 (socket says 172.21.15.97:38274) 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b31800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c1017c0 0x7f2b2c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b30e00640 1 -- 172.21.15.97:0/479730971 learned_addr learned my addr 172.21.15.97:0/479730971 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b32200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b2c103370 0x7f2b2c1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b30e00640 1 -- 172.21.15.97:0/479730971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b2c103370 msgr2=0x7f2b2c1a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b30e00640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b2c103370 0x7f2b2c1a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b30e00640 1 -- 172.21.15.97:0/479730971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c1017c0 msgr2=0x7f2b2c199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b30e00640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c1017c0 0x7f2b2c199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b30e00640 1 -- 172.21.15.97:0/479730971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c1a18e0 con 0x7f2b2c1029b0 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.657+0000 7f2b32200640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b2c103370 0x7f2b2c1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.658+0000 7f2b30e00640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b2c1029b0 0x7f2b2c19a280 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009230 tx=0x7f2b1802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.658+0000 7f2b2a000640 1 -- 172.21.15.97:0/479730971 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18041070 con 0x7f2b2c1029b0 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.658+0000 7f2b2a000640 1 -- 172.21.15.97:0/479730971 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b18004450 con 0x7f2b2c1029b0 2024-03-26T19:15:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.658+0000 7f2b2a000640 1 -- 172.21.15.97:0/479730971 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18003db0 con 0x7f2b2c1029b0 2024-03-26T19:15:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.658+0000 7f2b32c00640 1 -- 172.21.15.97:0/479730971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2c1a1b70 con 0x7f2b2c1029b0 2024-03-26T19:15:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.658+0000 7f2b32c00640 1 -- 172.21.15.97:0/479730971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2c1a2010 con 0x7f2b2c1029b0 2024-03-26T19:15:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.659+0000 7f2b32c00640 1 -- 172.21.15.97:0/479730971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2c1a21e0 con 0x7f2b2c1029b0 2024-03-26T19:15:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.661+0000 7f2b2a000640 1 -- 172.21.15.97:0/479730971 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f2b18044070 con 0x7f2b2c1029b0 2024-03-26T19:15:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.662+0000 7f2b2a000640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2b140a2fa0 0x7f2b140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.663+0000 7f2b2a000640 1 -- 172.21.15.97:0/479730971 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f2b180f0f50 con 0x7f2b2c1029b0 2024-03-26T19:15:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.663+0000 7f2b31800640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2b140a2fa0 0x7f2b140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.663+0000 7f2b31800640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2b140a2fa0 0x7f2b140a5460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0ff070 tx=0x7f2b20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.666+0000 7f2b2a000640 1 -- 172.21.15.97:0/479730971 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b180bdd50 con 0x7f2b2c1029b0 2024-03-26T19:15:21.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:21 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/648074952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T19:15:21.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:21 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/3720196900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T19:15:21.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.881+0000 7f2b32c00640 1 -- 172.21.15.97:0/479730971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2b2c1a2570 con 0x7f2b2c1029b0 2024-03-26T19:15:21.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.882+0000 7f2b2a000640 1 -- 172.21.15.97:0/479730971 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f2b180bd650 con 0x7f2b2c1029b0 2024-03-26T19:15:21.883 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-26T19:15:21.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:21 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/648074952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T19:15:21.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:21 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/3720196900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T19:15:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.884+0000 7f2b32c00640 1 -- 172.21.15.97:0/479730971 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2b140a2fa0 msgr2=0x7f2b140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.884+0000 7f2b32c00640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2b140a2fa0 0x7f2b140a5460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0ff070 tx=0x7f2b20005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.884+0000 7f2b32c00640 1 -- 172.21.15.97:0/479730971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b2c1029b0 msgr2=0x7f2b2c19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.884+0000 7f2b32c00640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b2c1029b0 0x7f2b2c19a280 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009230 tx=0x7f2b1802c650 comp rx=0 tx=0).stop 2024-03-26T19:15:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.885+0000 7f2b32c00640 1 -- 172.21.15.97:0/479730971 shutdown_connections 2024-03-26T19:15:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.885+0000 7f2b32c00640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2b140a2fa0 0x7f2b140a5460 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.885+0000 7f2b32c00640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b2c103370 0x7f2b2c1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.885+0000 7f2b32c00640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b2c1029b0 0x7f2b2c19a280 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.885+0000 7f2b32c00640 1 --2- 172.21.15.97:0/479730971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c1017c0 0x7f2b2c199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.885+0000 7f2b32c00640 1 -- 172.21.15.97:0/479730971 >> 172.21.15.97:0/479730971 conn(0x7f2b2c0fcd30 msgr2=0x7f2b2c073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.885+0000 7f2b32c00640 1 -- 172.21.15.97:0/479730971 shutdown_connections 2024-03-26T19:15:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:21.885+0000 7f2b32c00640 1 -- 172.21.15.97:0/479730971 wait complete. 2024-03-26T19:15:21.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T19:15:21.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:21 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/648074952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T19:15:21.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:21 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/3720196900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T19:15:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.120+0000 7f0fad400640 1 -- 172.21.15.97:0/3314819602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fa8103370 msgr2=0x7f0fa8109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.120+0000 7f0fad400640 1 --2- 172.21.15.97:0/3314819602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fa8103370 0x7f0fa8109c00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009ca0 tx=0x7f0fa002c0f0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.121+0000 7f0fad400640 1 -- 172.21.15.97:0/3314819602 shutdown_connections 2024-03-26T19:15:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.121+0000 7f0fad400640 1 --2- 172.21.15.97:0/3314819602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fa8103370 0x7f0fa8109c00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.121+0000 7f0fad400640 1 --2- 172.21.15.97:0/3314819602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa81029b0 0x7f0fa8102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.121+0000 7f0fad400640 1 --2- 172.21.15.97:0/3314819602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa81017c0 0x7f0fa8101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.121+0000 7f0fad400640 1 -- 172.21.15.97:0/3314819602 >> 172.21.15.97:0/3314819602 conn(0x7f0fa80fcd30 msgr2=0x7f0fa80ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.121+0000 7f0fad400640 1 -- 172.21.15.97:0/3314819602 shutdown_connections 2024-03-26T19:15:22.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.121+0000 7f0fad400640 1 -- 172.21.15.97:0/3314819602 wait complete. 2024-03-26T19:15:22.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.122+0000 7f0fad400640 1 Processor -- start 2024-03-26T19:15:22.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.122+0000 7f0fad400640 1 -- start start 2024-03-26T19:15:22.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0fad400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa81017c0 0x7f0fa8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:22.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0fad400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fa81029b0 0x7f0fa8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:22.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0fad400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa8103370 0x7f0fa819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:22.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0fad400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa810d440 con 0x7f0fa81017c0 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0fad400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa810d2c0 con 0x7f0fa81029b0 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0fad400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa810d5c0 con 0x7f0fa8103370 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0f9fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fa81029b0 0x7f0fa8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0fa7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa81017c0 0x7f0fa8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0faca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa8103370 0x7f0fa819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0f9fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fa81029b0 0x7f0fa8197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38302/0 (socket says 172.21.15.97:38302) 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0f9fe00640 1 -- 172.21.15.97:0/2406912227 learned_addr learned my addr 172.21.15.97:0/2406912227 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0f9fe00640 1 -- 172.21.15.97:0/2406912227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa8103370 msgr2=0x7f0fa819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0f9fe00640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa8103370 0x7f0fa819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0f9fe00640 1 -- 172.21.15.97:0/2406912227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa81017c0 msgr2=0x7f0fa8197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0f9fe00640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa81017c0 0x7f0fa8197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0f9fe00640 1 -- 172.21.15.97:0/2406912227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa819f640 con 0x7f0fa81029b0 2024-03-26T19:15:22.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.123+0000 7f0faca00640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa8103370 0x7f0fa819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:15:22.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.124+0000 7f0f9fe00640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fa81029b0 0x7f0fa8197f60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000e9f0 tx=0x7f0f9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:22.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.124+0000 7f0fa5600640 1 -- 172.21.15.97:0/2406912227 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000cd80 con 0x7f0fa81029b0 2024-03-26T19:15:22.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.124+0000 7f0fa5600640 1 -- 172.21.15.97:0/2406912227 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f90010910 con 0x7f0fa81029b0 2024-03-26T19:15:22.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.124+0000 7f0fad400640 1 -- 172.21.15.97:0/2406912227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa819f930 con 0x7f0fa81029b0 2024-03-26T19:15:22.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.124+0000 7f0fa5600640 1 -- 172.21.15.97:0/2406912227 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90018990 con 0x7f0fa81029b0 2024-03-26T19:15:22.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.124+0000 7f0fad400640 1 -- 172.21.15.97:0/2406912227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa819fed0 con 0x7f0fa81029b0 2024-03-26T19:15:22.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.125+0000 7f0fad400640 1 -- 172.21.15.97:0/2406912227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa8108ae0 con 0x7f0fa81029b0 2024-03-26T19:15:22.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.127+0000 7f0fa5600640 1 -- 172.21.15.97:0/2406912227 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f0f90010ac0 con 0x7f0fa81029b0 2024-03-26T19:15:22.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.129+0000 7f0fa5600640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f8c0a2fa0 0x7f0f8c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:22.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.130+0000 7f0fa5600640 1 -- 172.21.15.97:0/2406912227 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f0f900cf680 con 0x7f0fa81029b0 2024-03-26T19:15:22.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.130+0000 7f0fa7e00640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f8c0a2fa0 0x7f0f8c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:22.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.130+0000 7f0fa7e00640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f8c0a2fa0 0x7f0f8c0a5460 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8102810 tx=0x7f0f98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:22.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.132+0000 7f0fa5600640 1 -- 172.21.15.97:0/2406912227 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0f9009be30 con 0x7f0fa81029b0 2024-03-26T19:15:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.332+0000 7f0fad400640 1 -- 172.21.15.97:0/2406912227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0fa802d240 con 0x7f0fa81029b0 2024-03-26T19:15:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.333+0000 7f0fa5600640 1 -- 172.21.15.97:0/2406912227 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f0f9009fed0 con 0x7f0fa81029b0 2024-03-26T19:15:22.334 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 -- 172.21.15.97:0/2406912227 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f8c0a2fa0 msgr2=0x7f0f8c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f8c0a2fa0 0x7f0f8c0a5460 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8102810 tx=0x7f0f98005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 -- 172.21.15.97:0/2406912227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fa81029b0 msgr2=0x7f0fa8197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fa81029b0 0x7f0fa8197f60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000e9f0 tx=0x7f0f9000eeb0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 -- 172.21.15.97:0/2406912227 shutdown_connections 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f0f8c0a2fa0 0x7f0f8c0a5460 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa8103370 0x7f0fa819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fa81029b0 0x7f0fa8197f60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 --2- 172.21.15.97:0/2406912227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa81017c0 0x7f0fa8197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 -- 172.21.15.97:0/2406912227 >> 172.21.15.97:0/2406912227 conn(0x7f0fa80fcd30 msgr2=0x7f0fa8105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.337+0000 7f0fad400640 1 -- 172.21.15.97:0/2406912227 shutdown_connections 2024-03-26T19:15:22.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.338+0000 7f0fad400640 1 -- 172.21.15.97:0/2406912227 wait complete. 2024-03-26T19:15:22.358 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T19:15:22.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.576+0000 7f7f9ba00640 1 -- 172.21.15.97:0/3141376565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f94104c80 msgr2=0x7f7f94107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:22.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.576+0000 7f7f9ba00640 1 --2- 172.21.15.97:0/3141376565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f94104c80 0x7f7f94107070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009860 tx=0x7f7f7c02bb70 comp rx=0 tx=0).stop 2024-03-26T19:15:22.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.576+0000 7f7f9ba00640 1 -- 172.21.15.97:0/3141376565 shutdown_connections 2024-03-26T19:15:22.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.576+0000 7f7f9ba00640 1 --2- 172.21.15.97:0/3141376565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f941075b0 0x7f7f94109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.576+0000 7f7f9ba00640 1 --2- 172.21.15.97:0/3141376565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f94104c80 0x7f7f94107070 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.576+0000 7f7f9ba00640 1 --2- 172.21.15.97:0/3141376565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f94102350 0x7f7f94104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.576+0000 7f7f9ba00640 1 -- 172.21.15.97:0/3141376565 >> 172.21.15.97:0/3141376565 conn(0x7f7f940fbd60 msgr2=0x7f7f940fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:22.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.576+0000 7f7f9ba00640 1 -- 172.21.15.97:0/3141376565 shutdown_connections 2024-03-26T19:15:22.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.576+0000 7f7f9ba00640 1 -- 172.21.15.97:0/3141376565 wait complete. 2024-03-26T19:15:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.577+0000 7f7f9ba00640 1 Processor -- start 2024-03-26T19:15:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.577+0000 7f7f9ba00640 1 -- start start 2024-03-26T19:15:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f9ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f94102350 0x7f7f94199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f9ba00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f94104c80 0x7f7f9419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f9ba00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f941075b0 0x7f7f941a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f9ba00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9410d540 con 0x7f7f94102350 2024-03-26T19:15:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f9ba00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9410d3c0 con 0x7f7f941075b0 2024-03-26T19:15:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f9ba00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9410d6c0 con 0x7f7f94104c80 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f99000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f94102350 0x7f7f94199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f8be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f94104c80 0x7f7f9419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f8be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f94104c80 0x7f7f9419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:33524/0 (socket says 172.21.15.97:33524) 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f8be00640 1 -- 172.21.15.97:0/2097444108 learned_addr learned my addr 172.21.15.97:0/2097444108 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f99a00640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f941075b0 0x7f7f941a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f8be00640 1 -- 172.21.15.97:0/2097444108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f941075b0 msgr2=0x7f7f941a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f8be00640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f941075b0 0x7f7f941a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f8be00640 1 -- 172.21.15.97:0/2097444108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f94102350 msgr2=0x7f7f94199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f8be00640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f94102350 0x7f7f94199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.578+0000 7f7f8be00640 1 -- 172.21.15.97:0/2097444108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f941a1a50 con 0x7f7f94104c80 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.579+0000 7f7f8be00640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f94104c80 0x7f7f9419a370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0349e0 tx=0x7f7f7c032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.579+0000 7f7f92000640 1 -- 172.21.15.97:0/2097444108 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c041020 con 0x7f7f94104c80 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.579+0000 7f7f92000640 1 -- 172.21.15.97:0/2097444108 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c02c9f0 con 0x7f7f94104c80 2024-03-26T19:15:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.579+0000 7f7f92000640 1 -- 172.21.15.97:0/2097444108 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c039040 con 0x7f7f94104c80 2024-03-26T19:15:22.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.579+0000 7f7f9ba00640 1 -- 172.21.15.97:0/2097444108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f941a1ce0 con 0x7f7f94104c80 2024-03-26T19:15:22.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.579+0000 7f7f9ba00640 1 -- 172.21.15.97:0/2097444108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f941a2180 con 0x7f7f94104c80 2024-03-26T19:15:22.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.580+0000 7f7f9ba00640 1 -- 172.21.15.97:0/2097444108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f94109100 con 0x7f7f94104c80 2024-03-26T19:15:22.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.583+0000 7f7f92000640 1 -- 172.21.15.97:0/2097444108 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f7f7c044020 con 0x7f7f94104c80 2024-03-26T19:15:22.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.584+0000 7f7f92000640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7f780a2f20 0x7f7f780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:22.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.585+0000 7f7f92000640 1 -- 172.21.15.97:0/2097444108 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f7f7c0f0c80 con 0x7f7f94104c80 2024-03-26T19:15:22.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.585+0000 7f7f99000640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7f780a2f20 0x7f7f780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:22.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.585+0000 7f7f99000640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7f780a2f20 0x7f7f780a53e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009bd0 tx=0x7f7f84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:22.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.588+0000 7f7f92000640 1 -- 172.21.15.97:0/2097444108 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7f7c04b020 con 0x7f7f94104c80 2024-03-26T19:15:22.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:22 smithi097 ceph-mon[31955]: pgmap v884: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-03-26T19:15:22.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:22 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/479730971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T19:15:22.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:22 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2406912227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T19:15:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.799+0000 7f7f9ba00640 1 -- 172.21.15.97:0/2097444108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7f940624c0 con 0x7f7f94104c80 2024-03-26T19:15:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.800+0000 7f7f92000640 1 -- 172.21.15.97:0/2097444108 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+589 (secure 0 0 0) 0x7f7f7c0bda80 con 0x7f7f94104c80 2024-03-26T19:15:22.802 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/668477340 2024-03-27T18:48:28.768371+0000 2024-03-26T19:15:22.802 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2962857505 2024-03-27T18:48:28.768371+0000 2024-03-26T19:15:22.802 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/350003366 2024-03-27T18:47:55.672991+0000 2024-03-26T19:15:22.802 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/3462603610 2024-03-27T18:48:28.768371+0000 2024-03-26T19:15:22.802 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3308112091 2024-03-27T18:47:55.672991+0000 2024-03-26T19:15:22.802 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/3462603610 2024-03-27T18:48:28.768371+0000 2024-03-26T19:15:22.802 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/1051395080 2024-03-27T18:48:28.768371+0000 2024-03-26T19:15:22.802 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3851585874 2024-03-27T18:47:55.672991+0000 2024-03-26T19:15:22.802 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/350003366 2024-03-27T18:47:55.672991+0000 2024-03-26T19:15:22.802 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2299096028 2024-03-27T18:47:55.672991+0000 2024-03-26T19:15:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:listed 10 entries 2024-03-26T19:15:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.803+0000 7f7f9ba00640 1 -- 172.21.15.97:0/2097444108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7f780a2f20 msgr2=0x7f7f780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.803+0000 7f7f9ba00640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7f780a2f20 0x7f7f780a53e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009bd0 tx=0x7f7f84005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.804+0000 7f7f9ba00640 1 -- 172.21.15.97:0/2097444108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f94104c80 msgr2=0x7f7f9419a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.804+0000 7f7f9ba00640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f94104c80 0x7f7f9419a370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0349e0 tx=0x7f7f7c032430 comp rx=0 tx=0).stop 2024-03-26T19:15:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.804+0000 7f7f9ba00640 1 -- 172.21.15.97:0/2097444108 shutdown_connections 2024-03-26T19:15:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.804+0000 7f7f9ba00640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f7f780a2f20 0x7f7f780a53e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.804+0000 7f7f9ba00640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f941075b0 0x7f7f941a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.804+0000 7f7f9ba00640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f94104c80 0x7f7f9419a370 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.804+0000 7f7f9ba00640 1 --2- 172.21.15.97:0/2097444108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f94102350 0x7f7f94199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.804+0000 7f7f9ba00640 1 -- 172.21.15.97:0/2097444108 >> 172.21.15.97:0/2097444108 conn(0x7f7f940fbd60 msgr2=0x7f7f94101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.804+0000 7f7f9ba00640 1 -- 172.21.15.97:0/2097444108 shutdown_connections 2024-03-26T19:15:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:22.804+0000 7f7f9ba00640 1 -- 172.21.15.97:0/2097444108 wait complete. 2024-03-26T19:15:22.823 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-26T19:15:22.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T19:15:22.823 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /tmp/cephfs 2024-03-26T19:15:22.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:22 smithi144 ceph-mon[36053]: pgmap v884: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-03-26T19:15:22.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:22 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/479730971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T19:15:22.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:22 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2406912227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T19:15:22.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:22 smithi105 ceph-mon[36270]: pgmap v884: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-03-26T19:15:22.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:22 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/479730971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T19:15:22.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:22 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2406912227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T19:15:23.221 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-03-26T19:15:23.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T19:15:23.221 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-03-26T19:15:23.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T19:15:23.309 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-26T19:15:23.381 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-03-26T19:15:23.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T19:15:23.382 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-03-26T19:15:23.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T19:15:23.473 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-03-26T19:15:23.579 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-26T19:15:23.579 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-26T19:15:23.579 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T19:15:23.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053428,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/53ebef7fffc73d7a5234312a423c4ff21675115809a890202923ee9119056085/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LYYYRH7CAA7LURD6I4JGENBTFT:/var/lib/containers/storage/overlay/l/A26TKUG7LF3IPUPF7XKMDHRUCV,upperdir=/var/lib/containers/storage/overlay/53ebef7fffc73d7a5234312a423c4ff21675115809a890202923ee9119056085/diff,workdir=/var/lib/containers/storage/overlay/53ebef7fffc73d7a5234312a423c4ff21675115809a890202923ee9119056085/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ebe47a4e62dd33a062794becc44c5e5e3b0e90469a5a32bcb50e494d9825a378/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LYYYRH7CAA7LURD6I4JGENBTFT:/var/lib/containers/storage/overlay/l/A26TKUG7LF3IPUPF7XKMDHRUCV,upperdir=/var/lib/containers/storage/overlay/ebe47a4e62dd33a062794becc44c5e5e3b0e90469a5a32bcb50e494d9825a378/diff,workdir=/var/lib/containers/storage/overlay/ebe47a4e62dd33a062794becc44c5e5e3b0e90469a5a32bcb50e494d9825a378/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/fe00f7f4caa381f5dc969b1becaba4a0c8551022ef133dd9f9f662a51b8adcf0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LYYYRH7CAA7LURD6I4JGENBTFT:/var/lib/containers/storage/overlay/l/A26TKUG7LF3IPUPF7XKMDHRUCV,upperdir=/var/lib/containers/storage/overlay/fe00f7f4caa381f5dc969b1becaba4a0c8551022ef133dd9f9f662a51b8adcf0/diff,workdir=/var/lib/containers/storage/overlay/fe00f7f4caa381f5dc969b1becaba4a0c8551022ef133dd9f9f662a51b8adcf0/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d76fff54985537feddb7b0a4c1950f488d6e3a5b623dd5e6cff74023e6f53049/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LYYYRH7CAA7LURD6I4JGENBTFT:/var/lib/containers/storage/overlay/l/A26TKUG7LF3IPUPF7XKMDHRUCV,upperdir=/var/lib/containers/storage/overlay/d76fff54985537feddb7b0a4c1950f488d6e3a5b623dd5e6cff74023e6f53049/diff,workdir=/var/lib/containers/storage/overlay/d76fff54985537feddb7b0a4c1950f488d6e3a5b623dd5e6cff74023e6f53049/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c574e68b388806c4f3f349f682ee91b10f1b50472197369feacc0379e98dd231/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LYYYRH7CAA7LURD6I4JGENBTFT:/var/lib/containers/storage/overlay/l/A26TKUG7LF3IPUPF7XKMDHRUCV,upperdir=/var/lib/containers/storage/overlay/c574e68b388806c4f3f349f682ee91b10f1b50472197369feacc0379e98dd231/diff,workdir=/var/lib/containers/storage/overlay/c574e68b388806c4f3f349f682ee91b10f1b50472197369feacc0379e98dd231/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/283b90d802fa218d36c2c82a1ffa2387741fd124f6b34447e1524ac08b30735f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LYYYRH7CAA7LURD6I4JGENBTFT:/var/lib/containers/storage/overlay/l/A26TKUG7LF3IPUPF7XKMDHRUCV,upperdir=/var/lib/containers/storage/overlay/283b90d802fa218d36c2c82a1ffa2387741fd124f6b34447e1524ac08b30735f/diff,workdir=/var/lib/containers/storage/overlay/283b90d802fa218d36c2c82a1ffa2387741fd124f6b34447e1524ac08b30735f/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e95c4b9331438067ce15094ac211d3ffa61f30db3a47235d2e45587bb1d3f3d1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LYYYRH7CAA7LURD6I4JGENBTFT:/var/lib/containers/storage/overlay/l/A26TKUG7LF3IPUPF7XKMDHRUCV,upperdir=/var/lib/containers/storage/overlay/e95c4b9331438067ce15094ac211d3ffa61f30db3a47235d2e45587bb1d3f3d1/diff,workdir=/var/lib/containers/storage/overlay/e95c4b9331438067ce15094ac211d3ffa61f30db3a47235d2e45587bb1d3f3d1/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/61a6b2497d1d57b228690c30c0d179ca721805b1faff53bf6e304d91dfb878fd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LYYYRH7CAA7LURD6I4JGENBTFT:/var/lib/containers/storage/overlay/l/A26TKUG7LF3IPUPF7XKMDHRUCV,upperdir=/var/lib/containers/storage/overlay/61a6b2497d1d57b228690c30c0d179ca721805b1faff53bf6e304d91dfb878fd/diff,workdir=/var/lib/containers/storage/overlay/61a6b2497d1d57b228690c30c0d179ca721805b1faff53bf6e304d91dfb878fd/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/414d61253a3fc13d5868b8308bbbffc666a838d7402d59e3d50216de4ce07030/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LYYYRH7CAA7LURD6I4JGENBTFT:/var/lib/containers/storage/overlay/l/A26TKUG7LF3IPUPF7XKMDHRUCV,upperdir=/var/lib/containers/storage/overlay/414d61253a3fc13d5868b8308bbbffc666a838d7402d59e3d50216de4ce07030/diff,workdir=/var/lib/containers/storage/overlay/414d61253a3fc13d5868b8308bbbffc666a838d7402d59e3d50216de4ce07030/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ccface878d62f90dc9fd76dcfc35df619ea6f0efea88531a4a9c1afab52b3092/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LYYYRH7CAA7LURD6I4JGENBTFT:/var/lib/containers/storage/overlay/l/A26TKUG7LF3IPUPF7XKMDHRUCV,upperdir=/var/lib/containers/storage/overlay/ccface878d62f90dc9fd76dcfc35df619ea6f0efea88531a4a9c1afab52b3092/diff,workdir=/var/lib/containers/storage/overlay/ccface878d62f90dc9fd76dcfc35df619ea6f0efea88531a4a9c1afab52b3092/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 2024-03-26T19:15:23.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T19:15:23.626 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-26T19:15:23.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T19:15:23.708 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-03-26T19:15:23.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:23 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2097444108' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T19:15:23.831 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-03-26T19:15:23.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T19:15:23.834 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-26T19:15:23.844 INFO:tasks.cephadm:Teardown begin 2024-03-26T19:15:23.845 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-26T19:15:23.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T19:15:23.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:23 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2097444108' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T19:15:23.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:23 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2097444108' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T19:15:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.038+0000 7f2c8e200640 1 -- 172.21.15.97:0/1821212213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c881056b0 msgr2=0x7f2c8810bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.038+0000 7f2c8e200640 1 --2- 172.21.15.97:0/1821212213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c881056b0 0x7f2c8810bf40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009ca0 tx=0x7f2c8002c0f0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.039+0000 7f2c8e200640 1 -- 172.21.15.97:0/1821212213 shutdown_connections 2024-03-26T19:15:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.039+0000 7f2c8e200640 1 --2- 172.21.15.97:0/1821212213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c881056b0 0x7f2c8810bf40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.039+0000 7f2c8e200640 1 --2- 172.21.15.97:0/1821212213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c88104cf0 0x7f2c88105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.039+0000 7f2c8e200640 1 --2- 172.21.15.97:0/1821212213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88103b00 0x7f2c88103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.039+0000 7f2c8e200640 1 -- 172.21.15.97:0/1821212213 >> 172.21.15.97:0/1821212213 conn(0x7f2c880ff0b0 msgr2=0x7f2c881014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.039+0000 7f2c8e200640 1 -- 172.21.15.97:0/1821212213 shutdown_connections 2024-03-26T19:15:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.039+0000 7f2c8e200640 1 -- 172.21.15.97:0/1821212213 wait complete. 2024-03-26T19:15:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.039+0000 7f2c8e200640 1 Processor -- start 2024-03-26T19:15:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c8e200640 1 -- start start 2024-03-26T19:15:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c8e200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c88103b00 0x7f2c88199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c8e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104cf0 0x7f2c8819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c8e200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c881056b0 0x7f2c881a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c8e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8810f760 con 0x7f2c88104cf0 2024-03-26T19:15:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c8e200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8810f5e0 con 0x7f2c88103b00 2024-03-26T19:15:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c8e200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8810f8e0 con 0x7f2c881056b0 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c87400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c88103b00 0x7f2c88199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c87400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c88103b00 0x7f2c88199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38332/0 (socket says 172.21.15.97:38332) 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c87400640 1 -- 172.21.15.97:0/1026891981 learned_addr learned my addr 172.21.15.97:0/1026891981 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c87e00640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c881056b0 0x7f2c881a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c86a00640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104cf0 0x7f2c8819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c87400640 1 -- 172.21.15.97:0/1026891981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c881056b0 msgr2=0x7f2c881a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.040+0000 7f2c87400640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c881056b0 0x7f2c881a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.041+0000 7f2c87400640 1 -- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104cf0 msgr2=0x7f2c8819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.041+0000 7f2c87400640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104cf0 0x7f2c8819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.041+0000 7f2c87400640 1 -- 172.21.15.97:0/1026891981 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c881a19d0 con 0x7f2c88103b00 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.041+0000 7f2c86a00640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104cf0 0x7f2c8819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:15:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.041+0000 7f2c87400640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c88103b00 0x7f2c88199db0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800e9c0 tx=0x7f2c7800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.041+0000 7f2c7fe00640 1 -- 172.21.15.97:0/1026891981 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7800cd80 con 0x7f2c88103b00 2024-03-26T19:15:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.041+0000 7f2c7fe00640 1 -- 172.21.15.97:0/1026891981 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c780035f0 con 0x7f2c88103b00 2024-03-26T19:15:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.041+0000 7f2c7fe00640 1 -- 172.21.15.97:0/1026891981 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78010690 con 0x7f2c88103b00 2024-03-26T19:15:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.041+0000 7f2c8e200640 1 -- 172.21.15.97:0/1026891981 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c881a1cc0 con 0x7f2c88103b00 2024-03-26T19:15:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.041+0000 7f2c8e200640 1 -- 172.21.15.97:0/1026891981 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c881a21b0 con 0x7f2c88103b00 2024-03-26T19:15:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.043+0000 7f2c8e200640 1 -- 172.21.15.97:0/1026891981 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8810ae20 con 0x7f2c88103b00 2024-03-26T19:15:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.044+0000 7f2c7fe00640 1 -- 172.21.15.97:0/1026891981 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f2c780037a0 con 0x7f2c88103b00 2024-03-26T19:15:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.045+0000 7f2c7fe00640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2c6c0a2fa0 0x7f2c6c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.045+0000 7f2c7fe00640 1 -- 172.21.15.97:0/1026891981 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f2c780cf7d0 con 0x7f2c88103b00 2024-03-26T19:15:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.045+0000 7f2c86a00640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2c6c0a2fa0 0x7f2c6c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:24.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.046+0000 7f2c86a00640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2c6c0a2fa0 0x7f2c6c0a5460 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2c8819b2c0 tx=0x7f2c74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:24.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.049+0000 7f2c7fe00640 1 -- 172.21.15.97:0/1026891981 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2c7809c630 con 0x7f2c88103b00 2024-03-26T19:15:24.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.314+0000 7f2c8e200640 1 -- 172.21.15.97:0/1026891981 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c8802d240 con 0x7f2c88103b00 2024-03-26T19:15:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.315+0000 7f2c7fe00640 1 -- 172.21.15.97:0/1026891981 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f2c8802d240 con 0x7f2c88103b00 2024-03-26T19:15:24.317 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:15:24.317 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T18:56:23.642405+0000","modified":"2024-03-26T19:13:28.618565+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24535,"mds_3":24494,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2391073232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2391073232},{"type":"v1","addr":"172.21.15.105:6839","nonce":2391073232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26333":{"gid":26333,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/4179427850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4179427850},{"type":"v1","addr":"172.21.15.105:6837","nonce":4179427850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24481,24535,24581,24569]},"id":1}]} 2024-03-26T19:15:24.317 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 29 2024-03-26T19:15:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.317+0000 7f2c8e200640 1 -- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2c6c0a2fa0 msgr2=0x7f2c6c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.317+0000 7f2c8e200640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2c6c0a2fa0 0x7f2c6c0a5460 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2c8819b2c0 tx=0x7f2c74005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.317+0000 7f2c8e200640 1 -- 172.21.15.97:0/1026891981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c88103b00 msgr2=0x7f2c88199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.317+0000 7f2c8e200640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c88103b00 0x7f2c88199db0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800e9c0 tx=0x7f2c7800ee80 comp rx=0 tx=0).stop 2024-03-26T19:15:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.317+0000 7f2c8e200640 1 -- 172.21.15.97:0/1026891981 shutdown_connections 2024-03-26T19:15:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.317+0000 7f2c8e200640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f2c6c0a2fa0 0x7f2c6c0a5460 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.317+0000 7f2c8e200640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c881056b0 0x7f2c881a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.317+0000 7f2c8e200640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104cf0 0x7f2c8819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.317+0000 7f2c8e200640 1 --2- 172.21.15.97:0/1026891981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c88103b00 0x7f2c88199db0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.317+0000 7f2c8e200640 1 -- 172.21.15.97:0/1026891981 >> 172.21.15.97:0/1026891981 conn(0x7f2c880ff0b0 msgr2=0x7f2c88107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:24.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.318+0000 7f2c8e200640 1 -- 172.21.15.97:0/1026891981 shutdown_connections 2024-03-26T19:15:24.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.318+0000 7f2c8e200640 1 -- 172.21.15.97:0/1026891981 wait complete. 2024-03-26T19:15:24.336 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T19:15:24.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.517+0000 7fc8f5000640 1 -- 172.21.15.97:0/329811646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8f0101900 msgr2=0x7fc8f0101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:24.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.517+0000 7fc8f5000640 1 --2- 172.21.15.97:0/329811646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8f0101900 0x7fc8f0101d20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009900 tx=0x7fc8e002bc40 comp rx=0 tx=0).stop 2024-03-26T19:15:24.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.517+0000 7fc8f5000640 1 -- 172.21.15.97:0/329811646 shutdown_connections 2024-03-26T19:15:24.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.517+0000 7fc8f5000640 1 --2- 172.21.15.97:0/329811646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f01034b0 0x7fc8f0109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.517+0000 7fc8f5000640 1 --2- 172.21.15.97:0/329811646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f0102af0 0x7fc8f0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.517+0000 7fc8f5000640 1 --2- 172.21.15.97:0/329811646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8f0101900 0x7fc8f0101d20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.517+0000 7fc8f5000640 1 -- 172.21.15.97:0/329811646 >> 172.21.15.97:0/329811646 conn(0x7fc8f00fce70 msgr2=0x7fc8f00ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:24.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.517+0000 7fc8f5000640 1 -- 172.21.15.97:0/329811646 shutdown_connections 2024-03-26T19:15:24.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.517+0000 7fc8f5000640 1 -- 172.21.15.97:0/329811646 wait complete. 2024-03-26T19:15:24.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.518+0000 7fc8f5000640 1 Processor -- start 2024-03-26T19:15:24.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8f5000640 1 -- start start 2024-03-26T19:15:24.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8f5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8f0101900 0x7fc8f0197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:24.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8f5000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f0102af0 0x7fc8f0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:24.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8f5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f01034b0 0x7fc8f019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:24.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8f5000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f010d550 con 0x7fc8f01034b0 2024-03-26T19:15:24.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8f5000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f010d3d0 con 0x7fc8f0102af0 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8f5000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f010d6d0 con 0x7fc8f0101900 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ede00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8f0101900 0x7fc8f0197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ed400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f0102af0 0x7fc8f0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ede00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8f0101900 0x7fc8f0197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:33580/0 (socket says 172.21.15.97:33580) 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ede00640 1 -- 172.21.15.97:0/2757151944 learned_addr learned my addr 172.21.15.97:0/2757151944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ee800640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f01034b0 0x7fc8f019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ede00640 1 -- 172.21.15.97:0/2757151944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f0102af0 msgr2=0x7fc8f0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ede00640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f0102af0 0x7fc8f0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ede00640 1 -- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f01034b0 msgr2=0x7fc8f019f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ede00640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f01034b0 0x7fc8f019f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ede00640 1 -- 172.21.15.97:0/2757151944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f019f890 con 0x7fc8f0101900 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.519+0000 7fc8ee800640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f01034b0 0x7fc8f019f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.520+0000 7fc8ede00640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8f0101900 0x7fc8f0197cd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000c00 tx=0x7fc8e002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.520+0000 7fc8e6a00640 1 -- 172.21.15.97:0/2757151944 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0041070 con 0x7fc8f0101900 2024-03-26T19:15:24.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.520+0000 7fc8e6a00640 1 -- 172.21.15.97:0/2757151944 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e0004450 con 0x7fc8f0101900 2024-03-26T19:15:24.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.520+0000 7fc8f5000640 1 -- 172.21.15.97:0/2757151944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f019fb20 con 0x7fc8f0101900 2024-03-26T19:15:24.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.520+0000 7fc8e6a00640 1 -- 172.21.15.97:0/2757151944 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0034440 con 0x7fc8f0101900 2024-03-26T19:15:24.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.520+0000 7fc8f5000640 1 -- 172.21.15.97:0/2757151944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f01a0060 con 0x7fc8f0101900 2024-03-26T19:15:24.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.521+0000 7fc8f5000640 1 -- 172.21.15.97:0/2757151944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f0108d90 con 0x7fc8f0101900 2024-03-26T19:15:24.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.523+0000 7fc8e6a00640 1 -- 172.21.15.97:0/2757151944 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fc8e0044070 con 0x7fc8f0101900 2024-03-26T19:15:24.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.524+0000 7fc8e6a00640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc8d40a2fa0 0x7fc8d40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:24.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.524+0000 7fc8e6a00640 1 -- 172.21.15.97:0/2757151944 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc8e00f0fb0 con 0x7fc8f0101900 2024-03-26T19:15:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.524+0000 7fc8ed400640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc8d40a2fa0 0x7fc8d40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.525+0000 7fc8ed400640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc8d40a2fa0 0x7fc8d40a5460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009bd0 tx=0x7fc8dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.528+0000 7fc8e6a00640 1 -- 172.21.15.97:0/2757151944 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc8e00bddb0 con 0x7fc8f0101900 2024-03-26T19:15:24.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:24 smithi097 ceph-mon[31955]: pgmap v885: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-03-26T19:15:24.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:24 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1026891981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T19:15:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.807+0000 7fc8f5000640 1 -- 172.21.15.97:0/2757151944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8f002d240 con 0x7fc8f0101900 2024-03-26T19:15:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.808+0000 7fc8e6a00640 1 -- 172.21.15.97:0/2757151944 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fc8e00bd6b0 con 0x7fc8f0101900 2024-03-26T19:15:24.809 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:15:24.810 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1485859624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1485859624},{"type":"v1","addr":"172.21.15.97:6835","nonce":1485859624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/940320803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":940320803},{"type":"v1","addr":"172.21.15.97:6839","nonce":940320803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T18:56:23.642405+0000","modified":"2024-03-26T19:13:28.618565+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24535,"mds_3":24494,"mds_4":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3099308706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3099308706},{"type":"v1","addr":"172.21.15.144:6833","nonce":3099308706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/254387341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":254387341},{"type":"v1","addr":"172.21.15.105:6833","nonce":254387341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.97:6837/4124660285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":4124660285},{"type":"v1","addr":"172.21.15.97:6837","nonce":4124660285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6835/1538692215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1538692215},{"type":"v1","addr":"172.21.15.105:6835","nonce":1538692215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/411039946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":411039946},{"type":"v1","addr":"172.21.15.144:6835","nonce":411039946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1138636767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1138636767},{"type":"v1","addr":"172.21.15.144:6837","nonce":1138636767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.144:6839/994192292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":994192292},{"type":"v1","addr":"172.21.15.144:6839","nonce":994192292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6841/4165060179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":4165060179},{"type":"v1","addr":"172.21.15.97:6841","nonce":4165060179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2391073232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2391073232},{"type":"v1","addr":"172.21.15.105:6839","nonce":2391073232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26333":{"gid":26333,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/4179427850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4179427850},{"type":"v1","addr":"172.21.15.105:6837","nonce":4179427850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24481,24535,24581,24569]},"id":1}]} 2024-03-26T19:15:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 29 2024-03-26T19:15:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 -- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc8d40a2fa0 msgr2=0x7fc8d40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc8d40a2fa0 0x7fc8d40a5460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009bd0 tx=0x7fc8dc005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 -- 172.21.15.97:0/2757151944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8f0101900 msgr2=0x7fc8f0197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8f0101900 0x7fc8f0197cd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000c00 tx=0x7fc8e002c650 comp rx=0 tx=0).stop 2024-03-26T19:15:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 -- 172.21.15.97:0/2757151944 shutdown_connections 2024-03-26T19:15:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc8d40a2fa0 0x7fc8d40a5460 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f01034b0 0x7fc8f019f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f0102af0 0x7fc8f0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 --2- 172.21.15.97:0/2757151944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8f0101900 0x7fc8f0197cd0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:24.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 -- 172.21.15.97:0/2757151944 >> 172.21.15.97:0/2757151944 conn(0x7fc8f00fce70 msgr2=0x7fc8f0105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:24.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.812+0000 7fc8f5000640 1 -- 172.21.15.97:0/2757151944 shutdown_connections 2024-03-26T19:15:24.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:24.813+0000 7fc8f5000640 1 -- 172.21.15.97:0/2757151944 wait complete. 2024-03-26T19:15:24.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T19:15:24.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:24 smithi144 ceph-mon[36053]: pgmap v885: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-03-26T19:15:24.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:24 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1026891981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T19:15:24.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:24 smithi105 ceph-mon[36270]: pgmap v885: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-03-26T19:15:24.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:24 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1026891981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T19:15:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.040+0000 7fecf5a00640 1 -- 172.21.15.97:0/4096255187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf0101730 msgr2=0x7fecf0101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.040+0000 7fecf5a00640 1 --2- 172.21.15.97:0/4096255187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf0101730 0x7fecf0101b50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fece0009900 tx=0x7fece002bc30 comp rx=0 tx=0).stop 2024-03-26T19:15:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.040+0000 7fecf5a00640 1 -- 172.21.15.97:0/4096255187 shutdown_connections 2024-03-26T19:15:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.040+0000 7fecf5a00640 1 --2- 172.21.15.97:0/4096255187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecf01032e0 0x7fecf0109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.040+0000 7fecf5a00640 1 --2- 172.21.15.97:0/4096255187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf0102920 0x7fecf0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.040+0000 7fecf5a00640 1 --2- 172.21.15.97:0/4096255187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf0101730 0x7fecf0101b50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.040+0000 7fecf5a00640 1 -- 172.21.15.97:0/4096255187 >> 172.21.15.97:0/4096255187 conn(0x7fecf00fcca0 msgr2=0x7fecf00ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.040+0000 7fecf5a00640 1 -- 172.21.15.97:0/4096255187 shutdown_connections 2024-03-26T19:15:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.041+0000 7fecf5a00640 1 -- 172.21.15.97:0/4096255187 wait complete. 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.041+0000 7fecf5a00640 1 Processor -- start 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecf5a00640 1 -- start start 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecf5a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf0101730 0x7fecf0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecf5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecf0102920 0x7fecf0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecf5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf01032e0 0x7fecf019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecf5a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fecf010d3a0 con 0x7fecf0102920 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecf5a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecf010d220 con 0x7fecf0101730 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecf5a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fecf010d520 con 0x7fecf01032e0 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecefe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf0101730 0x7fecf0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecefe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf0101730 0x7fecf0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38354/0 (socket says 172.21.15.97:38354) 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecefe00640 1 -- 172.21.15.97:0/2216484987 learned_addr learned my addr 172.21.15.97:0/2216484987 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecef400640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecf0102920 0x7fecf0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecf5000640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf01032e0 0x7fecf019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecefe00640 1 -- 172.21.15.97:0/2216484987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf01032e0 msgr2=0x7fecf019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.042+0000 7fecefe00640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf01032e0 0x7fecf019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.043+0000 7fecefe00640 1 -- 172.21.15.97:0/2216484987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecf0102920 msgr2=0x7fecf0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.043+0000 7fecefe00640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecf0102920 0x7fecf0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.043+0000 7fecefe00640 1 -- 172.21.15.97:0/2216484987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf019f690 con 0x7fecf0101730 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.043+0000 7fecefe00640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf0101730 0x7fecf0197a70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fece0000c00 tx=0x7fece002c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.043+0000 7fececc00640 1 -- 172.21.15.97:0/2216484987 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0041070 con 0x7fecf0101730 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.043+0000 7fececc00640 1 -- 172.21.15.97:0/2216484987 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece0004450 con 0x7fecf0101730 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.043+0000 7fececc00640 1 -- 172.21.15.97:0/2216484987 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0034440 con 0x7fecf0101730 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.043+0000 7fecf5a00640 1 -- 172.21.15.97:0/2216484987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf019f920 con 0x7fecf0101730 2024-03-26T19:15:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.043+0000 7fecf5a00640 1 -- 172.21.15.97:0/2216484987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf019fdc0 con 0x7fecf0101730 2024-03-26T19:15:25.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.044+0000 7fecf5a00640 1 -- 172.21.15.97:0/2216484987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf0108ad0 con 0x7fecf0101730 2024-03-26T19:15:25.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.046+0000 7fececc00640 1 -- 172.21.15.97:0/2216484987 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fece0044070 con 0x7fecf0101730 2024-03-26T19:15:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.048+0000 7fececc00640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fecd40a2fa0 0x7fecd40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.048+0000 7fececc00640 1 -- 172.21.15.97:0/2216484987 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fece00f0e60 con 0x7fecf0101730 2024-03-26T19:15:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.048+0000 7fecef400640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fecd40a2fa0 0x7fecd40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.049+0000 7fecef400640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fecd40a2fa0 0x7fecd40a5460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fecf0198f80 tx=0x7fecdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.056+0000 7fececc00640 1 -- 172.21.15.97:0/2216484987 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fece00bd6a0 con 0x7fecf0101730 2024-03-26T19:15:25.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.371+0000 7fecf5a00640 1 -- 172.21.15.97:0/2216484987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecf002d460 con 0x7fecf0101730 2024-03-26T19:15:25.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.373+0000 7fececc00640 1 -- 172.21.15.97:0/2216484987 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22581 (secure 0 0 0) 0x7fece00c1740 con 0x7fecf0101730 2024-03-26T19:15:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-26T19:15:25.375 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"2172d50a-eba1-11ee-95d0-87774f69a715","created":"2024-03-26T18:47:23.209624+0000","modified":"2024-03-26T19:13:27.698010+0000","last_up_change":"2024-03-26T18:55:26.007291+0000","last_in_change":"2024-03-26T18:55:06.415863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T18:51:11.459500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T18:56:22.069048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T18:56:23.116030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb36143c-3d96-4a50-af1b-e1c3cdbd5681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6803","nonce":932099107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6805","nonce":932099107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6809","nonce":932099107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":932099107},{"type":"v1","addr":"172.21.15.97:6807","nonce":932099107}]},"public_addr":"172.21.15.97:6803/932099107","cluster_addr":"172.21.15.97:6805/932099107","heartbeat_back_addr":"172.21.15.97:6809/932099107","heartbeat_front_addr":"172.21.15.97:6807/932099107","state":["exists","up"]},{"osd":1,"uuid":"a895a67a-5554-4bad-ab17-6b6971466aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6801","nonce":3472523969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6803","nonce":3472523969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6807","nonce":3472523969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3472523969},{"type":"v1","addr":"172.21.15.105:6805","nonce":3472523969}]},"public_addr":"172.21.15.105:6801/3472523969","cluster_addr":"172.21.15.105:6803/3472523969","heartbeat_back_addr":"172.21.15.105:6807/3472523969","heartbeat_front_addr":"172.21.15.105:6805/3472523969","state":["exists","up"]},{"osd":2,"uuid":"9f46098e-3956-46ba-8a63-14e6557ef35f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6801","nonce":837505140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6803","nonce":837505140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6807","nonce":837505140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":837505140},{"type":"v1","addr":"172.21.15.144:6805","nonce":837505140}]},"public_addr":"172.21.15.144:6801/837505140","cluster_addr":"172.21.15.144:6803/837505140","heartbeat_back_addr":"172.21.15.144:6807/837505140","heartbeat_front_addr":"172.21.15.144:6805/837505140","state":["exists","up"]},{"osd":3,"uuid":"0937aa7b-00ea-417f-b74f-442ca05ac527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6811","nonce":1821717710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6813","nonce":1821717710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6817","nonce":1821717710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1821717710},{"type":"v1","addr":"172.21.15.97:6815","nonce":1821717710}]},"public_addr":"172.21.15.97:6811/1821717710","cluster_addr":"172.21.15.97:6813/1821717710","heartbeat_back_addr":"172.21.15.97:6817/1821717710","heartbeat_front_addr":"172.21.15.97:6815/1821717710","state":["exists","up"]},{"osd":4,"uuid":"3f20c598-4cd5-40d8-befd-e5e50d759bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6809","nonce":4057159020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6811","nonce":4057159020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6815","nonce":4057159020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":4057159020},{"type":"v1","addr":"172.21.15.105:6813","nonce":4057159020}]},"public_addr":"172.21.15.105:6809/4057159020","cluster_addr":"172.21.15.105:6811/4057159020","heartbeat_back_addr":"172.21.15.105:6815/4057159020","heartbeat_front_addr":"172.21.15.105:6813/4057159020","state":["exists","up"]},{"osd":5,"uuid":"d2a8ad7c-197a-4f72-b1fd-bb51a50a3e62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6809","nonce":356775844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6811","nonce":356775844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6815","nonce":356775844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":356775844},{"type":"v1","addr":"172.21.15.144:6813","nonce":356775844}]},"public_addr":"172.21.15.144:6809/356775844","cluster_addr":"172.21.15.144:6811/356775844","heartbeat_back_addr":"172.21.15.144:6815/356775844","heartbeat_front_addr":"172.21.15.144:6813/356775844","state":["exists","up"]},{"osd":6,"uuid":"9b32d8dc-0311-47c2-a2c4-4e324b70eefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6819","nonce":35447762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6821","nonce":35447762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6825","nonce":35447762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":35447762},{"type":"v1","addr":"172.21.15.97:6823","nonce":35447762}]},"public_addr":"172.21.15.97:6819/35447762","cluster_addr":"172.21.15.97:6821/35447762","heartbeat_back_addr":"172.21.15.97:6825/35447762","heartbeat_front_addr":"172.21.15.97:6823/35447762","state":["exists","up"]},{"osd":7,"uuid":"cbf3ac56-47cf-4b95-ba5f-5d8e72185128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6817","nonce":1025195601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6819","nonce":1025195601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6823","nonce":1025195601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":1025195601},{"type":"v1","addr":"172.21.15.105:6821","nonce":1025195601}]},"public_addr":"172.21.15.105:6817/1025195601","cluster_addr":"172.21.15.105:6819/1025195601","heartbeat_back_addr":"172.21.15.105:6823/1025195601","heartbeat_front_addr":"172.21.15.105:6821/1025195601","state":["exists","up"]},{"osd":8,"uuid":"c94dd159-fde0-4285-9b5d-e837afcb8f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6817","nonce":3722681651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6819","nonce":3722681651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6823","nonce":3722681651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3722681651},{"type":"v1","addr":"172.21.15.144:6821","nonce":3722681651}]},"public_addr":"172.21.15.144:6817/3722681651","cluster_addr":"172.21.15.144:6819/3722681651","heartbeat_back_addr":"172.21.15.144:6823/3722681651","heartbeat_front_addr":"172.21.15.144:6821/3722681651","state":["exists","up"]},{"osd":9,"uuid":"d179a4d5-7751-4d6b-a273-97807334dbe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6827","nonce":2932879599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6829","nonce":2932879599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6833","nonce":2932879599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2932879599},{"type":"v1","addr":"172.21.15.97:6831","nonce":2932879599}]},"public_addr":"172.21.15.97:6827/2932879599","cluster_addr":"172.21.15.97:6829/2932879599","heartbeat_back_addr":"172.21.15.97:6833/2932879599","heartbeat_front_addr":"172.21.15.97:6831/2932879599","state":["exists","up"]},{"osd":10,"uuid":"6e5fd1b9-0abc-48b2-991a-2e07f7606bfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6825","nonce":4281982959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6827","nonce":4281982959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6831","nonce":4281982959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4281982959},{"type":"v1","addr":"172.21.15.105:6829","nonce":4281982959}]},"public_addr":"172.21.15.105:6825/4281982959","cluster_addr":"172.21.15.105:6827/4281982959","heartbeat_back_addr":"172.21.15.105:6831/4281982959","heartbeat_front_addr":"172.21.15.105:6829/4281982959","state":["exists","up"]},{"osd":11,"uuid":"2347144e-d823-4816-ac8c-65722dd9cc62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6825","nonce":2295652267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6827","nonce":2295652267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6831","nonce":2295652267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2295652267},{"type":"v1","addr":"172.21.15.144:6829","nonce":2295652267}]},"public_addr":"172.21.15.144:6825/2295652267","cluster_addr":"172.21.15.144:6827/2295652267","heartbeat_back_addr":"172.21.15.144:6831/2295652267","heartbeat_front_addr":"172.21.15.144:6829/2295652267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:19.672679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:50:44.639096+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:07.885669+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:51:40.602368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:06.469614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:52:31.341130+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:05.654041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:32.232054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:53:57.484067+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:31.355153+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:54:59.004636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T18:55:24.327544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/668477340":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/2962857505":"2024-03-27T18:48:28.768371+0000","172.21.15.97:6800/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6800/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3308112091":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/3462603610":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/1051395080":"2024-03-27T18:48:28.768371+0000","172.21.15.97:0/3851585874":"2024-03-27T18:47:55.672991+0000","172.21.15.97:6801/350003366":"2024-03-27T18:47:55.672991+0000","172.21.15.97:0/2299096028":"2024-03-27T18:47:55.672991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T19:15:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 -- 172.21.15.97:0/2216484987 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fecd40a2fa0 msgr2=0x7fecd40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fecd40a2fa0 0x7fecd40a5460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fecf0198f80 tx=0x7fecdc005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 -- 172.21.15.97:0/2216484987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf0101730 msgr2=0x7fecf0197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf0101730 0x7fecf0197a70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fece0000c00 tx=0x7fece002c640 comp rx=0 tx=0).stop 2024-03-26T19:15:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 -- 172.21.15.97:0/2216484987 shutdown_connections 2024-03-26T19:15:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fecd40a2fa0 0x7fecd40a5460 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf01032e0 0x7fecf019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecf0102920 0x7fecf0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 --2- 172.21.15.97:0/2216484987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf0101730 0x7fecf0197a70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 -- 172.21.15.97:0/2216484987 >> 172.21.15.97:0/2216484987 conn(0x7fecf00fcca0 msgr2=0x7fecf0105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 -- 172.21.15.97:0/2216484987 shutdown_connections 2024-03-26T19:15:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.376+0000 7fecf5a00640 1 -- 172.21.15.97:0/2216484987 wait complete. 2024-03-26T19:15:25.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-26T19:15:25.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.609+0000 7feb84000640 1 -- 172.21.15.97:0/1203372611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb7c1056b0 msgr2=0x7feb7c10bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:25.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.609+0000 7feb84000640 1 --2- 172.21.15.97:0/1203372611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb7c1056b0 0x7feb7c10bf40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7feb74009ca0 tx=0x7feb7402c100 comp rx=0 tx=0).stop 2024-03-26T19:15:25.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.609+0000 7feb84000640 1 -- 172.21.15.97:0/1203372611 shutdown_connections 2024-03-26T19:15:25.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.609+0000 7feb84000640 1 --2- 172.21.15.97:0/1203372611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb7c1056b0 0x7feb7c10bf40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.609+0000 7feb84000640 1 --2- 172.21.15.97:0/1203372611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb7c104cf0 0x7feb7c105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.609+0000 7feb84000640 1 --2- 172.21.15.97:0/1203372611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c103b00 0x7feb7c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.609+0000 7feb84000640 1 -- 172.21.15.97:0/1203372611 >> 172.21.15.97:0/1203372611 conn(0x7feb7c0ff070 msgr2=0x7feb7c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:25.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.609+0000 7feb84000640 1 -- 172.21.15.97:0/1203372611 shutdown_connections 2024-03-26T19:15:25.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.609+0000 7feb84000640 1 -- 172.21.15.97:0/1203372611 wait complete. 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.611+0000 7feb84000640 1 Processor -- start 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.611+0000 7feb84000640 1 -- start start 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.611+0000 7feb84000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c103b00 0x7feb7c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb84000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb7c104cf0 0x7feb7c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb84000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb7c1056b0 0x7feb7c1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb84000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c10f760 con 0x7feb7c103b00 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb84000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c10f5e0 con 0x7feb7c104cf0 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb84000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c10f8e0 con 0x7feb7c1056b0 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb80c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb7c104cf0 0x7feb7c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb81600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c103b00 0x7feb7c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb80c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb7c104cf0 0x7feb7c19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.97:38376/0 (socket says 172.21.15.97:38376) 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb80c00640 1 -- 172.21.15.97:0/308539375 learned_addr learned my addr 172.21.15.97:0/308539375 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb82000640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb7c1056b0 0x7feb7c1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb80c00640 1 -- 172.21.15.97:0/308539375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb7c1056b0 msgr2=0x7feb7c1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb80c00640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb7c1056b0 0x7feb7c1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb80c00640 1 -- 172.21.15.97:0/308539375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c103b00 msgr2=0x7feb7c199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb80c00640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c103b00 0x7feb7c199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb80c00640 1 -- 172.21.15.97:0/308539375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c1a19b0 con 0x7feb7c104cf0 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.612+0000 7feb81600640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c103b00 0x7feb7c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.613+0000 7feb80c00640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb7c104cf0 0x7feb7c19a2e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7feb6800ea60 tx=0x7feb6800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.613+0000 7feb7a000640 1 -- 172.21.15.97:0/308539375 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800cde0 con 0x7feb7c104cf0 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.613+0000 7feb7a000640 1 -- 172.21.15.97:0/308539375 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb680035f0 con 0x7feb7c104cf0 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.613+0000 7feb84000640 1 -- 172.21.15.97:0/308539375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c1a1ca0 con 0x7feb7c104cf0 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.613+0000 7feb7a000640 1 -- 172.21.15.97:0/308539375 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68010670 con 0x7feb7c104cf0 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.613+0000 7feb84000640 1 -- 172.21.15.97:0/308539375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c1a2190 con 0x7feb7c104cf0 2024-03-26T19:15:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.614+0000 7feb84000640 1 -- 172.21.15.97:0/308539375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7c10ae20 con 0x7feb7c104cf0 2024-03-26T19:15:25.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.618+0000 7feb7a000640 1 -- 172.21.15.97:0/308539375 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7feb680037a0 con 0x7feb7c104cf0 2024-03-26T19:15:25.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.619+0000 7feb7a000640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb640a2fa0 0x7feb640a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:25.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.619+0000 7feb7a000640 1 -- 172.21.15.97:0/308539375 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7feb680cf700 con 0x7feb7c104cf0 2024-03-26T19:15:25.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.619+0000 7feb81600640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb640a2fa0 0x7feb640a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.620+0000 7feb81600640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb640a2fa0 0x7feb640a5460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feb7c19b270 tx=0x7feb70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:25.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.623+0000 7feb7a000640 1 -- 172.21.15.97:0/308539375 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feb6809c500 con 0x7feb7c104cf0 2024-03-26T19:15:25.802 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:25 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2757151944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T19:15:25.802 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:25 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/2216484987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T19:15:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:25 smithi097 ceph-mon[31955]: pgmap v886: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-03-26T19:15:25.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:25 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2757151944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T19:15:25.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:25 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/2216484987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T19:15:25.883 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:25 smithi144 ceph-mon[36053]: pgmap v886: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-03-26T19:15:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.951+0000 7feb84000640 1 -- 172.21.15.97:0/308539375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7feb7c0625c0 con 0x7feb7c104cf0 2024-03-26T19:15:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.960+0000 7feb7a000640 1 -- 172.21.15.97:0/308539375 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7feb6809be00 con 0x7feb7c104cf0 2024-03-26T19:15:25.964 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 -- 172.21.15.97:0/308539375 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb640a2fa0 msgr2=0x7feb640a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb640a2fa0 0x7feb640a5460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feb7c19b270 tx=0x7feb70005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 -- 172.21.15.97:0/308539375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb7c104cf0 msgr2=0x7feb7c19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb7c104cf0 0x7feb7c19a2e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7feb6800ea60 tx=0x7feb6800ef20 comp rx=0 tx=0).stop 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 -- 172.21.15.97:0/308539375 shutdown_connections 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7feb640a2fa0 0x7feb640a5460 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb7c1056b0 0x7feb7c1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb7c104cf0 0x7feb7c19a2e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 --2- 172.21.15.97:0/308539375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb7c103b00 0x7feb7c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 -- 172.21.15.97:0/308539375 >> 172.21.15.97:0/308539375 conn(0x7feb7c0ff070 msgr2=0x7feb7c107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 -- 172.21.15.97:0/308539375 shutdown_connections 2024-03-26T19:15:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:25.964+0000 7feb84000640 1 -- 172.21.15.97:0/308539375 wait complete. 2024-03-26T19:15:25.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:25 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2757151944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T19:15:25.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:25 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/2216484987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T19:15:25.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:25 smithi105 ceph-mon[36270]: pgmap v886: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-03-26T19:15:25.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-26T19:15:26.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:25 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a[31950]: 2024-03-26T19:15:25.953+0000 7faee5800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T19:15:26.068 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a[71193]: ignoring --setuser ceph since I am not root 2024-03-26T19:15:26.068 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a[71193]: ignoring --setgroup ceph since I am not root 2024-03-26T19:15:26.068 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mds[71201]: -- 172.21.15.97:0/2010969846 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a322b9a40 con 0x558a3228c800 2024-03-26T19:15:26.068 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a[71193]: starting mds.a at 2024-03-26T19:15:26.068 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d[70193]: ignoring --setuser ceph since I am not root 2024-03-26T19:15:26.068 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d[70193]: ignoring --setgroup ceph since I am not root 2024-03-26T19:15:26.068 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mds[70216]: -- 172.21.15.97:0/3744661708 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5581ec939a40 con 0x5581ec90d000 2024-03-26T19:15:26.069 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d[70193]: starting mds.d at 2024-03-26T19:15:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.185+0000 7f1911400640 1 -- 172.21.15.97:0/806062679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0774c0 msgr2=0x7f190c077920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.185+0000 7f1911400640 1 --2- 172.21.15.97:0/806062679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0774c0 0x7f190c077920 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009860 tx=0x7f18f402bb70 comp rx=0 tx=0).stop 2024-03-26T19:15:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.185+0000 7f1911400640 1 -- 172.21.15.97:0/806062679 shutdown_connections 2024-03-26T19:15:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.185+0000 7f1911400640 1 --2- 172.21.15.97:0/806062679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190c077e60 0x7f190c078700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.185+0000 7f1911400640 1 --2- 172.21.15.97:0/806062679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0774c0 0x7f190c077920 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.185+0000 7f1911400640 1 --2- 172.21.15.97:0/806062679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c076280 0x7f190c0766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.185+0000 7f1911400640 1 -- 172.21.15.97:0/806062679 >> 172.21.15.97:0/806062679 conn(0x7f190c0ff070 msgr2=0x7f190c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.186+0000 7f1911400640 1 -- 172.21.15.97:0/806062679 shutdown_connections 2024-03-26T19:15:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.186+0000 7f1911400640 1 -- 172.21.15.97:0/806062679 wait complete. 2024-03-26T19:15:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.187+0000 7f1911400640 1 Processor -- start 2024-03-26T19:15:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.187+0000 7f1911400640 1 -- start start 2024-03-26T19:15:26.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.187+0000 7f1911400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c076280 0x7f190c19e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:26.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f1911400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0774c0 0x7f190c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:26.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f1911400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190c077e60 0x7f190c1a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:26.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f1911400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f190c113b50 con 0x7f190c077e60 2024-03-26T19:15:26.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f1911400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1139d0 con 0x7f190c076280 2024-03-26T19:15:26.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f1911400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f190c113cd0 con 0x7f190c0774c0 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f1901e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0774c0 0x7f190c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f190a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c076280 0x7f190c19e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f1901e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0774c0 0x7f190c19e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:33656/0 (socket says 172.21.15.97:33656) 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f1901e00640 1 -- 172.21.15.97:0/298747008 learned_addr learned my addr 172.21.15.97:0/298747008 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f190b200640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190c077e60 0x7f190c1a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f190a800640 1 -- 172.21.15.97:0/298747008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0774c0 msgr2=0x7f190c19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f190a800640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0774c0 0x7f190c19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f190a800640 1 -- 172.21.15.97:0/298747008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190c077e60 msgr2=0x7f190c1a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f190a800640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190c077e60 0x7f190c1a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f190a800640 1 -- 172.21.15.97:0/298747008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c1a5e50 con 0x7f190c076280 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.188+0000 7f1901e00640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0774c0 0x7f190c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.189+0000 7f190a800640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c076280 0x7f190c19e230 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00eb00 tx=0x7f18fc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.189+0000 7f1903e00640 1 -- 172.21.15.97:0/298747008 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc010430 con 0x7f190c076280 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.189+0000 7f1903e00640 1 -- 172.21.15.97:0/298747008 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18fc010ab0 con 0x7f190c076280 2024-03-26T19:15:26.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.189+0000 7f1911400640 1 -- 172.21.15.97:0/298747008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c1a6140 con 0x7f190c076280 2024-03-26T19:15:26.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.189+0000 7f1903e00640 1 -- 172.21.15.97:0/298747008 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc017bb0 con 0x7f190c076280 2024-03-26T19:15:26.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.189+0000 7f1911400640 1 -- 172.21.15.97:0/298747008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c1a66e0 con 0x7f190c076280 2024-03-26T19:15:26.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.190+0000 7f1911400640 1 -- 172.21.15.97:0/298747008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c110470 con 0x7f190c076280 2024-03-26T19:15:26.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.192+0000 7f1903e00640 1 -- 172.21.15.97:0/298747008 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f18fc010c60 con 0x7f190c076280 2024-03-26T19:15:26.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.193+0000 7f1903e00640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18f00a2fa0 0x7f18f00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:26.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.193+0000 7f1903e00640 1 -- 172.21.15.97:0/298747008 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8612+0+0 (secure 0 0 0) 0x7f18fc0cff00 con 0x7f190c076280 2024-03-26T19:15:26.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.193+0000 7f1901e00640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18f00a2fa0 0x7f18f00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:26.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.194+0000 7f1901e00640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18f00a2fa0 0x7f18f00a5460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f190c19f740 tx=0x7f18f402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:26.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.197+0000 7f1903e00640 1 -- 172.21.15.97:0/298747008 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f18fc09cb50 con 0x7f190c076280 2024-03-26T19:15:26.384 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c[66669]: ignoring --setuser ceph since I am not root 2024-03-26T19:15:26.384 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c[66669]: ignoring --setgroup ceph since I am not root 2024-03-26T19:15:26.384 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mds[66673]: -- 172.21.15.144:0/3544549554 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5562d5d29a40 con 0x5562d5cfcc00 2024-03-26T19:15:26.384 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c[66669]: starting mds.c at 2024-03-26T19:15:26.384 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f[66360]: ignoring --setuser ceph since I am not root 2024-03-26T19:15:26.384 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f[66360]: ignoring --setgroup ceph since I am not root 2024-03-26T19:15:26.384 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mds[66364]: -- 172.21.15.144:0/396698061 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f8f611fa40 con 0x55f8f60f2800 2024-03-26T19:15:26.384 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f[66360]: starting mds.f at 2024-03-26T19:15:26.385 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l[66051]: ignoring --setuser ceph since I am not root 2024-03-26T19:15:26.385 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l[66051]: ignoring --setgroup ceph since I am not root 2024-03-26T19:15:26.385 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mds[66055]: -- 172.21.15.144:0/3187266366 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557469649a40 con 0x55746961cc00 2024-03-26T19:15:26.385 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l[66051]: starting mds.l at 2024-03-26T19:15:26.385 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i[66978]: ignoring --setuser ceph since I am not root 2024-03-26T19:15:26.385 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i[66978]: ignoring --setgroup ceph since I am not root 2024-03-26T19:15:26.385 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mds[66982]: -- 172.21.15.144:0/538352637 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c3c799a40 con 0x561c3c76d000 2024-03-26T19:15:26.385 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i[66978]: starting mds.i at 2024-03-26T19:15:26.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.430+0000 7f1911400640 1 -- 172.21.15.97:0/298747008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f190c0624c0 con 0x7f190c076280 2024-03-26T19:15:26.468 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h[65989]: ignoring --setuser ceph since I am not root 2024-03-26T19:15:26.468 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h[65989]: ignoring --setgroup ceph since I am not root 2024-03-26T19:15:26.468 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mds[65993]: -- 172.21.15.105:0/3202963958 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55754de35a40 con 0x55754de09000 2024-03-26T19:15:26.468 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h[65989]: starting mds.h at 2024-03-26T19:15:26.468 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b[66299]: ignoring --setuser ceph since I am not root 2024-03-26T19:15:26.469 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b[66299]: ignoring --setgroup ceph since I am not root 2024-03-26T19:15:26.469 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mds[66303]: -- 172.21.15.105:0/3619030694 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564c8b43a40 con 0x5564c8b17000 2024-03-26T19:15:26.469 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b[66299]: starting mds.b at 2024-03-26T19:15:26.469 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e[65680]: ignoring --setuser ceph since I am not root 2024-03-26T19:15:26.469 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e[65680]: ignoring --setgroup ceph since I am not root 2024-03-26T19:15:26.469 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mds[65684]: -- 172.21.15.105:0/3078932297 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5636bc5e01a0 con 0x5636bc5b6c00 2024-03-26T19:15:26.469 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mds[65684]: -- 172.21.15.105:0/3078932297 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636bc5e3a40 con 0x5636bc5b6c00 2024-03-26T19:15:26.469 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e[65680]: starting mds.e at 2024-03-26T19:15:26.470 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k[65359]: ignoring --setuser ceph since I am not root 2024-03-26T19:15:26.470 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k[65359]: ignoring --setgroup ceph since I am not root 2024-03-26T19:15:26.470 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mds[65363]: -- 172.21.15.105:0/1216171137 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55652f507a40 con 0x55652f4dac00 2024-03-26T19:15:26.470 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k[65359]: starting mds.k at 2024-03-26T19:15:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/308539375' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T19:15:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T19:15:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mon[31955]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T19:15:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mon[31955]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T19:15:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T19:15:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mon[31955]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T19:15:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mon[31955]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T19:15:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/298747008' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T19:15:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:26 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T19:15:26.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/308539375' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T19:15:26.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T19:15:26.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mon[36053]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T19:15:26.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mon[36053]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T19:15:26.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T19:15:26.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mon[36053]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T19:15:26.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mon[36053]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T19:15:26.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/298747008' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T19:15:26.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:26 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T19:15:26.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.963+0000 7f1903e00640 1 -- 172.21.15.97:0/298747008 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f18fc09c450 con 0x7f190c076280 2024-03-26T19:15:26.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.966+0000 7f1911400640 1 -- 172.21.15.97:0/298747008 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18f00a2fa0 msgr2=0x7f18f00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:26.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.966+0000 7f1911400640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18f00a2fa0 0x7f18f00a5460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f190c19f740 tx=0x7f18f402e040 comp rx=0 tx=0).stop 2024-03-26T19:15:26.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.966+0000 7f1911400640 1 -- 172.21.15.97:0/298747008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c076280 msgr2=0x7f190c19e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:26.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.966+0000 7f1911400640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c076280 0x7f190c19e230 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00eb00 tx=0x7f18fc00c2d0 comp rx=0 tx=0).stop 2024-03-26T19:15:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.966+0000 7f1911400640 1 -- 172.21.15.97:0/298747008 shutdown_connections 2024-03-26T19:15:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.966+0000 7f1911400640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f18f00a2fa0 0x7f18f00a5460 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.966+0000 7f1911400640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190c077e60 0x7f190c1a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.966+0000 7f1911400640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0774c0 0x7f190c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.966+0000 7f1911400640 1 --2- 172.21.15.97:0/298747008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c076280 0x7f190c19e230 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.966+0000 7f1911400640 1 -- 172.21.15.97:0/298747008 >> 172.21.15.97:0/298747008 conn(0x7f190c0ff070 msgr2=0x7f190c100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.967+0000 7f1911400640 1 -- 172.21.15.97:0/298747008 shutdown_connections 2024-03-26T19:15:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:26.967+0000 7f1911400640 1 -- 172.21.15.97:0/298747008 wait complete. 2024-03-26T19:15:26.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/308539375' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T19:15:26.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T19:15:26.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mon[36270]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T19:15:26.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mon[36270]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T19:15:26.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T19:15:26.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mon[36270]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T19:15:26.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mon[36270]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T19:15:26.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/298747008' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T19:15:26.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:26 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T19:15:26.991 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-26T19:15:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.182+0000 7f86fbe00640 1 -- 172.21.15.97:0/3576531256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86fc075900 msgr2=0x7f86fc073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.182+0000 7f86fbe00640 1 --2- 172.21.15.97:0/3576531256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86fc075900 0x7f86fc073d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009cd0 tx=0x7f86f402c0f0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.182+0000 7f86fbe00640 1 -- 172.21.15.97:0/3576531256 shutdown_connections 2024-03-26T19:15:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.182+0000 7f86fbe00640 1 --2- 172.21.15.97:0/3576531256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc10dd90 0x7f86fc110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.182+0000 7f86fbe00640 1 --2- 172.21.15.97:0/3576531256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86fc074320 0x7f86fc0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.182+0000 7f86fbe00640 1 --2- 172.21.15.97:0/3576531256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86fc075900 0x7f86fc073d50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.182+0000 7f86fbe00640 1 -- 172.21.15.97:0/3576531256 >> 172.21.15.97:0/3576531256 conn(0x7f86fc0fcd30 msgr2=0x7f86fc0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.182+0000 7f86fbe00640 1 -- 172.21.15.97:0/3576531256 shutdown_connections 2024-03-26T19:15:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.182+0000 7f86fbe00640 1 -- 172.21.15.97:0/3576531256 wait complete. 2024-03-26T19:15:27.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.183+0000 7f86fbe00640 1 Processor -- start 2024-03-26T19:15:27.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.183+0000 7f86fbe00640 1 -- start start 2024-03-26T19:15:27.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.183+0000 7f86fbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86fc074320 0x7f86fc19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:27.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.183+0000 7f86fbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86fc075900 0x7f86fc19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:27.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.183+0000 7f86fbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc10dd90 0x7f86fc1a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:27.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.183+0000 7f86fbe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc113a30 con 0x7f86fc074320 2024-03-26T19:15:27.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.183+0000 7f86fbe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1138b0 con 0x7f86fc10dd90 2024-03-26T19:15:27.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.183+0000 7f86fbe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc113bb0 con 0x7f86fc075900 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86faa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86fc074320 0x7f86fc19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86faa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86fc074320 0x7f86fc19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41540/0 (socket says 172.21.15.97:41540) 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86faa00640 1 -- 172.21.15.97:0/4264138648 learned_addr learned my addr 172.21.15.97:0/4264138648 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86fb400640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc10dd90 0x7f86fc1a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86fa000640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86fc075900 0x7f86fc19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86faa00640 1 -- 172.21.15.97:0/4264138648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86fc075900 msgr2=0x7f86fc19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86faa00640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86fc075900 0x7f86fc19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86faa00640 1 -- 172.21.15.97:0/4264138648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc10dd90 msgr2=0x7f86fc1a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86faa00640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc10dd90 0x7f86fc1a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86faa00640 1 -- 172.21.15.97:0/4264138648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc1a5d20 con 0x7f86fc074320 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86faa00640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86fc074320 0x7f86fc19e100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f86f4000f80 tx=0x7f86f4032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86f3400640 1 -- 172.21.15.97:0/4264138648 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4041020 con 0x7f86fc074320 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.184+0000 7f86f3400640 1 -- 172.21.15.97:0/4264138648 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f4034cd0 con 0x7f86fc074320 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.185+0000 7f86f3400640 1 -- 172.21.15.97:0/4264138648 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4039040 con 0x7f86fc074320 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.185+0000 7f86fbe00640 1 -- 172.21.15.97:0/4264138648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc1a5fb0 con 0x7f86fc074320 2024-03-26T19:15:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.185+0000 7f86fbe00640 1 -- 172.21.15.97:0/4264138648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc1a63b0 con 0x7f86fc074320 2024-03-26T19:15:27.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.186+0000 7f86fbe00640 1 -- 172.21.15.97:0/4264138648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86fc103f60 con 0x7f86fc074320 2024-03-26T19:15:27.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.188+0000 7f86f3400640 1 -- 172.21.15.97:0/4264138648 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7f86f4044020 con 0x7f86fc074320 2024-03-26T19:15:27.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.189+0000 7f86f3400640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f86e00a2f20 0x7f86e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:27.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.189+0000 7f86f3400640 1 -- 172.21.15.97:0/4264138648 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8612+0+0 (secure 0 0 0) 0x7f86f40f1380 con 0x7f86fc074320 2024-03-26T19:15:27.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.189+0000 7f86fa000640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f86e00a2f20 0x7f86e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:27.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.190+0000 7f86fa000640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f86e00a2f20 0x7f86e00a53e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86fc19f610 tx=0x7f86e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:27.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.194+0000 7f86f3400640 1 -- 172.21.15.97:0/4264138648 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f86f404b020 con 0x7f86fc074320 2024-03-26T19:15:27.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.394+0000 7f86fbe00640 1 -- 172.21.15.97:0/4264138648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f86fc02d240 con 0x7f86fc074320 2024-03-26T19:15:27.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T19:15:27.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T19:15:27.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: Cluster is now healthy 2024-03-26T19:15:27.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T19:15:27.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.144:6835/2579769609,v1:172.21.15.144:6837/2579769609] up:boot 2024-03-26T19:15:27.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6834/278290355,v1:172.21.15.105:6835/278290355] up:boot 2024-03-26T19:15:27.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6838/2105572322,v1:172.21.15.105:6839/2105572322] up:boot 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.144:6838/1544116499,v1:172.21.15.144:6839/1544116499] up:boot 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6832/1037460236,v1:172.21.15.105:6833/1037460236] up:boot 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.144:6832/3167050448,v1:172.21.15.144:6833/3167050448] up:boot 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.144:6834/801399578,v1:172.21.15.144:6836/801399578] up:boot 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.105:6836/1039166369,v1:172.21.15.105:6837/1039166369] up:boot 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.97:6836/1622539371,v1:172.21.15.97:6837/1622539371] up:boot 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: mds.? [v2:172.21.15.97:6840/1263702144,v1:172.21.15.97:6841/1263702144] up:boot 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: fsmap 12 up:standby 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T19:15:27.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T19:15:27.864 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T19:15:27.864 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T19:15:27.864 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T19:15:27.864 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T19:15:27.864 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/4264138648' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T19:15:27.864 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:27 smithi097 ceph-mon[31955]: pgmap v888: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: Cluster is now healthy 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.144:6835/2579769609,v1:172.21.15.144:6837/2579769609] up:boot 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6834/278290355,v1:172.21.15.105:6835/278290355] up:boot 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6838/2105572322,v1:172.21.15.105:6839/2105572322] up:boot 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.144:6838/1544116499,v1:172.21.15.144:6839/1544116499] up:boot 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6832/1037460236,v1:172.21.15.105:6833/1037460236] up:boot 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.144:6832/3167050448,v1:172.21.15.144:6833/3167050448] up:boot 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.144:6834/801399578,v1:172.21.15.144:6836/801399578] up:boot 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.105:6836/1039166369,v1:172.21.15.105:6837/1039166369] up:boot 2024-03-26T19:15:27.884 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.97:6836/1622539371,v1:172.21.15.97:6837/1622539371] up:boot 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: mds.? [v2:172.21.15.97:6840/1263702144,v1:172.21.15.97:6841/1263702144] up:boot 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: fsmap 12 up:standby 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/4264138648' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T19:15:27.885 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:27 smithi144 ceph-mon[36053]: pgmap v888: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-03-26T19:15:27.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T19:15:27.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T19:15:27.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: Cluster is now healthy 2024-03-26T19:15:27.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T19:15:27.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.144:6835/2579769609,v1:172.21.15.144:6837/2579769609] up:boot 2024-03-26T19:15:27.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6834/278290355,v1:172.21.15.105:6835/278290355] up:boot 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6838/2105572322,v1:172.21.15.105:6839/2105572322] up:boot 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.144:6838/1544116499,v1:172.21.15.144:6839/1544116499] up:boot 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6832/1037460236,v1:172.21.15.105:6833/1037460236] up:boot 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.144:6832/3167050448,v1:172.21.15.144:6833/3167050448] up:boot 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.144:6834/801399578,v1:172.21.15.144:6836/801399578] up:boot 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.105:6836/1039166369,v1:172.21.15.105:6837/1039166369] up:boot 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.97:6836/1622539371,v1:172.21.15.97:6837/1622539371] up:boot 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: mds.? [v2:172.21.15.97:6840/1263702144,v1:172.21.15.97:6841/1263702144] up:boot 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: fsmap 12 up:standby 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T19:15:27.969 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T19:15:27.970 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T19:15:27.970 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T19:15:27.970 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T19:15:27.970 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='mgr.14152 172.21.15.97:0/105253935' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T19:15:27.970 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/4264138648' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T19:15:27.970 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:27 smithi105 ceph-mon[36270]: pgmap v888: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-03-26T19:15:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.972+0000 7f86f3400640 1 -- 172.21.15.97:0/4264138648 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7f86f40348f0 con 0x7f86fc074320 2024-03-26T19:15:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-03-26T19:15:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.978+0000 7f86fbe00640 1 -- 172.21.15.97:0/4264138648 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f86e00a2f20 msgr2=0x7f86e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.978+0000 7f86fbe00640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f86e00a2f20 0x7f86e00a53e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86fc19f610 tx=0x7f86e4005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.978+0000 7f86fbe00640 1 -- 172.21.15.97:0/4264138648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86fc074320 msgr2=0x7f86fc19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:27.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.978+0000 7f86fbe00640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86fc074320 0x7f86fc19e100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f86f4000f80 tx=0x7f86f4032ec0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.980+0000 7f86fbe00640 1 -- 172.21.15.97:0/4264138648 shutdown_connections 2024-03-26T19:15:27.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.980+0000 7f86fbe00640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7f86e00a2f20 0x7f86e00a53e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.980+0000 7f86fbe00640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc10dd90 0x7f86fc1a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.980+0000 7f86fbe00640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86fc075900 0x7f86fc19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.980+0000 7f86fbe00640 1 --2- 172.21.15.97:0/4264138648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86fc074320 0x7f86fc19e100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:27.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.980+0000 7f86fbe00640 1 -- 172.21.15.97:0/4264138648 >> 172.21.15.97:0/4264138648 conn(0x7f86fc0fcd30 msgr2=0x7f86fc101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:27.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.980+0000 7f86fbe00640 1 -- 172.21.15.97:0/4264138648 shutdown_connections 2024-03-26T19:15:27.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:27.980+0000 7f86fbe00640 1 -- 172.21.15.97:0/4264138648 wait complete. 2024-03-26T19:15:28.006 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-26T19:15:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.197+0000 7fc97aa00640 1 -- 172.21.15.97:0/652205833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc974100af0 msgr2=0x7fc974100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.197+0000 7fc97aa00640 1 --2- 172.21.15.97:0/652205833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc974100af0 0x7fc974100f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009900 tx=0x7fc95c02bc30 comp rx=0 tx=0).stop 2024-03-26T19:15:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.197+0000 7fc97aa00640 1 -- 172.21.15.97:0/652205833 shutdown_connections 2024-03-26T19:15:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.197+0000 7fc97aa00640 1 --2- 172.21.15.97:0/652205833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc974101490 0x7fc974101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.197+0000 7fc97aa00640 1 --2- 172.21.15.97:0/652205833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc974100af0 0x7fc974100f50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.197+0000 7fc97aa00640 1 --2- 172.21.15.97:0/652205833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc974100190 0x7fc9741005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.197+0000 7fc97aa00640 1 -- 172.21.15.97:0/652205833 >> 172.21.15.97:0/652205833 conn(0x7fc9740fbd50 msgr2=0x7fc9740fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.197+0000 7fc97aa00640 1 -- 172.21.15.97:0/652205833 shutdown_connections 2024-03-26T19:15:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.197+0000 7fc97aa00640 1 -- 172.21.15.97:0/652205833 wait complete. 2024-03-26T19:15:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.198+0000 7fc97aa00640 1 Processor -- start 2024-03-26T19:15:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.198+0000 7fc97aa00640 1 -- start start 2024-03-26T19:15:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.198+0000 7fc97aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc974100190 0x7fc974199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.198+0000 7fc97aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc974100af0 0x7fc97419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.198+0000 7fc97aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc974101490 0x7fc9741a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.198+0000 7fc97aa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc97410f780 con 0x7fc974101490 2024-03-26T19:15:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.198+0000 7fc97aa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc97410f600 con 0x7fc974100190 2024-03-26T19:15:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.198+0000 7fc97aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc97410f900 con 0x7fc974100af0 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc978a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc974101490 0x7fc9741a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc973400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc974100af0 0x7fc97419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc973400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc974100af0 0x7fc97419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:33684/0 (socket says 172.21.15.97:33684) 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc973400640 1 -- 172.21.15.97:0/1677178581 learned_addr learned my addr 172.21.15.97:0/1677178581 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc973e00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc974100190 0x7fc974199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc978a00640 1 -- 172.21.15.97:0/1677178581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc974100af0 msgr2=0x7fc97419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc978a00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc974100af0 0x7fc97419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc978a00640 1 -- 172.21.15.97:0/1677178581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc974100190 msgr2=0x7fc974199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc978a00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc974100190 0x7fc974199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc978a00640 1 -- 172.21.15.97:0/1677178581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9741a1a10 con 0x7fc974101490 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.199+0000 7fc978a00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc974101490 0x7fc9741a13a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009fd0 tx=0x7fc96c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.200+0000 7fc970c00640 1 -- 172.21.15.97:0/1677178581 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00ce20 con 0x7fc974101490 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.200+0000 7fc970c00640 1 -- 172.21.15.97:0/1677178581 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96c005020 con 0x7fc974101490 2024-03-26T19:15:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.200+0000 7fc97aa00640 1 -- 172.21.15.97:0/1677178581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9741a1d00 con 0x7fc974101490 2024-03-26T19:15:28.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.200+0000 7fc970c00640 1 -- 172.21.15.97:0/1677178581 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c010720 con 0x7fc974101490 2024-03-26T19:15:28.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.200+0000 7fc97aa00640 1 -- 172.21.15.97:0/1677178581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9741a21d0 con 0x7fc974101490 2024-03-26T19:15:28.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.201+0000 7fc97aa00640 1 -- 172.21.15.97:0/1677178581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97410b010 con 0x7fc974101490 2024-03-26T19:15:28.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.205+0000 7fc970c00640 1 -- 172.21.15.97:0/1677178581 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 20) ==== 137854+0+0 (secure 0 0 0) 0x7fc96c005540 con 0x7fc974101490 2024-03-26T19:15:28.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.206+0000 7fc970c00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc9580a2f20 0x7fc9580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T19:15:28.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.206+0000 7fc970c00640 1 -- 172.21.15.97:0/1677178581 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8058+0+0 (secure 0 0 0) 0x7fc96c0cf9c0 con 0x7fc974101490 2024-03-26T19:15:28.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.206+0000 7fc973e00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc9580a2f20 0x7fc9580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T19:15:28.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.207+0000 7fc973e00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc9580a2f20 0x7fc9580a53e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc964009bd0 tx=0x7fc964005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T19:15:28.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.208+0000 7fc970c00640 1 -- 172.21.15.97:0/1677178581 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc96c09c980 con 0x7fc974101490 2024-03-26T19:15:28.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.410+0000 7fc97aa00640 1 -- 172.21.15.97:0/1677178581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc97402d240 con 0x7fc974101490 2024-03-26T19:15:28.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.974+0000 7fc970c00640 1 -- 172.21.15.97:0/1677178581 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7fc96c09c280 con 0x7fc974101490 2024-03-26T19:15:28.976 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-03-26T19:15:28.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.978+0000 7fc97aa00640 1 -- 172.21.15.97:0/1677178581 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc9580a2f20 msgr2=0x7fc9580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:28.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.978+0000 7fc97aa00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc9580a2f20 0x7fc9580a53e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc964009bd0 tx=0x7fc964005f90 comp rx=0 tx=0).stop 2024-03-26T19:15:28.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.978+0000 7fc97aa00640 1 -- 172.21.15.97:0/1677178581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc974101490 msgr2=0x7fc9741a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T19:15:28.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.978+0000 7fc97aa00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc974101490 0x7fc9741a13a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009fd0 tx=0x7fc96c00ef10 comp rx=0 tx=0).stop 2024-03-26T19:15:28.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.979+0000 7fc97aa00640 1 -- 172.21.15.97:0/1677178581 shutdown_connections 2024-03-26T19:15:28.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.979+0000 7fc97aa00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.97:6800/3680188982,v1:172.21.15.97:6801/3680188982] conn(0x7fc9580a2f20 0x7fc9580a53e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:28.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.979+0000 7fc97aa00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc974101490 0x7fc9741a13a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:28.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.979+0000 7fc97aa00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc974100af0 0x7fc97419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:28.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.979+0000 7fc97aa00640 1 --2- 172.21.15.97:0/1677178581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc974100190 0x7fc974199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T19:15:28.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.979+0000 7fc97aa00640 1 -- 172.21.15.97:0/1677178581 >> 172.21.15.97:0/1677178581 conn(0x7fc9740fbd50 msgr2=0x7fc9740fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T19:15:28.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.980+0000 7fc97aa00640 1 -- 172.21.15.97:0/1677178581 shutdown_connections 2024-03-26T19:15:28.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-26T19:15:28.984+0000 7fc97aa00640 1 -- 172.21.15.97:0/1677178581 wait complete. 2024-03-26T19:15:29.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T19:15:29.098 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T19:15:29.138 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T19:15:29.174 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-26T19:15:29.174 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T19:15:29.225 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T19:15:29.240 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T19:15:29.254 INFO:tasks.cephadm:Stopping all daemons... 2024-03-26T19:15:29.254 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-26T19:15:29.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.a 2024-03-26T19:15:29.329 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:28 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/4264138648' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T19:15:29.329 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:28 smithi097 ceph-mon[31955]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T19:15:29.329 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:28 smithi097 ceph-mon[31955]: from='client.? 172.21.15.97:0/1677178581' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T19:15:29.383 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:28 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/4264138648' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T19:15:29.383 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:28 smithi144 ceph-mon[36053]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T19:15:29.383 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:28 smithi144 ceph-mon[36053]: from='client.? 172.21.15.97:0/1677178581' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T19:15:29.468 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:28 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/4264138648' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T19:15:29.468 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:28 smithi105 ceph-mon[36270]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T19:15:29.468 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:28 smithi105 ceph-mon[36270]: from='client.? 172.21.15.97:0/1677178581' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T19:15:29.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:29 smithi097 systemd[1]: Stopping Ceph mon.a for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:15:30.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:29 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a[31950]: 2024-03-26T19:15:29.772+0000 7faeec600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:15:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:29 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a[31950]: 2024-03-26T19:15:29.772+0000 7faeec600700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-26T19:15:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:29 smithi097 podman[89818]: 2024-03-26 19:15:29.907680317 +0000 UTC m=+0.285543960 container died 67e023dc813fba1d66fcb80bb4e19d8ca4fca3a0445c139afa7d12f574757fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:15:30.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:30 smithi097 podman[89818]: 2024-03-26 19:15:30.425427243 +0000 UTC m=+0.803290846 container cleanup 67e023dc813fba1d66fcb80bb4e19d8ca4fca3a0445c139afa7d12f574757fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-26T19:15:30.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:30 smithi097 bash[89818]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a 2024-03-26T19:15:31.025 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 26 19:15:30 smithi097 podman[89832]: 2024-03-26 19:15:30.660563195 +0000 UTC m=+0.738995372 container remove 67e023dc813fba1d66fcb80bb4e19d8ca4fca3a0445c139afa7d12f574757fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-a, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T19:15:31.060 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.a.service' 2024-03-26T19:15:31.204 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:15:31.204 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-26T19:15:31.204 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-26T19:15:31.204 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.b 2024-03-26T19:15:31.509 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:31 smithi105 systemd[1]: Stopping Ceph mon.b for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:15:31.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:31 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-b[36266]: 2024-03-26T19:15:31.508+0000 7f452e7ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:15:31.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:31 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-b[36266]: 2024-03-26T19:15:31.508+0000 7f452e7ab700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-26T19:15:31.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:31 smithi105 podman[71698]: 2024-03-26 19:15:31.692325032 +0000 UTC m=+0.297587189 container died 5a28e055d823f7f36efe6d1c635b3a982498c31bc93f4220f0e2221c23d5e40d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-b, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T19:15:32.394 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:32 smithi105 podman[71698]: 2024-03-26 19:15:32.067881813 +0000 UTC m=+0.673143960 container cleanup 5a28e055d823f7f36efe6d1c635b3a982498c31bc93f4220f0e2221c23d5e40d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-03-26T19:15:32.394 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 26 19:15:32 smithi105 bash[71698]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-b 2024-03-26T19:15:32.535 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.b.service' 2024-03-26T19:15:32.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:15:32.581 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-26T19:15:32.581 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-26T19:15:32.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.c 2024-03-26T19:15:32.878 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:32 smithi144 systemd[1]: Stopping Ceph mon.c for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:15:33.134 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:32 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-c[36049]: 2024-03-26T19:15:32.877+0000 7fbac0a27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:15:33.134 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:32 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-c[36049]: 2024-03-26T19:15:32.877+0000 7fbac0a27700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-26T19:15:33.134 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:32 smithi144 podman[72010]: 2024-03-26 19:15:32.919158807 +0000 UTC m=+0.150979940 container died a632f2217aa92887c9a1d1075afdc47830df9369eb293ca614f35146b90644ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-c, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:15:33.504 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:33 smithi144 podman[72010]: 2024-03-26 19:15:33.311690507 +0000 UTC m=+0.543511626 container cleanup a632f2217aa92887c9a1d1075afdc47830df9369eb293ca614f35146b90644ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-c, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T19:15:33.504 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:33 smithi144 bash[72010]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-c 2024-03-26T19:15:33.731 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.c.service' 2024-03-26T19:15:33.755 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:33 smithi144 podman[72025]: 2024-03-26 19:15:33.504979109 +0000 UTC m=+0.585541792 container remove a632f2217aa92887c9a1d1075afdc47830df9369eb293ca614f35146b90644ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mon-c, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:15:33.756 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:33 smithi144 systemd[1]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.c.service: Deactivated successfully. 2024-03-26T19:15:33.756 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:33 smithi144 systemd[1]: Stopped Ceph mon.c for 2172d50a-eba1-11ee-95d0-87774f69a715. 2024-03-26T19:15:33.756 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 19:15:33 smithi144 systemd[1]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mon.c.service: Consumed 14.603s CPU time. 2024-03-26T19:15:33.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:15:33.788 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-26T19:15:33.788 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-26T19:15:33.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.x 2024-03-26T19:15:34.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 19:15:33 smithi097 systemd[1]: Stopping Ceph mgr.x for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:15:34.861 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 19:15:34 smithi097 podman[89938]: 2024-03-26 19:15:34.490208909 +0000 UTC m=+0.246929137 container died ad6d5e2f3c879e733a11fc8ac5d803847e35ef8203f11838db8ecb8d40be2db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T19:15:35.301 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 19:15:35 smithi097 podman[89938]: 2024-03-26 19:15:35.039798616 +0000 UTC m=+0.796518838 container cleanup ad6d5e2f3c879e733a11fc8ac5d803847e35ef8203f11838db8ecb8d40be2db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-03-26T19:15:35.301 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 19:15:35 smithi097 bash[89938]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x 2024-03-26T19:15:35.610 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 26 19:15:35 smithi097 podman[89950]: 2024-03-26 19:15:35.299868993 +0000 UTC m=+0.791456898 container remove ad6d5e2f3c879e733a11fc8ac5d803847e35ef8203f11838db8ecb8d40be2db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T19:15:35.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-26T19:15:35.894 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:15:35.894 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-26T19:15:35.894 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-26T19:15:35.894 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.y 2024-03-26T19:15:36.210 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 19:15:35 smithi105 systemd[1]: Stopping Ceph mgr.y for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:15:36.468 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 19:15:36 smithi105 podman[71831]: 2024-03-26 19:15:36.210413387 +0000 UTC m=+0.125459318 container died 201c4d33d4dbf42f9ca88e2960713facb67690f6fb117acf452409a2aeca77ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-26T19:15:36.968 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 19:15:36 smithi105 podman[71831]: 2024-03-26 19:15:36.615606344 +0000 UTC m=+0.530652264 container cleanup 201c4d33d4dbf42f9ca88e2960713facb67690f6fb117acf452409a2aeca77ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T19:15:36.968 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 26 19:15:36 smithi105 bash[71831]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-y 2024-03-26T19:15:37.196 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-26T19:15:37.234 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:15:37.234 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-26T19:15:37.234 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-26T19:15:37.235 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.z 2024-03-26T19:15:37.588 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 19:15:37 smithi144 systemd[1]: Stopping Ceph mgr.z for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:15:37.876 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 19:15:37 smithi144 podman[72135]: 2024-03-26 19:15:37.588261102 +0000 UTC m=+0.159095322 container died 88fa8a76e5eb8ffa3e602f508f30b32b8bf5b4fb361f9d9d29fb09c9416b80d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-26T19:15:38.134 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 19:15:37 smithi144 podman[72135]: 2024-03-26 19:15:37.976126038 +0000 UTC m=+0.546960264 container cleanup 88fa8a76e5eb8ffa3e602f508f30b32b8bf5b4fb361f9d9d29fb09c9416b80d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-26T19:15:38.134 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 19:15:37 smithi144 bash[72135]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mgr-z 2024-03-26T19:15:38.403 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mgr.z.service' 2024-03-26T19:15:38.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:15:38.440 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-26T19:15:38.440 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-26T19:15:38.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.0 2024-03-26T19:15:38.860 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:38 smithi097 systemd[1]: Stopping Ceph osd.0 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:15:39.362 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:38 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0[43972]: 2024-03-26T19:15:38.967+0000 7f4791a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:15:39.362 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:38 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0[43972]: 2024-03-26T19:15:38.967+0000 7f4791a00700 -1 osd.0 91 *** Got signal Terminated *** 2024-03-26T19:15:39.362 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:38 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0[43972]: 2024-03-26T19:15:38.967+0000 7f4791a00700 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:15:44.861 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:44 smithi097 podman[90057]: 2024-03-26 19:15:44.647448116 +0000 UTC m=+5.808760963 container died acf5a90a907ea4fbec30cdb57b7e102dd39137119083ef44599373672e745ed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319) 2024-03-26T19:15:45.361 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:45 smithi097 podman[90057]: 2024-03-26 19:15:45.087069199 +0000 UTC m=+6.248382089 container cleanup acf5a90a907ea4fbec30cdb57b7e102dd39137119083ef44599373672e745ed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-26T19:15:45.361 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:45 smithi097 bash[90057]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0 2024-03-26T19:15:45.860 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:45 smithi097 podman[90081]: 2024-03-26 19:15:45.555748671 +0000 UTC m=+0.903789626 container remove acf5a90a907ea4fbec30cdb57b7e102dd39137119083ef44599373672e745ed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:15:46.653 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:46 smithi097 podman[90173]: 2024-03-26 19:15:46.268852977 +0000 UTC m=+0.035145932 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:15:46.653 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:46 smithi097 podman[90173]: 2024-03-26 19:15:46.381504862 +0000 UTC m=+0.147797829 container create f390531e7dae7708b233f503b11064ab9665a37ea6d331ba311154ac3b667af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-26T19:15:46.653 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:46 smithi097 podman[90173]: 2024-03-26 19:15:46.652128197 +0000 UTC m=+0.418421219 container init f390531e7dae7708b233f503b11064ab9665a37ea6d331ba311154ac3b667af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-26T19:15:47.092 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:46 smithi097 podman[90173]: 2024-03-26 19:15:46.662988041 +0000 UTC m=+0.429280988 container start f390531e7dae7708b233f503b11064ab9665a37ea6d331ba311154ac3b667af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:15:47.092 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:46 smithi097 podman[90173]: 2024-03-26 19:15:46.742468764 +0000 UTC m=+0.508761773 container attach f390531e7dae7708b233f503b11064ab9665a37ea6d331ba311154ac3b667af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-26T19:15:47.360 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 26 19:15:47 smithi097 podman[90173]: 2024-03-26 19:15:47.092207373 +0000 UTC m=+0.858500360 container died f390531e7dae7708b233f503b11064ab9665a37ea6d331ba311154ac3b667af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-03-26T19:15:47.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.0.service' 2024-03-26T19:15:48.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:15:48.055 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-26T19:15:48.055 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-26T19:15:48.055 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.1 2024-03-26T19:15:48.354 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:48 smithi105 systemd[1]: Stopping Ceph osd.1 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:15:48.718 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:48 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1[40864]: 2024-03-26T19:15:48.353+0000 7f58d05fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:15:48.718 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:48 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1[40864]: 2024-03-26T19:15:48.353+0000 7f58d05fd700 -1 osd.1 91 *** Got signal Terminated *** 2024-03-26T19:15:48.718 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:48 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1[40864]: 2024-03-26T19:15:48.353+0000 7f58d05fd700 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:15:54.105 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:53 smithi105 podman[71954]: 2024-03-26 19:15:53.829533348 +0000 UTC m=+5.577381878 container died 6857bb0ab775e0a8f2b61512b26621ef94f77ed851d18e7c2f66ac78b6cb732b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T19:15:54.468 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:54 smithi105 podman[71954]: 2024-03-26 19:15:54.187214201 +0000 UTC m=+5.935062729 container cleanup 6857bb0ab775e0a8f2b61512b26621ef94f77ed851d18e7c2f66ac78b6cb732b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T19:15:54.468 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:54 smithi105 bash[71954]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1 2024-03-26T19:15:54.848 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:54 smithi105 podman[71971]: 2024-03-26 19:15:54.497861522 +0000 UTC m=+0.662304436 container remove 6857bb0ab775e0a8f2b61512b26621ef94f77ed851d18e7c2f66ac78b6cb732b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-03-26T19:15:55.135 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:54 smithi105 podman[72063]: 2024-03-26 19:15:54.751192766 +0000 UTC m=+0.018432511 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:15:55.135 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:54 smithi105 podman[72063]: 2024-03-26 19:15:54.882155817 +0000 UTC m=+0.149395549 container create c5720673c19e4b70b0e192dcbceb0d9b9b412ccfb81f613199ec75b87fc00edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T19:15:55.425 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:55 smithi105 podman[72063]: 2024-03-26 19:15:55.135381325 +0000 UTC m=+0.402621060 container init c5720673c19e4b70b0e192dcbceb0d9b9b412ccfb81f613199ec75b87fc00edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:15:55.425 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:55 smithi105 podman[72063]: 2024-03-26 19:15:55.138603297 +0000 UTC m=+0.405843024 container start c5720673c19e4b70b0e192dcbceb0d9b9b412ccfb81f613199ec75b87fc00edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:15:55.426 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:55 smithi105 podman[72063]: 2024-03-26 19:15:55.218376468 +0000 UTC m=+0.485616221 container attach c5720673c19e4b70b0e192dcbceb0d9b9b412ccfb81f613199ec75b87fc00edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-26T19:15:55.718 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 26 19:15:55 smithi105 podman[72063]: 2024-03-26 19:15:55.42586357 +0000 UTC m=+0.693103307 container died c5720673c19e4b70b0e192dcbceb0d9b9b412ccfb81f613199ec75b87fc00edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T19:15:56.073 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.1.service' 2024-03-26T19:15:56.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:15:56.114 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-26T19:15:56.114 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-26T19:15:56.114 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.2 2024-03-26T19:15:56.421 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:15:56 smithi144 systemd[1]: Stopping Ceph osd.2 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:15:56.689 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:15:56 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2[41198]: 2024-03-26T19:15:56.420+0000 7f70fbd74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:15:56.689 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:15:56 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2[41198]: 2024-03-26T19:15:56.420+0000 7f70fbd74700 -1 osd.2 91 *** Got signal Terminated *** 2024-03-26T19:15:56.689 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:15:56 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2[41198]: 2024-03-26T19:15:56.420+0000 7f70fbd74700 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:16:02.298 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:16:02 smithi144 podman[72261]: 2024-03-26 19:16:02.014416091 +0000 UTC m=+5.712963274 container died 6769defc7394179bf80177143d96a116b269417ee303faff73762d8aeab75a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:16:02.634 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:16:02 smithi144 podman[72261]: 2024-03-26 19:16:02.380302822 +0000 UTC m=+6.078850011 container cleanup 6769defc7394179bf80177143d96a116b269417ee303faff73762d8aeab75a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-26T19:16:02.634 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:16:02 smithi144 bash[72261]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2 2024-03-26T19:16:02.944 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:16:02 smithi144 podman[72290]: 2024-03-26 19:16:02.648390935 +0000 UTC m=+0.637426768 container remove 6769defc7394179bf80177143d96a116b269417ee303faff73762d8aeab75a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-26T19:16:03.210 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:16:02 smithi144 podman[72384]: 2024-03-26 19:16:02.847986056 +0000 UTC m=+0.018211455 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:16:03.211 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:16:02 smithi144 podman[72384]: 2024-03-26 19:16:02.974533944 +0000 UTC m=+0.144759373 container create 734a511a6c1a64c14c9ea9018a4038dfee6a9b5a75b6f0a8c6f2d88eccc98238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-26T19:16:03.211 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:16:03 smithi144 podman[72384]: 2024-03-26 19:16:03.210581872 +0000 UTC m=+0.380807261 container init 734a511a6c1a64c14c9ea9018a4038dfee6a9b5a75b6f0a8c6f2d88eccc98238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-26T19:16:03.505 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:16:03 smithi144 podman[72384]: 2024-03-26 19:16:03.214258548 +0000 UTC m=+0.384483943 container start 734a511a6c1a64c14c9ea9018a4038dfee6a9b5a75b6f0a8c6f2d88eccc98238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-26T19:16:03.505 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:16:03 smithi144 podman[72384]: 2024-03-26 19:16:03.310972871 +0000 UTC m=+0.481198277 container attach 734a511a6c1a64c14c9ea9018a4038dfee6a9b5a75b6f0a8c6f2d88eccc98238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:16:03.797 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 19:16:03 smithi144 podman[72384]: 2024-03-26 19:16:03.505770316 +0000 UTC m=+0.675995715 container died 734a511a6c1a64c14c9ea9018a4038dfee6a9b5a75b6f0a8c6f2d88eccc98238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-26T19:16:04.112 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.2.service' 2024-03-26T19:16:04.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:16:04.153 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-26T19:16:04.154 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-26T19:16:04.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.3 2024-03-26T19:16:04.581 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:04 smithi097 systemd[1]: Stopping Ceph osd.3 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:16:04.861 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:04 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3[50794]: 2024-03-26T19:16:04.789+0000 7f0fb8000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:16:04.861 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:04 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3[50794]: 2024-03-26T19:16:04.789+0000 7f0fb8000700 -1 osd.3 91 *** Got signal Terminated *** 2024-03-26T19:16:04.861 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:04 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3[50794]: 2024-03-26T19:16:04.789+0000 7f0fb8000700 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:16:05.861 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:05 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3[50794]: 2024-03-26T19:16:05.486+0000 7f0fae800700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:38.173225+0000 front 2024-03-26T19:15:38.173334+0000 (oldest deadline 2024-03-26T19:16:05.173703+0000) 2024-03-26T19:16:06.277 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:05 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:05.868+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:06.611 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:06 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3[50794]: 2024-03-26T19:16:06.508+0000 7f0fae800700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:38.173225+0000 front 2024-03-26T19:15:38.173334+0000 (oldest deadline 2024-03-26T19:16:05.173703+0000) 2024-03-26T19:16:06.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:06 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:06.363+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:07.111 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:06 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:06.826+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:07.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:07.319+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:07.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:07.132+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:07.861 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:07 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3[50794]: 2024-03-26T19:16:07.548+0000 7f0fae800700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:38.173225+0000 front 2024-03-26T19:15:38.173334+0000 (oldest deadline 2024-03-26T19:16:05.173703+0000) 2024-03-26T19:16:07.968 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:07 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:07.809+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:08.257 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:07 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:07.867+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:08.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:08 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:08.108+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:08.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:08 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:08.323+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:08.573 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:08 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:08.257+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:08.861 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:08 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3[50794]: 2024-03-26T19:16:08.572+0000 7f0fae800700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:38.173225+0000 front 2024-03-26T19:15:38.173334+0000 (oldest deadline 2024-03-26T19:16:05.173703+0000) 2024-03-26T19:16:09.218 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:08 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:08.839+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:09.299 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:08 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:08.907+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:09.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:09.084+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:09.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:09.317+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:09.610 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:09 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:09.298+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:09.947 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:09 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3[50794]: 2024-03-26T19:16:09.610+0000 7f0fae800700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:38.173225+0000 front 2024-03-26T19:15:38.173334+0000 (oldest deadline 2024-03-26T19:16:05.173703+0000) 2024-03-26T19:16:10.218 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:09 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:09.873+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:10.324 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:09 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:09.946+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:10.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:10 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:10.095+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:10.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:10 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:10.269+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:10.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:10 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:10.323+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:10.611 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:10 smithi097 podman[90276]: 2024-03-26 19:16:10.49413415 +0000 UTC m=+5.846952480 container died 2e35504da4612ebb516ad582f6b634386642d775be0de425218f4b3b7d68771d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-26T19:16:11.111 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:10 smithi097 podman[90276]: 2024-03-26 19:16:10.974080996 +0000 UTC m=+6.326899292 container cleanup 2e35504da4612ebb516ad582f6b634386642d775be0de425218f4b3b7d68771d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-26T19:16:11.111 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:10 smithi097 bash[90276]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3 2024-03-26T19:16:11.111 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:10 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:10.981+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:11.218 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:10 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:10.909+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:11.372 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:11 smithi097 podman[90307]: 2024-03-26 19:16:11.250317824 +0000 UTC m=+0.744584855 container remove 2e35504da4612ebb516ad582f6b634386642d775be0de425218f4b3b7d68771d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T19:16:11.634 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:11 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:11.140+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:11.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:11 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:11.219+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:11.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:11 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:11.371+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:12.218 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:11 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:11.944+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:12.303 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:11 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:11.934+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:12.303 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:12 smithi097 podman[90394]: 2024-03-26 19:16:11.974010943 +0000 UTC m=+0.035697813 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:16:12.303 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:12 smithi097 podman[90394]: 2024-03-26 19:16:12.092956948 +0000 UTC m=+0.154643855 container create 6145f1cffc968b6e0d863b2698b95a974c6ba744f33ff86bc25b70043073f11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:16:12.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:12 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:12.391+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:12.611 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:12 smithi097 podman[90394]: 2024-03-26 19:16:12.430255456 +0000 UTC m=+0.491942369 container init 6145f1cffc968b6e0d863b2698b95a974c6ba744f33ff86bc25b70043073f11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:16:12.611 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:12 smithi097 podman[90394]: 2024-03-26 19:16:12.443926162 +0000 UTC m=+0.505613063 container start 6145f1cffc968b6e0d863b2698b95a974c6ba744f33ff86bc25b70043073f11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:16:12.611 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:12 smithi097 podman[90394]: 2024-03-26 19:16:12.509780384 +0000 UTC m=+0.571467331 container attach 6145f1cffc968b6e0d863b2698b95a974c6ba744f33ff86bc25b70043073f11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-26T19:16:12.634 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:12 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:12.157+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:12.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:12 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:12.256+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:13.243 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 26 19:16:12 smithi097 podman[90394]: 2024-03-26 19:16:12.870849805 +0000 UTC m=+0.932536678 container died 6145f1cffc968b6e0d863b2698b95a974c6ba744f33ff86bc25b70043073f11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T19:16:13.243 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:12 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:12.924+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:13.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:12 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:12.970+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:13.582 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:13 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:13.366+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:13.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.3.service' 2024-03-26T19:16:13.635 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:13.171+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:13.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:13.279+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:13.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:16:13.762 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-26T19:16:13.762 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-26T19:16:13.763 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.4 2024-03-26T19:16:14.045 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:13 smithi105 systemd[1]: Stopping Ceph osd.4 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:16:14.045 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:14 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:14.001+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:14.045 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:14 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:14.001+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:14.357 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:13 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:13.924+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:14.468 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:14 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4[47371]: 2024-03-26T19:16:14.044+0000 7fba6d28e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:16:14.468 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:14 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4[47371]: 2024-03-26T19:16:14.044+0000 7fba6d28e700 -1 osd.4 91 *** Got signal Terminated *** 2024-03-26T19:16:14.468 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:14 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4[47371]: 2024-03-26T19:16:14.044+0000 7fba6d28e700 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:16:14.610 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:14 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:14.356+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:14.634 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:14 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:14.154+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:14.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:14 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:14.292+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:15.347 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:15 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:14.964+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:15.347 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:15 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:14.964+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:15.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:15.320+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:15.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:15.121+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:15.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:15.121+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:15.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:15 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:15.025+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:15.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:15 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:15.025+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:15.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:15 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:15.346+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:15.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:15 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:15.346+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:16.353 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:16 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:16.073+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:16.353 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:16 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:16.073+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:16.359 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:15 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:15.961+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:16.359 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:15 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:15.961+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:16.361 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:16.159+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:16.361 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:16.159+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:16.361 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:16.100+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:16.610 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:16 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:16.358+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:16.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:16 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:16.358+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:16.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:16.360+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:16.718 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:16 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4[47371]: 2024-03-26T19:16:16.352+0000 7fba69882700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.915564+0000 front 2024-03-26T19:15:50.915532+0000 (oldest deadline 2024-03-26T19:16:15.615717+0000) 2024-03-26T19:16:17.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:16 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:16.994+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:17.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:16 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:16.994+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:17.375 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:17 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:17.085+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:17.375 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:17 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:17.085+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:17.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:17.084+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:17.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:17.130+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:17.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:17.130+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:17.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:17.320+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:17.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:17.320+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:17.718 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:17 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4[47371]: 2024-03-26T19:16:17.374+0000 7fba69882700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.915564+0000 front 2024-03-26T19:15:50.915532+0000 (oldest deadline 2024-03-26T19:16:15.615717+0000) 2024-03-26T19:16:17.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:17 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:17.381+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:17.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:17 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:17.381+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:18.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:18 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:18.022+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:18.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:18 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:18.022+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:18.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:18 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:18.101+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:18.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:18 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:18.130+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:18.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:18 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:18.130+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:18.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:18 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:18.328+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:18.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:18 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:18.328+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:18.422 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:18 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:18.112+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:18.422 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:18 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:18.112+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:18.718 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:18 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4[47371]: 2024-03-26T19:16:18.421+0000 7fba69882700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.915564+0000 front 2024-03-26T19:15:50.915532+0000 (oldest deadline 2024-03-26T19:16:15.615717+0000) 2024-03-26T19:16:18.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:18 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:18.400+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:18.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:18 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:18.400+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:19.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:18 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:18.919+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:19.219 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:19 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:19.130+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:19.219 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:19 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:19.130+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:19.362 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:19 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:19.051+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:19.362 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:19 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:19.051+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:19.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:19 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:19.067+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:19.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:19 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:19.115+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:19.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:19 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:19.115+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:19.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:19 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:19.301+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:19.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:19 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:19.301+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:19.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:19 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:19.424+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:19.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:19 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:19.424+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:19.891 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:19 smithi105 podman[72192]: 2024-03-26 19:16:19.6214168 +0000 UTC m=+5.661646931 container died d05565a46841118830f84361b599e29dc5e4fa56d85c4210d5fe4b7e60173ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-26T19:16:20.166 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:19 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:19.890+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:20.166 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:20 smithi105 podman[72192]: 2024-03-26 19:16:20.045172696 +0000 UTC m=+6.085402817 container cleanup d05565a46841118830f84361b599e29dc5e4fa56d85c4210d5fe4b7e60173ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T19:16:20.166 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:20 smithi105 bash[72192]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4 2024-03-26T19:16:20.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:20.070+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:20.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:20.070+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:20.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:20 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:20.091+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:20.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:20 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:20.104+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:20.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:20 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:20.104+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:20.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:20 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:20.278+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:20.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:20 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:20.278+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:20.468 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:20 smithi105 podman[72209]: 2024-03-26 19:16:20.238323854 +0000 UTC m=+0.616480722 container remove d05565a46841118830f84361b599e29dc5e4fa56d85c4210d5fe4b7e60173ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-03-26T19:16:20.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:20 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:20.165+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:20.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:20 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:20.165+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:20.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:20.418+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:20.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:20.418+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:20.942 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:20 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:20.873+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:20.942 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:20 smithi105 podman[72301]: 2024-03-26 19:16:20.556386906 +0000 UTC m=+0.017914058 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:16:20.942 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:20 smithi105 podman[72301]: 2024-03-26 19:16:20.680959105 +0000 UTC m=+0.142486294 container create 6514ef0d53b6dfdf6cf4600242d9c6653be0288600000fbb64636ff464fedb55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:16:20.943 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:20 smithi105 podman[72301]: 2024-03-26 19:16:20.941614699 +0000 UTC m=+0.403141848 container init 6514ef0d53b6dfdf6cf4600242d9c6653be0288600000fbb64636ff464fedb55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0) 2024-03-26T19:16:21.218 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:21 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:21.116+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:21.218 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:21 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:21.116+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:21.219 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:20 smithi105 podman[72301]: 2024-03-26 19:16:20.944763845 +0000 UTC m=+0.406290996 container start 6514ef0d53b6dfdf6cf4600242d9c6653be0288600000fbb64636ff464fedb55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-26T19:16:21.219 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:21 smithi105 podman[72301]: 2024-03-26 19:16:21.008328599 +0000 UTC m=+0.469855765 container attach 6514ef0d53b6dfdf6cf4600242d9c6653be0288600000fbb64636ff464fedb55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-26T19:16:21.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:21 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:21.100+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:21.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:21 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:21.100+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:21.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:21 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:21.087+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:21.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:21 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:21.118+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:21.384 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:21 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:21.118+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:21.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:21 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:21.325+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:21.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:21 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:21.325+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:21.570 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 26 19:16:21 smithi105 podman[72301]: 2024-03-26 19:16:21.235450675 +0000 UTC m=+0.696977832 container died 6514ef0d53b6dfdf6cf4600242d9c6653be0288600000fbb64636ff464fedb55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T19:16:21.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:21 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:21.406+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:21.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:21 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:21.406+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:21.882 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.4.service' 2024-03-26T19:16:21.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:16:21.925 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-26T19:16:21.926 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-26T19:16:21.926 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.5 2024-03-26T19:16:22.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:21 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:21.842+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:22.218 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:22 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:22.071+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:22.219 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:22 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:22.071+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:22.226 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:22 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:22.072+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:22.226 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:21 smithi144 systemd[1]: Stopping Ceph osd.5 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:16:22.226 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:22 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:22.113+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:22.226 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:22 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:22.113+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:22.443 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:22 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:22.110+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:22.443 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:22 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:22.110+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:22.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:22 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:22.291+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:22.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:22 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:22.291+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:22.634 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:22 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:22.225+0000 7fe652508700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:16:22.635 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:22 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:22.225+0000 7fe652508700 -1 osd.5 91 *** Got signal Terminated *** 2024-03-26T19:16:22.635 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:22 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:22.225+0000 7fe652508700 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:16:22.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:22 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:22.441+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:22.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:22 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:22.441+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:23.088 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:22 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:22.796+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:23.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:23 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:23.063+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:23.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:23 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:23.063+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:23.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:23 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:23.088+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:23.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:23 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:23.088+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:23.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:23 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:23.274+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:23.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:23 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:23.274+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:23.385 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:23 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:23.065+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:23.385 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:23 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:23.065+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:23.385 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:23 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:23.065+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.149710+0000 front 2024-03-26T19:16:00.149780+0000 (oldest deadline 2024-03-26T19:16:23.049857+0000) 2024-03-26T19:16:23.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:23 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:23.087+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:23.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:23 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:23.087+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:23.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:23 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:23.406+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:23.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:23 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:23.406+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:24.130 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:23 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:23.776+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:24.131 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:23 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:23.776+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:24.323 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:24 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:24.040+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:24.323 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:24 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:24.040+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:24.323 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:24 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:24.040+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.149710+0000 front 2024-03-26T19:16:00.149780+0000 (oldest deadline 2024-03-26T19:16:23.049857+0000) 2024-03-26T19:16:24.324 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:24 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:24.117+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:24.324 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:24 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:24.117+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:24.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:24.034+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:24.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:24.034+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:24.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:24.034+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:24.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:24 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:24.130+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:24.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:24 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:24.130+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:24.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:24 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:24.130+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:24.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:24 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:24.322+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:24.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:24 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:24.322+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:24.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:24 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:24.322+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:24.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:24.369+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:24.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:24.369+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:24.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:24.369+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:25.164 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:24 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:24.771+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:25.164 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:24 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:24.771+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:25.288 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:25 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:25.068+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:25.288 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:25 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:25.068+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:25.288 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:25 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:25.008+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:25.288 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:25 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:25.008+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:25.288 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:25 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:25.008+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.149710+0000 front 2024-03-26T19:16:00.149780+0000 (oldest deadline 2024-03-26T19:16:23.049857+0000) 2024-03-26T19:16:25.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:24.992+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:25.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:24.992+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:25.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:24 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:24.992+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:25.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:25 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:25.163+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:25.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:25 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:25.163+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:25.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:25 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:25.163+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:25.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:25 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:25.287+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:25.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:25 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:25.287+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:25.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:25 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:25.287+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:25.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:25 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:25.364+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:25.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:25 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:25.364+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:25.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:25 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:25.364+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:26.212 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:25 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:25.788+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:26.212 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:25 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:25.788+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:26.327 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:26.083+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:26.327 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:26.083+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:26.327 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:26.012+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:26.327 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:26.012+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:26.327 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:26.013+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.149710+0000 front 2024-03-26T19:16:00.149780+0000 (oldest deadline 2024-03-26T19:16:23.049857+0000) 2024-03-26T19:16:26.343 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:26.018+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:26.343 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:26.018+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:26.344 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:26.018+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:26.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:26.211+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:26.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:26.211+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:26.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:26.211+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:26.602 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:26.326+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:26.602 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:26.326+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:26.602 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:26 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:26.326+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:26.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:26.342+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:26.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:26.342+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:26.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:26.342+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:27.214 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:26.761+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:27.214 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:26 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:26.761+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:27.343 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:27 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:27.063+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:27.344 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:27 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:27.063+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:27.344 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:27 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:27.044+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.047337+0000 front 2024-03-26T19:15:42.047127+0000 (oldest deadline 2024-03-26T19:16:06.746907+0000) 2024-03-26T19:16:27.344 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:27 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:27.044+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:50.848261+0000 front 2024-03-26T19:15:50.848276+0000 (oldest deadline 2024-03-26T19:16:14.348297+0000) 2024-03-26T19:16:27.344 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:27 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5[48166]: 2024-03-26T19:16:27.044+0000 7fe64eafc700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.149710+0000 front 2024-03-26T19:16:00.149780+0000 (oldest deadline 2024-03-26T19:16:23.049857+0000) 2024-03-26T19:16:27.353 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:26.987+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:27.353 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:26.987+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:27.354 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:26 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:26.987+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:27.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:27 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:27.213+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:27.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:27 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:27.213+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:27.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:27 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:27.213+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:27.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:27.352+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:27.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:27.352+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:27.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:27.352+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:27.633 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:27 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:27.342+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:27.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:27 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:27.342+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:27.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:27 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:27.342+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:28.020 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:27 smithi144 podman[72491]: 2024-03-26 19:16:27.703085911 +0000 UTC m=+5.590362159 container died db7b879727e0d60eb29d4c9ba53955fdc8f61d74f55818513c27557d9b3e143c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-26T19:16:28.178 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:27 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:27.728+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:28.178 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:27 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:27.728+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:28.321 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:28 smithi144 podman[72491]: 2024-03-26 19:16:28.111506779 +0000 UTC m=+5.998783027 container cleanup db7b879727e0d60eb29d4c9ba53955fdc8f61d74f55818513c27557d9b3e143c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-26T19:16:28.321 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:28 smithi144 bash[72491]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5 2024-03-26T19:16:28.321 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:28 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:28.071+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:28.321 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:28 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:28.071+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:28.325 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:27.972+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:28.325 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:27.972+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:28.325 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:27.972+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:28.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:28 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:28.177+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:28.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:28 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:28.177+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:28.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:28 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:28.177+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:28.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:28 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:28.324+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:28.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:28 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:28.324+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:28.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:28 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:28.324+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:28.620 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:28 smithi144 podman[72521]: 2024-03-26 19:16:28.321341516 +0000 UTC m=+0.612749898 container remove db7b879727e0d60eb29d4c9ba53955fdc8f61d74f55818513c27557d9b3e143c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-26T19:16:28.620 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:28 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:28.340+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:28.620 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:28 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:28.340+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:28.620 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:28 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:28.340+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:28.875 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:28 smithi144 podman[72613]: 2024-03-26 19:16:28.522390022 +0000 UTC m=+0.018219380 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:16:28.875 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:28 smithi144 podman[72613]: 2024-03-26 19:16:28.630748761 +0000 UTC m=+0.126578098 container create a3b960af6473e5e618a77ca29049267cec7487eb1e6194e6a5e80f60d6bdab9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T19:16:28.876 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:28 smithi144 podman[72613]: 2024-03-26 19:16:28.875249661 +0000 UTC m=+0.371079005 container init a3b960af6473e5e618a77ca29049267cec7487eb1e6194e6a5e80f60d6bdab9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-26T19:16:28.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:28 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:28.712+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:28.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:28 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:28.712+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:29.134 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:28 smithi144 podman[72613]: 2024-03-26 19:16:28.878534754 +0000 UTC m=+0.374364092 container start a3b960af6473e5e618a77ca29049267cec7487eb1e6194e6a5e80f60d6bdab9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:16:29.134 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:28 smithi144 podman[72613]: 2024-03-26 19:16:28.966843986 +0000 UTC m=+0.462673348 container attach a3b960af6473e5e618a77ca29049267cec7487eb1e6194e6a5e80f60d6bdab9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:16:29.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:29 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:29.041+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:29.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:29 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:29.041+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:29.344 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:28 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:28.945+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:29.344 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:28 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:28.945+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:29.344 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:28 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:28.945+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:29.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:29 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:29.132+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:29.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:29 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:29.132+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:29.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:29 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:29.132+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:29.524 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 19:16:29 smithi144 podman[72613]: 2024-03-26 19:16:29.169623654 +0000 UTC m=+0.665452999 container died a3b960af6473e5e618a77ca29049267cec7487eb1e6194e6a5e80f60d6bdab9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-26T19:16:29.524 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:29 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:29.300+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:29.524 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:29 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:29.300+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:29.525 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:29 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:29.300+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:29.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:29 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:29.342+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:29.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:29 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:29.342+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:29.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:29 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:29.342+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:29.921 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.5.service' 2024-03-26T19:16:29.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:16:29.963 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-26T19:16:29.963 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-26T19:16:29.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.6 2024-03-26T19:16:29.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:29 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:29.707+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:29.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:29 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:29.707+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:30.346 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:29 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:29.964+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:30.347 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:29 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:29.964+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:30.347 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:29 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:29.964+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:30.347 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:30 smithi097 systemd[1]: Stopping Ceph osd.6 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:16:30.349 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:29 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:29.996+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:30.349 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:29 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:29.996+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:30.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:30 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:30.152+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:30.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:30 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:30.152+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:30.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:30 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:30.152+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:30.611 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:30.581+0000 7f5104000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:16:30.611 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:30.581+0000 7f5104000700 -1 osd.6 91 *** Got signal Terminated *** 2024-03-26T19:16:30.611 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:30.581+0000 7f5104000700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:16:30.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:30.343+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:30.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:30.343+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:30.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:30.343+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:30.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:30 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:30.348+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:30.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:30 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:30.348+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:30.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:30 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:30.348+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:31.191 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:30 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:30.719+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:31.191 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:30 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:30.719+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:31.312 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:30.980+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:31.312 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:30.980+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:31.313 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:30 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:30.980+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:31.342 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:30 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:30.953+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:31.342 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:30 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:30.953+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:31.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:31 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:31.191+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:31.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:31 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:31.191+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:31.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:31 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:31.191+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:31.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:31 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:31.311+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:31.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:31 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:31.311+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:31.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:31 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:31.311+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:31.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:31 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:31.342+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:31.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:31 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:31.342+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:31.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:31 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:31.342+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:32.197 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:31 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:31.731+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:32.197 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:31 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:31.731+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:32.279 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:32 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:32.007+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:32.279 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:32 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:32.007+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:32.279 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:32 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:32.007+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:32.339 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:31 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:31.914+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:32.340 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:31 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:31.914+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:32.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:32 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:32.196+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:32.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:32 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:32.196+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:32.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:32 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:32.196+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:32.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:32 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:32.277+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:32.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:32 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:32.277+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:32.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:32 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:32.277+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:32.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:32 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:32.338+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:32.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:32 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:32.338+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:32.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:32 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:32.338+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:32.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:32 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:32.707+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:32.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:32 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:32.707+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:32.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:32 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:32.707+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:33.312 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:33 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:33.038+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:33.312 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:33 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:33.038+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:33.313 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:33 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:33.038+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:33.379 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:32 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:32.947+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:33.380 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:32 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:32.947+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:33.380 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:32 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:32.947+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:33.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:33 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:33.176+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:33.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:33 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:33.176+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:33.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:33 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:33.176+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:33.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:33 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:33.311+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:33.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:33 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:33.311+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:33.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:33 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:33.311+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:33.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:33 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:33.311+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:33.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:33 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:33.379+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:33.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:33 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:33.379+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:33.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:33 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:33.379+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:34.166 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:33 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:33.754+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:34.167 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:33 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:33.754+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:34.167 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:33 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:33.754+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:34.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:34.288+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:34.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:34.288+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:34.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:34.288+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:34.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:34.288+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:34.362 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:34.083+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:34.362 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:34.083+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:34.362 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:34.083+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:34.362 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:34 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:34.083+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:09.641862+0000 front 2024-03-26T19:16:09.641963+0000 (oldest deadline 2024-03-26T19:16:33.742097+0000) 2024-03-26T19:16:34.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:33 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:33.925+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:34.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:33 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:33.925+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:34.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:33 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:33.925+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:34.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:34 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:34.165+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:34.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:34 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:34.165+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:34.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:34 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:34.165+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:34.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:34 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:34.165+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:34.884 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:34 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:34.390+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:34.884 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:34 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:34.390+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:34.884 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:34 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:34.390+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:34.884 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:34 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:34.390+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:35.190 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:34 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:34.778+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:35.190 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:34 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:34.778+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:35.190 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:34 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:34.778+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:35.348 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:34 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:34.936+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:35.349 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:34 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:34.936+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:35.349 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:34 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:34.936+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:35.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:35.034+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.533151+0000 front 2024-03-26T19:15:43.533933+0000 (oldest deadline 2024-03-26T19:16:05.734234+0000) 2024-03-26T19:16:35.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:35.034+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.036137+0000 front 2024-03-26T19:15:51.036195+0000 (oldest deadline 2024-03-26T19:16:14.536358+0000) 2024-03-26T19:16:35.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:35.034+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.739907+0000 front 2024-03-26T19:15:59.739844+0000 (oldest deadline 2024-03-26T19:16:23.239639+0000) 2024-03-26T19:16:35.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6[57472]: 2024-03-26T19:16:35.034+0000 7f50ff800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:09.641862+0000 front 2024-03-26T19:16:09.641963+0000 (oldest deadline 2024-03-26T19:16:33.742097+0000) 2024-03-26T19:16:35.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:35.273+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:35.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:35.273+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:35.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:35.273+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:35.362 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:35 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:35.273+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:35.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:35 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:35.189+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:35.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:35 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:35.189+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:35.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:35 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:35.189+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:35.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:35 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:35.189+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:35.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:35 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:35.348+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:35.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:35 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:35.348+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:35.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:35 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:35.348+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:35.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:35 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:35.348+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:36.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:35 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:35.737+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:36.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:35 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:35.737+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:36.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:35 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:35.737+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:36.281 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:36 smithi097 podman[90499]: 2024-03-26 19:16:36.173438028 +0000 UTC m=+5.743275230 container died 5b2bb91e0a07c477e720443c4144969ccaaee5e81274bdabaee79a724c4737ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD) 2024-03-26T19:16:36.327 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:35 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:35.946+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:36.328 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:35 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:35.946+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:36.328 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:35 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:35.946+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:36.579 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:36 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:36.279+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:36.580 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:36 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:36.279+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:36.580 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:36 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:36.279+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:36.580 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:36 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:36.279+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:36.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:36 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:36.326+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:36.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:36 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:36.326+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:36.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:36 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:36.326+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:36.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:36 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:36.326+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:36.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:36 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:36.235+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:36.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:36 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:36.235+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:36.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:36 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:36.235+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:36.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:36 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:36.235+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:36.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:36 smithi097 podman[90499]: 2024-03-26 19:16:36.699527507 +0000 UTC m=+6.269364716 container cleanup 5b2bb91e0a07c477e720443c4144969ccaaee5e81274bdabaee79a724c4737ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-26T19:16:36.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:36 smithi097 bash[90499]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6 2024-03-26T19:16:37.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:36 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:36.781+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:37.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:36 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:36.781+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:37.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:36 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:36.781+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:37.238 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:36 smithi097 podman[90530]: 2024-03-26 19:16:36.975684082 +0000 UTC m=+0.790680293 container remove 5b2bb91e0a07c477e720443c4144969ccaaee5e81274bdabaee79a724c4737ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-26T19:16:37.292 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:36 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:36.897+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:37.292 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:36 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:36.897+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:37.292 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:36 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:36.897+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:37.488 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:37 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:37.282+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:37.488 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:37 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:37.282+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:37.488 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:37 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:37.282+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:37.488 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:37 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:37.282+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:37.610 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:37 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:37.291+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:37.610 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:37 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:37.291+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:37.610 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:37 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:37.291+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:37.610 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:37 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:37.291+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:37.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:37 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:37.237+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:37.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:37 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:37.237+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:37.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:37 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:37.237+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:37.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:37 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:37.237+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:38.025 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:37 smithi097 podman[90620]: 2024-03-26 19:16:37.613167996 +0000 UTC m=+0.038364729 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:16:38.026 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:37 smithi097 podman[90620]: 2024-03-26 19:16:37.726488814 +0000 UTC m=+0.151685573 container create 3ee1e729c6ac6a17fddd6eecfb6862ee9d7af994bacdb53dfb93ecaf18af1546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-26T19:16:38.026 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:38 smithi097 podman[90620]: 2024-03-26 19:16:38.024404083 +0000 UTC m=+0.449600848 container init 3ee1e729c6ac6a17fddd6eecfb6862ee9d7af994bacdb53dfb93ecaf18af1546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-26T19:16:38.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:37 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:37.771+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:38.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:37 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:37.771+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:38.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:37 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:37.771+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:38.247 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:37 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:37.936+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:38.247 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:37 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:37.936+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:38.247 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:37 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:37.936+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:38.361 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:38 smithi097 podman[90620]: 2024-03-26 19:16:38.033232582 +0000 UTC m=+0.458429364 container start 3ee1e729c6ac6a17fddd6eecfb6862ee9d7af994bacdb53dfb93ecaf18af1546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-26T19:16:38.362 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:38 smithi097 podman[90620]: 2024-03-26 19:16:38.137140267 +0000 UTC m=+0.562337070 container attach 3ee1e729c6ac6a17fddd6eecfb6862ee9d7af994bacdb53dfb93ecaf18af1546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-26T19:16:38.362 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:38 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:38.261+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:38.362 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:38 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:38.261+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:38.362 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:38 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:38.261+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:38.362 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:38 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:38.261+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:38.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:38 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:38.246+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:38.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:38 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:38.246+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:38.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:38 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:38.246+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:38.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:38 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:38.246+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:38.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:38 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:38.325+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:38.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:38 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:38.325+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:38.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:38 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:38.325+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:38.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:38 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:38.325+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:38.853 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 26 19:16:38 smithi097 podman[90620]: 2024-03-26 19:16:38.456972456 +0000 UTC m=+0.882169235 container died 3ee1e729c6ac6a17fddd6eecfb6862ee9d7af994bacdb53dfb93ecaf18af1546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-26T19:16:39.216 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:38 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:38.955+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:39.216 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:38 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:38.955+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:39.216 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:38 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:38.955+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:39.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:38 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:38.752+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:39.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:38 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:38.753+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:39.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:38 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:38.753+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:39.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.6.service' 2024-03-26T19:16:39.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:16:39.387 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-26T19:16:39.387 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-26T19:16:39.387 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.7 2024-03-26T19:16:39.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:39 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:39.225+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:39.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:39 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:39.225+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:39.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:39 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:39.225+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:39.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:39 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:39.225+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:39.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:39 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:39.214+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:39.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:39 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:39.215+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:39.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:39 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:39.215+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:39.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:39 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:39.215+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:39.675 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:39 smithi105 systemd[1]: Stopping Ceph osd.7 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:16:39.675 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:39 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:39.284+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:39.675 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:39 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:39.284+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:39.676 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:39 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:39.284+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:39.676 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:39 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:39.284+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:39.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:39 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:39.674+0000 7f383f598700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:16:39.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:39 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:39.674+0000 7f383f598700 -1 osd.7 91 *** Got signal Terminated *** 2024-03-26T19:16:39.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:39 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:39.674+0000 7f383f598700 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:16:39.969 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:39 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:39.783+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:39.969 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:39 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:39.783+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:39.969 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:39 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:39.783+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:40.198 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:39 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:39.925+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:40.198 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:39 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:39.925+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:40.198 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:39 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:39.925+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:40.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:40 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:40.250+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:40.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:40 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:40.250+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:40.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:40 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:40.250+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:40.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:40 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:40.250+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:40.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:40 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:40.196+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:40.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:40 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:40.196+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:40.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:40 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:40.196+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:40.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:40 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:40.196+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:40.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:40 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:40.196+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:40.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:40 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:40.313+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:40.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:40 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:40.313+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:40.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:40 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:40.313+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:40.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:40 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:40.313+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:41.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:40 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:40.830+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:41.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:40 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:40.830+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:41.219 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:40 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:40.830+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:41.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:40 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:40.919+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:41.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:40 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:40.919+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:41.384 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:40 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:40.919+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:41.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:41 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:41.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:41.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:41 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:41.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:41.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:41 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:41.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:41.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:41 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:41.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:41.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:41 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:41.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:41.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:41 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:41.236+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:41.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:41 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:41.236+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:41.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:41 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:41.236+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:41.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:41 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:41.236+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:41.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:41 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:41.341+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:41.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:41 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:41.341+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:41.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:41 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:41.341+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:41.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:41 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:41.341+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:42.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:41 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:41.882+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:42.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:41 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:41.882+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:42.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:41 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:41.882+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:42.135 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:41 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:41.882+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:42.135 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:42 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:42.126+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:42.135 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:42 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:42.126+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:42.135 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:42 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:42.126+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:42.135 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:42 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:42.126+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:42.135 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:42 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:42.126+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:42.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:41 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:41.842+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:42.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:41 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:41.842+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:42.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:41 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:41.842+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:42.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:41 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:41.842+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:17.635928+0000 front 2024-03-26T19:16:17.635999+0000 (oldest deadline 2024-03-26T19:16:41.136033+0000) 2024-03-26T19:16:42.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:42 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:42.200+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:42.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:42 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:42.200+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:42.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:42 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:42.200+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:42.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:42 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:42.200+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:42.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:42.348+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:42.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:42.348+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:42.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:42.348+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:42.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:42.348+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:42.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:42.348+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:43.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:42 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:42.881+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:43.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:42 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:42.881+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:43.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:42 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:42.881+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:43.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:42 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:42.881+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:43.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:42.807+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:43.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:42.807+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:43.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:42.807+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:43.219 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:42.807+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:17.635928+0000 front 2024-03-26T19:16:17.635999+0000 (oldest deadline 2024-03-26T19:16:41.136033+0000) 2024-03-26T19:16:43.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:43 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:43.193+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:43.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:43 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:43.193+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:43.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:43 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:43.193+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:43.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:43 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:43.193+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:43.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:43 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:43.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:43.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:43 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:43.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:43.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:43 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:43.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:43.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:43 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:43.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:43.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:43 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:43.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:43.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:43 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:43.372+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:43.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:43 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:43.372+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:43.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:43 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:43.372+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:43.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:43 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:43.372+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:43.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:43 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:43.372+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:44.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:43 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:43.874+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:44.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:43 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:43.874+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:44.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:43 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:43.874+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:44.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:43 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:43.874+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:44.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:43 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:43.763+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.131390+0000 front 2024-03-26T19:15:52.131464+0000 (oldest deadline 2024-03-26T19:16:18.031582+0000) 2024-03-26T19:16:44.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:43 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:43.763+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:00.932735+0000 front 2024-03-26T19:16:00.932760+0000 (oldest deadline 2024-03-26T19:16:23.232764+0000) 2024-03-26T19:16:44.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:43 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:43.763+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.434786+0000 front 2024-03-26T19:16:08.434597+0000 (oldest deadline 2024-03-26T19:16:31.934243+0000) 2024-03-26T19:16:44.218 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:43 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7[53934]: 2024-03-26T19:16:43.763+0000 7f383bb8c700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:17.635928+0000 front 2024-03-26T19:16:17.635999+0000 (oldest deadline 2024-03-26T19:16:41.136033+0000) 2024-03-26T19:16:44.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:44 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:44.191+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:44.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:44 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:44.191+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:44.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:44 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:44.191+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:44.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:44 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:44.191+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:44.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:44 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:44.191+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:44.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:44 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:44.163+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:44.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:44 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:44.163+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:44.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:44 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:44.163+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:44.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:44 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:44.163+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:44.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:44 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:44.163+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:44.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:44 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:44.372+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:44.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:44 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:44.372+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:44.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:44 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:44.372+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:44.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:44 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:44.372+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:44.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:44 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:44.372+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:45.129 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:44 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:44.834+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:45.129 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:44 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:44.834+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:45.129 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:44 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:44.834+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:45.129 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:44 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:44.834+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:45.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:45 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:45.128+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:45.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:45 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:45.128+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:45.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:45 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:45.128+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:45.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:45 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:45.128+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:45.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:45 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:45.128+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:45.576 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:45 smithi105 podman[72418]: 2024-03-26 19:16:45.279898785 +0000 UTC m=+5.701100029 container died 068bbc59e3191e814d6908e318b66c563d1cdd8dc956dd72099d653935f0fac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:16:45.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:45 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:45.420+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:45.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:45 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:45.420+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:45.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:45 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:45.420+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:45.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:45 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:45.420+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:45.577 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:45 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:45.420+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:45.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:45 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:45.228+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:45.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:45 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:45.228+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:45.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:45 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:45.228+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:45.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:45 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:45.228+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:45.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:45 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:45.228+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:45.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:45 smithi105 podman[72418]: 2024-03-26 19:16:45.667035395 +0000 UTC m=+6.088236611 container cleanup 068bbc59e3191e814d6908e318b66c563d1cdd8dc956dd72099d653935f0fac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-26T19:16:45.968 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:45 smithi105 bash[72418]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7 2024-03-26T19:16:46.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:45 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:45.785+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:46.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:45 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:45.785+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:46.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:45 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:45.785+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:46.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:45 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:45.785+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:46.326 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:45 smithi105 podman[72435]: 2024-03-26 19:16:45.97730664 +0000 UTC m=+0.696053309 container remove 068bbc59e3191e814d6908e318b66c563d1cdd8dc956dd72099d653935f0fac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-26T19:16:46.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:46 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:46.218+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:46.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:46 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:46.218+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:46.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:46 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:46.218+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:46.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:46 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:46.218+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:46.361 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:46 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:46.218+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:46.606 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:46 smithi105 podman[72525]: 2024-03-26 19:16:46.229833979 +0000 UTC m=+0.017704967 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:16:46.606 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:46 smithi105 podman[72525]: 2024-03-26 19:16:46.361761666 +0000 UTC m=+0.149632653 container create a0c5b6a800f4fadf239caa35231f8ae27775f8e7e95de64d6ebc637d3335dc65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T19:16:46.606 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:46 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:46.391+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:46.606 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:46 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:46.391+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:46.606 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:46 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:46.391+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:46.606 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:46 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:46.391+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:46.607 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:46 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:46.391+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:46.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:46 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:46.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:46.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:46 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:46.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:46.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:46 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:46.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:46.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:46 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:46.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:46.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:46 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:46.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:46.895 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:46 smithi105 podman[72525]: 2024-03-26 19:16:46.605911222 +0000 UTC m=+0.393782213 container init a0c5b6a800f4fadf239caa35231f8ae27775f8e7e95de64d6ebc637d3335dc65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:16:46.895 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:46 smithi105 podman[72525]: 2024-03-26 19:16:46.609288964 +0000 UTC m=+0.397159944 container start a0c5b6a800f4fadf239caa35231f8ae27775f8e7e95de64d6ebc637d3335dc65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-26T19:16:46.895 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:46 smithi105 podman[72525]: 2024-03-26 19:16:46.680851618 +0000 UTC m=+0.468722625 container attach a0c5b6a800f4fadf239caa35231f8ae27775f8e7e95de64d6ebc637d3335dc65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-26T19:16:47.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:46 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:46.772+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:47.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:46 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:46.772+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:47.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:46 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:46.772+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:47.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:46 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:46.772+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:47.175 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 26 19:16:46 smithi105 podman[72525]: 2024-03-26 19:16:46.895496416 +0000 UTC m=+0.683367413 container died a0c5b6a800f4fadf239caa35231f8ae27775f8e7e95de64d6ebc637d3335dc65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS) 2024-03-26T19:16:47.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:47 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:47.380+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:47.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:47 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:47.380+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:47.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:47 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:47.380+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:47.469 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:47 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:47.380+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:47.469 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:47 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:47.380+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:47.526 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.7.service' 2024-03-26T19:16:47.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:16:47.570 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-26T19:16:47.570 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-26T19:16:47.570 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.8 2024-03-26T19:16:47.599 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:47.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:47.599 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:47.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:47.599 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:47.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:47.599 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:47.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:47.599 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:47.173+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:47.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:47 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:47.265+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:47.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:47 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:47.265+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:47.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:47 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:47.265+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:47.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:47 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:47.265+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:47.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:47 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:47.265+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:47.882 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:47 smithi144 systemd[1]: Stopping Ceph osd.8 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:16:47.882 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:47.779+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:47.882 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:47.779+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:47.882 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:47.779+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:47.882 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:47.779+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:47.883 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:47.881+0000 7fc260427700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:16:47.883 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:47.881+0000 7fc260427700 -1 osd.8 91 *** Got signal Terminated *** 2024-03-26T19:16:47.883 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:47 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:47.881+0000 7fc260427700 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:16:48.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:48 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:48.271+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:48.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:48 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:48.271+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:48.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:48 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:48.271+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:48.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:48 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:48.271+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:48.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:48 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:48.271+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:48.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:48 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:48.201+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:48.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:48 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:48.201+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:48.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:48 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:48.201+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:48.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:48 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:48.201+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:48.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:48 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:48.201+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:48.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:48 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:48.407+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:48.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:48 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:48.407+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:48.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:48 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:48.407+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:48.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:48 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:48.407+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:48.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:48 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:48.407+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:48.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:48 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:48.407+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:49.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:48 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:48.769+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:49.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:48 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:48.769+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:49.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:48 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:48.769+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:49.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:48 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:48.769+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:49.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:49.251+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:49.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:49.251+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:49.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:49.251+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:49.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:49.251+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:49.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:49 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:49.251+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:49.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:49.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:49.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:49.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:49.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:49.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:49.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:49.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:49.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:49.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:49.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:49.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:49.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:49 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:49.369+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:49.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:49 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:49.369+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:49.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:49 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:49.369+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:49.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:49 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:49.369+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:49.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:49 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:49.369+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:49.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:49 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:49.369+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:50.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:49.788+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:50.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:49.788+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:50.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:49.788+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:50.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:49 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:49.788+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:50.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:50 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:50.298+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:50.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:50 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:50.298+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:50.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:50 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:50.298+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:50.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:50 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:50.298+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:50.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:50 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:50.298+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:50.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:50 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:50.298+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:16:50.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:50.217+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:50.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:50.217+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:50.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:50.217+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:50.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:50.217+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:50.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:50.217+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:50.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:50.217+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:50.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:50 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:50.370+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:50.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:50 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:50.370+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:50.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:50 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:50.370+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:50.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:50 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:50.370+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:50.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:50 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:50.370+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:50.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:50 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:50.371+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:51.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:50.821+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:51.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:50.821+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:51.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:50.821+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:51.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:50 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:50.821+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:51.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:51 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:51.343+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:51.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:51 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:51.343+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:51.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:51 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:51.343+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:51.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:51 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:51.343+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:51.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:51 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:51.343+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:51.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:51 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:51.343+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:16:51.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:51.233+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:51.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:51.233+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:51.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:51.233+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:51.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:51.233+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:51.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:51.233+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:51.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:51.233+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:51.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:51 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:51.331+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:51.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:51 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:51.331+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:51.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:51 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:51.331+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:51.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:51 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:51.331+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:51.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:51 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:51.331+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:51.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:51 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:51.331+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:52.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:51.801+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:52.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:51.801+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:52.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:51.801+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:52.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:51 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:51.801+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:52.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:52 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:52.307+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:52.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:52 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:52.307+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:52.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:52 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:52.307+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:52.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:52 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:52.307+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:52.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:52 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:52.307+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:52.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:52 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:52.307+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:16:52.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:52.237+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:52.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:52.237+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:52.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:52.237+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:52.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:52.237+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:52.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:52.237+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:52.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:52.237+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:52.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:52.291+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:52.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:52.291+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:52.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:52.291+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:52.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:52.291+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:52.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:52.291+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:52.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:52 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:52.291+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:53.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:52.795+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.311836+0000 front 2024-03-26T19:15:52.311821+0000 (oldest deadline 2024-03-26T19:16:15.211845+0000) 2024-03-26T19:16:53.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:52.795+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.312739+0000 front 2024-03-26T19:15:59.312954+0000 (oldest deadline 2024-03-26T19:16:22.812848+0000) 2024-03-26T19:16:53.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:52.795+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.114157+0000 front 2024-03-26T19:16:08.114030+0000 (oldest deadline 2024-03-26T19:16:32.813895+0000) 2024-03-26T19:16:53.134 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:52 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8[54746]: 2024-03-26T19:16:52.795+0000 7fc258a13700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.715059+0000 front 2024-03-26T19:16:15.714988+0000 (oldest deadline 2024-03-26T19:16:41.614842+0000) 2024-03-26T19:16:53.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:53 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:53.310+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:53.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:53 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:53.310+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:53.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:53 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:53.310+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:53.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:53 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:53.310+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:53.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:53 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:53.310+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:53.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:53 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:53.310+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:16:53.634 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:53 smithi144 podman[72719]: 2024-03-26 19:16:53.348273756 +0000 UTC m=+5.581476362 container died 1bdf1962d786b3afa531f04445bcf7e90381c03431992663b41a3ab01f906110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-26T19:16:53.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:53 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:53.235+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:53.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:53 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:53.235+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:53.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:53 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:53.236+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:53.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:53 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:53.236+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:53.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:53 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:53.236+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:53.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:53 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:53.236+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:53.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:53.294+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:53.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:53.294+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:53.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:53.294+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:53.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:53.294+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:53.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:53.294+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:53.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:53 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:53.294+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:53.975 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:53 smithi144 podman[72719]: 2024-03-26 19:16:53.756721784 +0000 UTC m=+5.989924380 container cleanup 1bdf1962d786b3afa531f04445bcf7e90381c03431992663b41a3ab01f906110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:16:53.975 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:53 smithi144 bash[72719]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8 2024-03-26T19:16:54.262 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:53 smithi144 podman[72737]: 2024-03-26 19:16:53.974679417 +0000 UTC m=+0.629030645 container remove 1bdf1962d786b3afa531f04445bcf7e90381c03431992663b41a3ab01f906110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-26T19:16:54.519 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:54 smithi144 podman[72828]: 2024-03-26 19:16:54.277466312 +0000 UTC m=+0.017805577 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:16:54.519 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:54 smithi144 podman[72828]: 2024-03-26 19:16:54.375497479 +0000 UTC m=+0.115836755 container create 69b0ceaf69f009d3093ea04402cd25823258eaec86976dd4a46a67724a1c5b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-26T19:16:54.519 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:54 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:54.260+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:54.520 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:54 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:54.260+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:54.520 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:54 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:54.260+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:54.520 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:54 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:54.260+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:54.520 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:54 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:54.260+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:54.520 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:54 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:54.260+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:54.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:54 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:54.305+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:54.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:54 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:54.305+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:54.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:54 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:54.305+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:54.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:54 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:54.305+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:54.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:54 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:54.305+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:54.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:54 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:54.305+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:16:54.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:54 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:54.330+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:54.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:54 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:54.330+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:54.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:54 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:54.330+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:54.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:54 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:54.330+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:54.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:54 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:54.330+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:54.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:54 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:54.330+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:54.884 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:54 smithi144 podman[72828]: 2024-03-26 19:16:54.628477611 +0000 UTC m=+0.368816877 container init 69b0ceaf69f009d3093ea04402cd25823258eaec86976dd4a46a67724a1c5b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.build-date=20240319) 2024-03-26T19:16:54.884 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:54 smithi144 podman[72828]: 2024-03-26 19:16:54.631582671 +0000 UTC m=+0.371921921 container start 69b0ceaf69f009d3093ea04402cd25823258eaec86976dd4a46a67724a1c5b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-26T19:16:54.884 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:54 smithi144 podman[72828]: 2024-03-26 19:16:54.720206889 +0000 UTC m=+0.460546155 container attach 69b0ceaf69f009d3093ea04402cd25823258eaec86976dd4a46a67724a1c5b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:16:55.225 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 19:16:54 smithi144 podman[72828]: 2024-03-26 19:16:54.916618106 +0000 UTC m=+0.656957363 container died 69b0ceaf69f009d3093ea04402cd25823258eaec86976dd4a46a67724a1c5b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T19:16:55.585 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:55 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:55.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:55.586 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:55 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:55.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:55.586 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:55 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:55.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:55.586 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:55 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:55.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:55.586 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:55 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:55.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:55.586 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:55 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:55.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:55.600 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.8.service' 2024-03-26T19:16:55.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:55 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:55.294+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:55.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:55 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:55.294+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:55.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:55 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:55.294+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:55.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:55 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:55.294+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:55.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:55 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:55.294+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:55.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:55 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:55.294+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:16:55.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:16:55.683 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-26T19:16:55.683 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-26T19:16:55.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.9 2024-03-26T19:16:55.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:55.284+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:55.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:55.284+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:55.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:55.284+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:55.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:55.284+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:55.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:55.284+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:55.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:55 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:55.284+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:56.110 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:55 smithi097 systemd[1]: Stopping Ceph osd.9 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:16:56.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:56.208+0000 7f05e3200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:16:56.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:56.208+0000 7f05e3200700 -1 osd.9 91 *** Got signal Terminated *** 2024-03-26T19:16:56.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:56.208+0000 7f05e3200700 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:16:56.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:56.329+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:56.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:56.329+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:56.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:56.329+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:56.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:56.329+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:56.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:56.329+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:56.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:56 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:56.329+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:16:56.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:56 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:56.270+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:56.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:56 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:56.270+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:56.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:56 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:56.270+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:56.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:56 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:56.270+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:56.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:56 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:56.270+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:56.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:56 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:56.270+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:56.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:56 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:56.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:56.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:56 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:56.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:56.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:56 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:56.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:56.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:56 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:56.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:56.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:56 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:56.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:56.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:56 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:56.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:57.575 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:57 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:57.290+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:57.575 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:57 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:57.290+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:57.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:57 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:57.290+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:57.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:57 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:57.290+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:57.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:57 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:57.290+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:57.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:57 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:57.290+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:57.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:57 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:57.290+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:16:57.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:57 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:57.259+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:57.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:57 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:57.259+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:57.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:57 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:57.259+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:57.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:57 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:57.259+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:57.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:57 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:57.259+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:57.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:57 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:57.259+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:57.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:57 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:57.376+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:57.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:57 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:57.376+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:57.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:57 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:57.376+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:57.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:57 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:57.376+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:57.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:57 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:57.376+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:57.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:57 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:57.376+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:16:58.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:58 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:58.335+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:58.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:58 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:58.335+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:58.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:58 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:58.335+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:58.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:58 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:58.335+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:58.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:58 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:58.335+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:58.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:58 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:58.335+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:16:58.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:58 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:58.280+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:58.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:58 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:58.280+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:58.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:58 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:58.280+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:58.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:58 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:58.280+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:58.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:58 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:58.280+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:58.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:58 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:58.280+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:58.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:58 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:58.316+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:58.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:58 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:58.316+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:58.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:58 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:58.316+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:58.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:58 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:58.316+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:58.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:58 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:58.316+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:58.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:58 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:58.316+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:58.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:58 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:58.316+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:16:59.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:59 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:59.323+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:16:59.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:59 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:59.323+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:16:59.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:59 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:59.323+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:16:59.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:59 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:59.323+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:16:59.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:59 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:59.323+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:16:59.612 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:16:59 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:16:59.323+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:16:59.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:59 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:59.328+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:16:59.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:59 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:59.328+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:16:59.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:59 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:59.328+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:16:59.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:59 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:59.328+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:16:59.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:59 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:59.328+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:16:59.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:16:59 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:16:59.328+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:16:59.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:59 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:59.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:16:59.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:59 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:59.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:16:59.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:59 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:59.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:16:59.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:59 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:59.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:16:59.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:59 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:59.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:16:59.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:59 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:59.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:16:59.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:16:59 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:16:59.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:00.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:00 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:00.294+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:00.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:00 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:00.294+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:00.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:00 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:00.294+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:00.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:00 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:00.294+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:00.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:00 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:00.294+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:00.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:00 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:00.294+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:00.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:00 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:00.394+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:17:00.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:00 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:00.394+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:17:00.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:00 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:00.394+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:17:00.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:00 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:00.394+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:17:00.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:00 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:00.394+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:17:00.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:00 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:00.394+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:17:00.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:00 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:00.394+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:00.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:00 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:17:00.363+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:41.683585+0000 front 2024-03-26T19:15:41.683909+0000 (oldest deadline 2024-03-26T19:16:07.583465+0000) 2024-03-26T19:17:00.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:00 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:17:00.363+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:51.686285+0000 front 2024-03-26T19:15:51.685856+0000 (oldest deadline 2024-03-26T19:16:15.185914+0000) 2024-03-26T19:17:00.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:00 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:17:00.363+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:16:01.087893+0000 front 2024-03-26T19:16:01.087845+0000 (oldest deadline 2024-03-26T19:16:23.987519+0000) 2024-03-26T19:17:00.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:00 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:17:00.363+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.490106+0000 front 2024-03-26T19:16:07.489967+0000 (oldest deadline 2024-03-26T19:16:32.789394+0000) 2024-03-26T19:17:00.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:00 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:17:00.363+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.994430+0000 front 2024-03-26T19:16:18.994491+0000 (oldest deadline 2024-03-26T19:16:43.595505+0000) 2024-03-26T19:17:00.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:00 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:17:00.363+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.596692+0000 front 2024-03-26T19:16:23.596486+0000 (oldest deadline 2024-03-26T19:16:49.496225+0000) 2024-03-26T19:17:00.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:00 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9[64392]: 2024-03-26T19:17:00.363+0000 7f05d9a00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:34.198131+0000 front 2024-03-26T19:16:34.198176+0000 (oldest deadline 2024-03-26T19:17:00.097768+0000) 2024-03-26T19:17:01.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:01 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:01.257+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:01.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:01 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:01.257+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:01.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:01 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:01.257+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:01.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:01 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:01.257+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:01.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:01 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:01.257+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:01.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:01 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:01.257+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:01.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:01 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:01.414+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:17:01.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:01 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:01.414+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:17:01.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:01 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:01.414+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:17:01.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:01 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:01.414+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:17:01.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:01 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:01.414+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:17:01.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:01 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:01.414+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:17:01.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:01 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:01.414+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:02.111 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:01 smithi097 podman[90727]: 2024-03-26 19:17:01.734493961 +0000 UTC m=+5.664436479 container died d7c9aae045a2dc68b939dcb4297165525a46278cd64c4b5421b31c6beac79ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-26T19:17:02.478 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:02 smithi097 podman[90727]: 2024-03-26 19:17:02.234267073 +0000 UTC m=+6.164209617 container cleanup d7c9aae045a2dc68b939dcb4297165525a46278cd64c4b5421b31c6beac79ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-26T19:17:02.478 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:02 smithi097 bash[90727]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9 2024-03-26T19:17:02.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:02 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:02.275+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:02.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:02 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:02.275+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:02.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:02 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:02.275+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:02.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:02 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:02.275+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:02.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:02 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:02.275+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:02.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:02 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:02.275+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:02.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:02 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:02.388+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:17:02.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:02 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:02.388+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:17:02.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:02 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:02.388+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:17:02.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:02 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:02.388+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:17:02.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:02 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:02.388+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:17:02.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:02 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:02.388+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:17:02.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:02 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:02.388+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:02.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:02 smithi097 podman[90744]: 2024-03-26 19:17:02.476916717 +0000 UTC m=+0.732681836 container remove d7c9aae045a2dc68b939dcb4297165525a46278cd64c4b5421b31c6beac79ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T19:17:03.599 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:03 smithi097 podman[90830]: 2024-03-26 19:17:03.169512942 +0000 UTC m=+0.033314007 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:17:03.599 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:03 smithi097 podman[90830]: 2024-03-26 19:17:03.302768714 +0000 UTC m=+0.166569811 container create e93308afcd0b5cc32fe51f5bb0c31126ac808972ace5ef5fa40cf2f1537ba765 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-26T19:17:03.599 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:03 smithi097 podman[90830]: 2024-03-26 19:17:03.598084961 +0000 UTC m=+0.461886081 container init e93308afcd0b5cc32fe51f5bb0c31126ac808972ace5ef5fa40cf2f1537ba765 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T19:17:03.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:03 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:03.239+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:03.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:03 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:03.239+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:03.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:03 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:03.239+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:03.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:03 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:03.239+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:03.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:03 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:03.239+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:03.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:03 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:03.239+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:03.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:03 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:03.357+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:17:03.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:03 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:03.357+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:17:03.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:03 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:03.357+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:17:03.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:03 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:03.357+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:17:03.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:03 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:03.357+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:17:03.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:03 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:03.357+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:17:03.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:03 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:03.357+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:03.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:03 smithi097 podman[90830]: 2024-03-26 19:17:03.609909327 +0000 UTC m=+0.473710392 container start e93308afcd0b5cc32fe51f5bb0c31126ac808972ace5ef5fa40cf2f1537ba765 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-26T19:17:03.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:03 smithi097 podman[90830]: 2024-03-26 19:17:03.679963774 +0000 UTC m=+0.543764851 container attach e93308afcd0b5cc32fe51f5bb0c31126ac808972ace5ef5fa40cf2f1537ba765 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T19:17:04.332 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 26 19:17:04 smithi097 podman[90856]: 2024-03-26 19:17:04.091049171 +0000 UTC m=+0.032546048 container died e93308afcd0b5cc32fe51f5bb0c31126ac808972ace5ef5fa40cf2f1537ba765 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T19:17:04.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:04 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:04.249+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:04.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:04 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:04.249+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:04.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:04 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:04.249+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:04.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:04 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:04.249+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:04.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:04 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:04.249+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:04.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:04 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:04.249+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:04.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:04 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:04.344+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:17:04.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:04 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:04.344+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:17:04.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:04 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:04.344+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:17:04.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:04 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:04.344+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:17:04.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:04 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:04.344+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:17:04.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:04 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:04.344+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:17:04.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:04 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:04.344+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:04.758 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.9.service' 2024-03-26T19:17:04.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:04.855 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-26T19:17:04.855 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-26T19:17:04.855 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.10 2024-03-26T19:17:05.146 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:04 smithi105 systemd[1]: Stopping Ceph osd.10 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:05.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:05 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:05.144+0000 7f87aeba6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:05.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:05 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:05.144+0000 7f87aeba6700 -1 osd.10 91 *** Got signal Terminated *** 2024-03-26T19:17:05.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:05 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:05.144+0000 7f87aeba6700 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:17:05.469 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:05 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:05.353+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:17:05.469 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:05 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:05.353+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:17:05.469 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:05 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:05.353+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:17:05.469 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:05 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:05.353+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:17:05.469 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:05 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:05.353+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:17:05.469 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:05 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:05.353+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:17:05.469 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:05 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:05.353+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:05.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:05 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:05.246+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:05.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:05 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:05.246+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:05.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:05 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:05.246+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:05.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:05 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:05.246+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:05.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:05 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:05.246+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:05.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:05 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:05.246+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:06.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:06 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:06.245+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:06.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:06 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:06.245+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:06.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:06 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:06.245+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:06.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:06 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:06.245+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:06.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:06 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:06.245+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:06.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:06 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:06.245+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:06.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:06 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:06.245+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:06.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:06 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:06.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:17:06.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:06 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:06.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:17:06.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:06 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:06.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:17:06.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:06 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:06.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:17:06.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:06 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:06.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:17:06.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:06 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:06.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:17:06.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:06 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:06.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:07.620 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:07 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:07.327+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:17:07.620 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:07 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:07.327+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:17:07.620 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:07 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:07.327+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:17:07.620 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:07 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:07.327+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:17:07.620 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:07 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:07.327+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:17:07.620 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:07 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:07.327+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:17:07.620 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:07 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:07.327+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:07.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:07.207+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:07.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:07.207+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:07.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:07.207+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:07.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:07.207+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:07.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:07.207+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:07.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:07.207+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:07.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:07.207+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:08.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:08 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:08.223+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:08.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:08 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:08.223+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:08.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:08 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:08.223+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:08.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:08 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:08.223+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:08.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:08 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:08.223+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:08.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:08 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:08.223+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:08.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:08 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:08.223+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:08.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:08 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:08.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:17:08.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:08 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:08.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:17:08.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:08 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:08.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:17:08.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:08 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:08.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:17:08.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:08 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:08.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:17:08.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:08 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:08.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:17:08.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:08 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:08.324+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:09.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:09.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:09.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:09.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:09.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:09.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:09.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:09.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:09.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:09.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:09.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:09.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:09.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:09 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:09.225+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:09.718 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:09 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:09.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:43.517739+0000 front 2024-03-26T19:15:43.517494+0000 (oldest deadline 2024-03-26T19:16:07.518033+0000) 2024-03-26T19:17:09.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:09 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:09.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:52.818857+0000 front 2024-03-26T19:15:52.818893+0000 (oldest deadline 2024-03-26T19:16:13.918916+0000) 2024-03-26T19:17:09.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:09 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:09.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:57.821148+0000 front 2024-03-26T19:15:57.821103+0000 (oldest deadline 2024-03-26T19:16:23.721074+0000) 2024-03-26T19:17:09.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:09 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:09.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:08.322919+0000 front 2024-03-26T19:16:08.322810+0000 (oldest deadline 2024-03-26T19:16:33.622538+0000) 2024-03-26T19:17:09.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:09 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:09.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:18.824163+0000 front 2024-03-26T19:16:18.824119+0000 (oldest deadline 2024-03-26T19:16:42.324153+0000) 2024-03-26T19:17:09.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:09 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:09.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:26.326016+0000 front 2024-03-26T19:16:26.326031+0000 (oldest deadline 2024-03-26T19:16:48.025774+0000) 2024-03-26T19:17:09.719 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:09 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10[60601]: 2024-03-26T19:17:09.349+0000 7f87ab19a700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-26T19:16:33.227524+0000 front 2024-03-26T19:16:33.227465+0000 (oldest deadline 2024-03-26T19:16:56.727110+0000) 2024-03-26T19:17:10.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:10 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:10.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:10.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:10 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:10.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:10.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:10 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:10.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:10.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:10 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:10.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:10.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:10 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:10.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:10.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:10 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:10.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:10.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:10 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:10.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:10.968 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:10 smithi105 podman[72633]: 2024-03-26 19:17:10.680193159 +0000 UTC m=+5.635188813 container died 6dd29fcf6d1d9a9b650c2534db55a75a36b6364656815f909b1a2aec76941ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-26T19:17:11.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:10 smithi105 podman[72633]: 2024-03-26 19:17:10.994917839 +0000 UTC m=+5.949913478 container cleanup 6dd29fcf6d1d9a9b650c2534db55a75a36b6364656815f909b1a2aec76941ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T19:17:11.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:10 smithi105 bash[72633]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10 2024-03-26T19:17:11.468 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:11 smithi105 podman[72662]: 2024-03-26 19:17:11.187558703 +0000 UTC m=+0.506786109 container remove 6dd29fcf6d1d9a9b650c2534db55a75a36b6364656815f909b1a2aec76941ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T19:17:11.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:11 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:11.197+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:11.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:11 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:11.197+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:11.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:11 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:11.197+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:11.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:11 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:11.197+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:11.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:11 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:11.197+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:11.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:11 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:11.197+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:11.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:11 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:11.197+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:11.916 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:11 smithi105 podman[72752]: 2024-03-26 19:17:11.505643765 +0000 UTC m=+0.017772248 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:17:11.916 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:11 smithi105 podman[72752]: 2024-03-26 19:17:11.621944061 +0000 UTC m=+0.134072545 container create 0c82728f106eb03a12e763210755fb940bcc1ae75cfbacc329ac76bef0d83152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T19:17:11.916 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:11 smithi105 podman[72752]: 2024-03-26 19:17:11.832676511 +0000 UTC m=+0.344804997 container init 0c82728f106eb03a12e763210755fb940bcc1ae75cfbacc329ac76bef0d83152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-26T19:17:11.916 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:11 smithi105 podman[72752]: 2024-03-26 19:17:11.835677379 +0000 UTC m=+0.347805862 container start 0c82728f106eb03a12e763210755fb940bcc1ae75cfbacc329ac76bef0d83152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-26T19:17:12.218 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:11 smithi105 podman[72752]: 2024-03-26 19:17:11.915792928 +0000 UTC m=+0.427921423 container attach 0c82728f106eb03a12e763210755fb940bcc1ae75cfbacc329ac76bef0d83152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-26T19:17:12.219 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 26 19:17:12 smithi105 podman[72752]: 2024-03-26 19:17:12.123426337 +0000 UTC m=+0.635554821 container died 0c82728f106eb03a12e763210755fb940bcc1ae75cfbacc329ac76bef0d83152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-26T19:17:12.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:12 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:12.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:12.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:12 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:12.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:12.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:12 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:12.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:12.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:12 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:12.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:12.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:12 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:12.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:12.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:12 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:12.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:12.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:12 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:12.169+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:12.693 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.10.service' 2024-03-26T19:17:12.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:12.735 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-26T19:17:12.735 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-26T19:17:12.735 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.11 2024-03-26T19:17:13.026 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:12 smithi144 systemd[1]: Stopping Ceph osd.11 for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:13.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:13.025+0000 7f43c1777700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:13.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:13.025+0000 7f43c1777700 -1 osd.11 91 *** Got signal Terminated *** 2024-03-26T19:17:13.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:13.025+0000 7f43c1777700 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T19:17:13.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:13.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:13.384 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:13.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:13.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:13.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:13.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:13.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:13.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:13.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:13.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:13.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:13.385 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:13 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:13.162+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:14.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:14 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:14.140+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:14.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:14 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:14.140+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:14.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:14 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:14.140+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:14.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:14 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:14.141+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:14.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:14 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:14.141+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:14.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:14 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:14.141+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:14.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:14 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:14.141+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:14.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:14 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:14.141+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6820 osd.8 since back 2024-03-26T19:16:49.090992+0000 front 2024-03-26T19:16:49.090957+0000 (oldest deadline 2024-03-26T19:17:13.790948+0000) 2024-03-26T19:17:15.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:15.168+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:15.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:15.168+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:15.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:15.168+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:15.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:15.168+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:15.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:15.168+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:15.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:15.168+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:15.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:15.168+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:15.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:15 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:15.168+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6820 osd.8 since back 2024-03-26T19:16:49.090992+0000 front 2024-03-26T19:16:49.090957+0000 (oldest deadline 2024-03-26T19:17:13.790948+0000) 2024-03-26T19:17:16.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:16.145+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:16.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:16.145+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:16.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:16.145+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:16.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:16.145+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:16.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:16.145+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:16.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:16.145+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:16.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:16.145+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:16.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:16 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:16.145+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6820 osd.8 since back 2024-03-26T19:16:49.090992+0000 front 2024-03-26T19:16:49.090957+0000 (oldest deadline 2024-03-26T19:17:13.790948+0000) 2024-03-26T19:17:17.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:17.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-26T19:15:42.080047+0000 front 2024-03-26T19:15:42.080171+0000 (oldest deadline 2024-03-26T19:16:06.179948+0000) 2024-03-26T19:17:17.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:17.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-26T19:15:53.082777+0000 front 2024-03-26T19:15:53.082864+0000 (oldest deadline 2024-03-26T19:16:17.182538+0000) 2024-03-26T19:17:17.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:17.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T19:15:59.384256+0000 front 2024-03-26T19:15:59.384285+0000 (oldest deadline 2024-03-26T19:16:23.484400+0000) 2024-03-26T19:17:17.634 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:17.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-26T19:16:07.585590+0000 front 2024-03-26T19:16:07.585684+0000 (oldest deadline 2024-03-26T19:16:33.485568+0000) 2024-03-26T19:17:17.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:17.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-26T19:16:15.186539+0000 front 2024-03-26T19:16:15.186695+0000 (oldest deadline 2024-03-26T19:16:39.886580+0000) 2024-03-26T19:17:17.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:17.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T19:16:23.387476+0000 front 2024-03-26T19:16:23.387415+0000 (oldest deadline 2024-03-26T19:16:48.687529+0000) 2024-03-26T19:17:17.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:17.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-26T19:16:42.090086+0000 front 2024-03-26T19:16:42.090048+0000 (oldest deadline 2024-03-26T19:17:05.589934+0000) 2024-03-26T19:17:17.635 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:17 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11[61380]: 2024-03-26T19:17:17.189+0000 7f43bdd6b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.144:6820 osd.8 since back 2024-03-26T19:16:49.090992+0000 front 2024-03-26T19:16:49.090957+0000 (oldest deadline 2024-03-26T19:17:13.790948+0000) 2024-03-26T19:17:18.692 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:18 smithi144 podman[72947]: 2024-03-26 19:17:18.434508432 +0000 UTC m=+5.510130913 container died 0b6f84205f0075eed9d45e34a2a2bf0d6edc9104e42cd5808d90c25c7243b403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-26T19:17:19.134 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:18 smithi144 podman[72947]: 2024-03-26 19:17:18.691936613 +0000 UTC m=+5.767559083 container cleanup 0b6f84205f0075eed9d45e34a2a2bf0d6edc9104e42cd5808d90c25c7243b403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-26T19:17:19.134 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:18 smithi144 bash[72947]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11 2024-03-26T19:17:19.134 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:18 smithi144 podman[72964]: 2024-03-26 19:17:18.90988581 +0000 UTC m=+0.472150427 container remove 0b6f84205f0075eed9d45e34a2a2bf0d6edc9104e42cd5808d90c25c7243b403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True) 2024-03-26T19:17:19.538 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:19 smithi144 podman[73055]: 2024-03-26 19:17:19.129286833 +0000 UTC m=+0.017343437 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T19:17:19.538 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:19 smithi144 podman[73055]: 2024-03-26 19:17:19.235772983 +0000 UTC m=+0.123829616 container create 7159723b7e5ae94ccdd41bb94479dfaae8a48c6c0b63cd50d0d39fdbb7ecf4ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:17:19.538 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:19 smithi144 podman[73055]: 2024-03-26 19:17:19.462890222 +0000 UTC m=+0.350946817 container init 7159723b7e5ae94ccdd41bb94479dfaae8a48c6c0b63cd50d0d39fdbb7ecf4ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:17:19.538 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:19 smithi144 podman[73055]: 2024-03-26 19:17:19.466149928 +0000 UTC m=+0.354206531 container start 7159723b7e5ae94ccdd41bb94479dfaae8a48c6c0b63cd50d0d39fdbb7ecf4ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T19:17:19.538 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:19 smithi144 podman[73055]: 2024-03-26 19:17:19.537901105 +0000 UTC m=+0.425957696 container attach 7159723b7e5ae94ccdd41bb94479dfaae8a48c6c0b63cd50d0d39fdbb7ecf4ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-26T19:17:19.884 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:19 smithi144 podman[73055]: 2024-03-26 19:17:19.751878043 +0000 UTC m=+0.639934645 container died 7159723b7e5ae94ccdd41bb94479dfaae8a48c6c0b63cd50d0d39fdbb7ecf4ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-26T19:17:20.343 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 19:17:20 smithi144 podman[73081]: 2024-03-26 19:17:20.341053084 +0000 UTC m=+0.579773504 container remove 7159723b7e5ae94ccdd41bb94479dfaae8a48c6c0b63cd50d0d39fdbb7ecf4ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T19:17:20.363 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@osd.11.service' 2024-03-26T19:17:20.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:20.402 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-26T19:17:20.402 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-26T19:17:20.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.a 2024-03-26T19:17:20.861 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:17:20 smithi097 systemd[1]: Stopping Ceph mds.a for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:21.361 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:17:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a[71193]: 2024-03-26T19:17:20.983+0000 7fba1b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:21.361 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:17:20 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a[71193]: 2024-03-26T19:17:20.983+0000 7fba1b200700 -1 mds.a *** got signal Terminated *** 2024-03-26T19:17:22.361 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:17:22 smithi097 podman[90947]: 2024-03-26 19:17:22.077026849 +0000 UTC m=+1.260845907 container died 99cab503a0a8cdb3cf9f756634304f28eaab1ee49139c04d5598dd7b03c40965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-03-26T19:17:22.753 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:17:22 smithi097 podman[90947]: 2024-03-26 19:17:22.518185001 +0000 UTC m=+1.702004071 container cleanup 99cab503a0a8cdb3cf9f756634304f28eaab1ee49139c04d5598dd7b03c40965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T19:17:22.754 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:17:22 smithi097 bash[90947]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a 2024-03-26T19:17:23.111 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 26 19:17:22 smithi097 podman[90961]: 2024-03-26 19:17:22.752689407 +0000 UTC m=+0.660173016 container remove 99cab503a0a8cdb3cf9f756634304f28eaab1ee49139c04d5598dd7b03c40965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T19:17:23.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.a.service' 2024-03-26T19:17:23.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:23.338 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-26T19:17:23.338 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-26T19:17:23.338 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.d 2024-03-26T19:17:23.861 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:17:23 smithi097 systemd[1]: Stopping Ceph mds.d for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:24.361 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:17:23 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d[70193]: 2024-03-26T19:17:23.926+0000 7f661e400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:24.361 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:17:23 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d[70193]: 2024-03-26T19:17:23.926+0000 7f661e400700 -1 mds.d *** got signal Terminated *** 2024-03-26T19:17:26.361 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:17:26 smithi097 podman[91067]: 2024-03-26 19:17:26.067856888 +0000 UTC m=+2.289740409 container stop 3c4c8e413743621cb19e1adc1e68913174b6651b8d0a07eb8b2bbdcdf55b1dab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-26T19:17:26.361 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:17:26 smithi097 podman[91067]: 2024-03-26 19:17:26.097823592 +0000 UTC m=+2.319707096 container died 3c4c8e413743621cb19e1adc1e68913174b6651b8d0a07eb8b2bbdcdf55b1dab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-26T19:17:26.861 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:17:26 smithi097 podman[91067]: 2024-03-26 19:17:26.555032445 +0000 UTC m=+2.776915959 container cleanup 3c4c8e413743621cb19e1adc1e68913174b6651b8d0a07eb8b2bbdcdf55b1dab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:17:26.861 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:17:26 smithi097 bash[91067]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d 2024-03-26T19:17:27.132 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 26 19:17:26 smithi097 podman[91081]: 2024-03-26 19:17:26.864210738 +0000 UTC m=+0.775191989 container remove 3c4c8e413743621cb19e1adc1e68913174b6651b8d0a07eb8b2bbdcdf55b1dab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-d, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T19:17:27.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.d.service' 2024-03-26T19:17:27.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:27.261 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-26T19:17:27.261 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-26T19:17:27.261 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.g 2024-03-26T19:17:27.861 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 26 19:17:27 smithi097 systemd[1]: Stopping Ceph mds.g for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:28.361 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 26 19:17:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-g[70837]: 2024-03-26T19:17:27.871+0000 7ff6d1a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:28.361 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 26 19:17:27 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-g[70837]: 2024-03-26T19:17:27.871+0000 7ff6d1a00700 -1 mds.g *** got signal Terminated *** 2024-03-26T19:17:30.854 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 26 19:17:30 smithi097 podman[91187]: 2024-03-26 19:17:30.571220547 +0000 UTC m=+2.833009437 container died 1446ef437b538c0fa1cb28b84c2a1bb66459c5f976387dea8915f252b9506126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-26T19:17:31.111 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 26 19:17:30 smithi097 podman[91187]: 2024-03-26 19:17:30.975011583 +0000 UTC m=+3.236800495 container cleanup 1446ef437b538c0fa1cb28b84c2a1bb66459c5f976387dea8915f252b9506126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-g, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319) 2024-03-26T19:17:31.111 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 26 19:17:30 smithi097 bash[91187]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-g 2024-03-26T19:17:31.611 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 26 19:17:31 smithi097 podman[91201]: 2024-03-26 19:17:31.27625011 +0000 UTC m=+0.687058393 container remove 1446ef437b538c0fa1cb28b84c2a1bb66459c5f976387dea8915f252b9506126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-g, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T19:17:31.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.g.service' 2024-03-26T19:17:31.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:31.875 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-26T19:17:31.875 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-26T19:17:31.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.j 2024-03-26T19:17:32.111 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 19:17:31 smithi097 systemd[1]: Stopping Ceph mds.j for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:32.611 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 19:17:32 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-j[69846]: 2024-03-26T19:17:32.335+0000 7f71daa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:32.611 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 19:17:32 smithi097 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-j[69846]: 2024-03-26T19:17:32.335+0000 7f71daa00700 -1 mds.j *** got signal Terminated *** 2024-03-26T19:17:36.216 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 19:17:35 smithi097 podman[91306]: 2024-03-26 19:17:35.95100609 +0000 UTC m=+3.710239123 container died 6365380a10d2f9e6f36d3b9213a2bed770cbfdf31c630d2c100a9389ffbb98ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-03-26T19:17:36.580 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 19:17:36 smithi097 podman[91306]: 2024-03-26 19:17:36.386354725 +0000 UTC m=+4.145587735 container cleanup 6365380a10d2f9e6f36d3b9213a2bed770cbfdf31c630d2c100a9389ffbb98ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-j, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T19:17:36.580 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 19:17:36 smithi097 bash[91306]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-j 2024-03-26T19:17:36.861 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 26 19:17:36 smithi097 podman[91321]: 2024-03-26 19:17:36.578904956 +0000 UTC m=+0.609092203 container remove 6365380a10d2f9e6f36d3b9213a2bed770cbfdf31c630d2c100a9389ffbb98ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-26T19:17:37.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.j.service' 2024-03-26T19:17:37.262 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:37.263 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-26T19:17:37.263 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-26T19:17:37.263 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.b 2024-03-26T19:17:37.628 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 19:17:37 smithi105 systemd[1]: Stopping Ceph mds.b for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:37.888 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 19:17:37 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b[66299]: 2024-03-26T19:17:37.627+0000 7f3c8e3e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:37.888 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 19:17:37 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b[66299]: 2024-03-26T19:17:37.627+0000 7f3c8e3e3700 -1 mds.b *** got signal Terminated *** 2024-03-26T19:17:41.912 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 19:17:41 smithi105 podman[72857]: 2024-03-26 19:17:41.637431564 +0000 UTC m=+4.184586207 container died ccc6801e72c38410dde60e5f5420a0f1b59ec2d9953d6e069fab6a49e6615f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=) 2024-03-26T19:17:42.218 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 19:17:41 smithi105 podman[72857]: 2024-03-26 19:17:41.994369792 +0000 UTC m=+4.541524434 container cleanup ccc6801e72c38410dde60e5f5420a0f1b59ec2d9953d6e069fab6a49e6615f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-26T19:17:42.218 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 26 19:17:41 smithi105 bash[72857]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-b 2024-03-26T19:17:42.415 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.b.service' 2024-03-26T19:17:42.453 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:42.454 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-26T19:17:42.454 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-26T19:17:42.454 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.e 2024-03-26T19:17:42.969 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:17:42 smithi105 systemd[1]: Stopping Ceph mds.e for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:42.969 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:17:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e[65680]: 2024-03-26T19:17:42.764+0000 7f0e282b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:42.969 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:17:42 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e[65680]: 2024-03-26T19:17:42.764+0000 7f0e282b8700 -1 mds.e *** got signal Terminated *** 2024-03-26T19:17:46.298 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:17:46 smithi105 podman[72993]: 2024-03-26 19:17:46.034951189 +0000 UTC m=+3.369100226 container died 2fff5cc0ee46be0bdac89e3ef2fea1f87560b55fe741d3fc83417e76d5538aef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-26T19:17:46.298 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:17:46 smithi105 podman[72993]: 2024-03-26 19:17:46.297479782 +0000 UTC m=+3.631628808 container cleanup 2fff5cc0ee46be0bdac89e3ef2fea1f87560b55fe741d3fc83417e76d5538aef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T19:17:46.651 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:17:46 smithi105 bash[72993]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e 2024-03-26T19:17:46.651 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 26 19:17:46 smithi105 podman[73008]: 2024-03-26 19:17:46.540338931 +0000 UTC m=+0.500269994 container remove 2fff5cc0ee46be0bdac89e3ef2fea1f87560b55fe741d3fc83417e76d5538aef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-03-26T19:17:46.666 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.e.service' 2024-03-26T19:17:46.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:46.703 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-26T19:17:46.703 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-26T19:17:46.703 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.h 2024-03-26T19:17:46.968 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:17:46 smithi105 systemd[1]: Stopping Ceph mds.h for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:47.468 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:17:47 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h[65989]: 2024-03-26T19:17:47.025+0000 7f923b8d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:47.469 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:17:47 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h[65989]: 2024-03-26T19:17:47.025+0000 7f923b8d1700 -1 mds.h *** got signal Terminated *** 2024-03-26T19:17:50.302 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:17:50 smithi105 podman[73116]: 2024-03-26 19:17:50.028762862 +0000 UTC m=+3.109158356 container died b609505c9761d26eac2f5ea3bd7e6f2c6878db7773a9e8d9748e793c6b7f0db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T19:17:50.619 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:17:50 smithi105 podman[73116]: 2024-03-26 19:17:50.392936947 +0000 UTC m=+3.473332435 container cleanup b609505c9761d26eac2f5ea3bd7e6f2c6878db7773a9e8d9748e793c6b7f0db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T19:17:50.619 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 26 19:17:50 smithi105 bash[73116]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-h 2024-03-26T19:17:50.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.h.service' 2024-03-26T19:17:50.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:50.804 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-26T19:17:50.804 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-26T19:17:50.804 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.k 2024-03-26T19:17:51.218 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 19:17:50 smithi105 systemd[1]: Stopping Ceph mds.k for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:51.219 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 19:17:51 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k[65359]: 2024-03-26T19:17:51.112+0000 7f2cd41e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:51.219 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 19:17:51 smithi105 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k[65359]: 2024-03-26T19:17:51.112+0000 7f2cd41e0700 -1 mds.k *** got signal Terminated *** 2024-03-26T19:17:54.280 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 19:17:54 smithi105 podman[73237]: 2024-03-26 19:17:54.022517318 +0000 UTC m=+3.005855850 container died 5a839fe235117cadc51db139b09ff714318f3f7de9ba59a19d13e5bec6622f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T19:17:54.564 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 19:17:54 smithi105 podman[73237]: 2024-03-26 19:17:54.362951937 +0000 UTC m=+3.346290468 container cleanup 5a839fe235117cadc51db139b09ff714318f3f7de9ba59a19d13e5bec6622f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-26T19:17:54.564 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 26 19:17:54 smithi105 bash[73237]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-k 2024-03-26T19:17:54.778 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.k.service' 2024-03-26T19:17:54.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:54.811 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-26T19:17:54.811 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-26T19:17:54.812 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.c 2024-03-26T19:17:55.132 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 19:17:54 smithi144 systemd[1]: Stopping Ceph mds.c for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:55.384 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 19:17:55 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c[66669]: 2024-03-26T19:17:55.131+0000 7f311a0e4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:55.385 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 19:17:55 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c[66669]: 2024-03-26T19:17:55.131+0000 7f311a0e4700 -1 mds.c *** got signal Terminated *** 2024-03-26T19:17:58.341 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 19:17:58 smithi144 podman[73161]: 2024-03-26 19:17:58.024844282 +0000 UTC m=+3.011579438 container died a57bdb37035e9fdb40cf5a94ac8658839d25d98d3ef76755e72032ad1d79ff5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True) 2024-03-26T19:17:58.634 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 19:17:58 smithi144 podman[73161]: 2024-03-26 19:17:58.414974531 +0000 UTC m=+3.401709674 container cleanup a57bdb37035e9fdb40cf5a94ac8658839d25d98d3ef76755e72032ad1d79ff5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-26T19:17:58.634 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 19:17:58 smithi144 bash[73161]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-c 2024-03-26T19:17:58.839 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.c.service' 2024-03-26T19:17:58.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:17:58.877 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-26T19:17:58.877 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-26T19:17:58.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.f 2024-03-26T19:17:59.194 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 19:17:58 smithi144 systemd[1]: Stopping Ceph mds.f for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:17:59.634 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 19:17:59 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f[66360]: 2024-03-26T19:17:59.193+0000 7fe182244700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:17:59.634 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 19:17:59 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f[66360]: 2024-03-26T19:17:59.193+0000 7fe182244700 -1 mds.f *** got signal Terminated *** 2024-03-26T19:18:02.303 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 19:18:02 smithi144 podman[73293]: 2024-03-26 19:18:02.03108596 +0000 UTC m=+2.937103883 container died 81527c8e835fe449f644c5561d81b36ffc45561ee6e6b745fb060ffa2f2ed3bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T19:18:02.620 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 19:18:02 smithi144 podman[73293]: 2024-03-26 19:18:02.39420725 +0000 UTC m=+3.300225173 container cleanup 81527c8e835fe449f644c5561d81b36ffc45561ee6e6b745fb060ffa2f2ed3bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-26T19:18:02.620 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 19:18:02 smithi144 bash[73293]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-f 2024-03-26T19:18:02.762 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.f.service' 2024-03-26T19:18:02.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:18:02.798 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-26T19:18:02.799 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-26T19:18:02.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.i 2024-03-26T19:18:03.134 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 19:18:02 smithi144 systemd[1]: Stopping Ceph mds.i for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:18:03.134 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 19:18:03 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i[66978]: 2024-03-26T19:18:03.114+0000 7f461ed8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:18:03.134 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 19:18:03 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i[66978]: 2024-03-26T19:18:03.114+0000 7f461ed8c700 -1 mds.i *** got signal Terminated *** 2024-03-26T19:18:06.290 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 19:18:06 smithi144 podman[73416]: 2024-03-26 19:18:06.030450076 +0000 UTC m=+3.012990360 container died a28535586c94ea935e3b1c8ff7d2dc7eaa5d237a526b22398cf97ef34215217d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-26T19:18:06.634 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 19:18:06 smithi144 podman[73416]: 2024-03-26 19:18:06.381186866 +0000 UTC m=+3.363727122 container cleanup a28535586c94ea935e3b1c8ff7d2dc7eaa5d237a526b22398cf97ef34215217d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319) 2024-03-26T19:18:06.634 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 19:18:06 smithi144 bash[73416]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-i 2024-03-26T19:18:06.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.i.service' 2024-03-26T19:18:06.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:18:06.834 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-26T19:18:06.834 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-26T19:18:06.834 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.l 2024-03-26T19:18:07.384 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:18:06 smithi144 systemd[1]: Stopping Ceph mds.l for 2172d50a-eba1-11ee-95d0-87774f69a715... 2024-03-26T19:18:07.384 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:18:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l[66051]: 2024-03-26T19:18:07.142+0000 7f5538798700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T19:18:07.385 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:18:07 smithi144 ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l[66051]: 2024-03-26T19:18:07.142+0000 7f5538798700 -1 mds.l *** got signal Terminated *** 2024-03-26T19:18:10.302 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:18:10 smithi144 podman[73536]: 2024-03-26 19:18:10.032889094 +0000 UTC m=+2.980125813 container died ca0f9421c468d7abd974c9aaea97fb818f0edf061f8349b595668f98b9f7d2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-26T19:18:10.634 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:18:10 smithi144 podman[73536]: 2024-03-26 19:18:10.376370706 +0000 UTC m=+3.323607414 container cleanup ca0f9421c468d7abd974c9aaea97fb818f0edf061f8349b595668f98b9f7d2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-26T19:18:10.634 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:18:10 smithi144 bash[73536]: ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l 2024-03-26T19:18:10.635 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 19:18:10 smithi144 podman[73550]: 2024-03-26 19:18:10.54398142 +0000 UTC m=+0.503432415 container remove ca0f9421c468d7abd974c9aaea97fb818f0edf061f8349b595668f98b9f7d2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-2172d50a-eba1-11ee-95d0-87774f69a715-mds-l, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-26T19:18:10.672 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2172d50a-eba1-11ee-95d0-87774f69a715@mds.l.service' 2024-03-26T19:18:10.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T19:18:10.706 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-26T19:18:10.707 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T19:18:11.027 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T19:18:15.746 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T19:18:15.939 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T19:18:18.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T19:18:18.673 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T19:18:21.141 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T19:18:21.250 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T19:18:21.284 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T19:18:21.313 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-26T19:18:21.315 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817/remote/smithi097/crash 2024-03-26T19:18:21.316 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/crash -- . 2024-03-26T19:18:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T19:18:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-03-26T19:18:21.426 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817/remote/smithi105/crash 2024-03-26T19:18:21.427 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/crash -- . 2024-03-26T19:18:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T19:18:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now 2024-03-26T19:18:21.469 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817/remote/smithi144/crash 2024-03-26T19:18:21.470 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/crash -- . 2024-03-26T19:18:21.515 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T19:18:21.515 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-03-26T19:18:21.516 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-26T19:18:21.516 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v '\(PG_DEGRADED' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-26T19:18:21.626 INFO:tasks.cephadm:Compressing logs... 2024-03-26T19:18:21.627 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T19:18:21.631 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T19:18:21.634 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T19:18:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T19:18:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:: No such file or directory 2024-03-26T19:18:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.l.log 2024-03-26T19:18:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T19:18:21.663 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.i.log 2024-03-26T19:18:21.663 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.11.log 2024-03-26T19:18:21.663 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mgr.z.log 2024-03-26T19:18:21.663 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.8.log 2024-03-26T19:18:21.663 INFO:teuthology.orchestra.run.smithi144.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T19:18:21.663 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-26T19:18:21.663 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.f.log 2024-03-26T19:18:21.663 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T19:18:21.667 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.f.log: 90.4% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T19:18:21.667 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.log 2024-03-26T19:18:21.668 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.cephadm.log: 82.1% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T19:18:21.668 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-26T19:18:21.670 INFO:teuthology.orchestra.run.smithi105.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:: No such file or directory 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.10.log 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.b.log 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.k.log 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.h.log 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.e.log 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr: 90.4% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.audit.log.gz/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.7.log: 2024-03-26T19:18:21.671 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-26T19:18:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.log: 87.0% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T19:18:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.c.log 2024-03-26T19:18:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.2.log: 93.2% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mgr.z.log.gz 2024-03-26T19:18:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T19:18:21.679 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T19:18:21.683 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-26T19:18:21.686 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.log 2024-03-26T19:18:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T19:18:21.695 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T19:18:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.log: 93.1% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-26T19:18:21.697 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T19:18:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-26T19:18:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-26T19:18:21.699 INFO:teuthology.orchestra.run.smithi105.stderr: 82.1% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T19:18:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T19:18:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.g.log 2024-03-26T19:18:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.j.log 2024-03-26T19:18:21.704 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-26T19:18:21.705 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.j.log: gzip -5 90.2% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T19:18:21.705 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-26T19:18:21.707 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-26T19:18:21.708 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mgr.x.log: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T19:18:21.709 INFO:teuthology.orchestra.run.smithi105.stderr: 87.0% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T19:18:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T19:18:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.0.log: 94.4% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.g.log.gz 2024-03-26T19:18:21.711 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T19:18:21.711 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-26T19:18:21.712 INFO:teuthology.orchestra.run.smithi097.stderr: 94.4% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.j.log.gz 2024-03-26T19:18:21.712 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.log 2024-03-26T19:18:21.714 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T19:18:21.714 INFO:teuthology.orchestra.run.smithi097.stderr: 82.5% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T19:18:21.715 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.9.log 2024-03-26T19:18:21.719 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.9.log: 87.0% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T19:18:21.719 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.d.log 2024-03-26T19:18:21.722 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.a.log 2024-03-26T19:18:21.724 INFO:teuthology.orchestra.run.smithi144.stderr: 94.1% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.i.log.gz 2024-03-26T19:18:21.732 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.a.log: 90.8% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T19:18:21.738 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T19:18:21.745 INFO:teuthology.orchestra.run.smithi144.stderr: 94.7% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.l.log.gz 2024-03-26T19:18:21.746 INFO:teuthology.orchestra.run.smithi105.stderr: 94.6% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.k.log.gz 2024-03-26T19:18:21.755 INFO:teuthology.orchestra.run.smithi105.stderr: 94.1% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.e.log.gz 2024-03-26T19:18:21.764 INFO:teuthology.orchestra.run.smithi144.stderr: 94.6% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.f.log.gz 2024-03-26T19:18:21.765 INFO:teuthology.orchestra.run.smithi144.stderr: 94.8% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.c.log.gz 2024-03-26T19:18:21.803 INFO:teuthology.orchestra.run.smithi097.stderr: 93.7% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.a.log.gz 2024-03-26T19:18:21.832 INFO:teuthology.orchestra.run.smithi097.stderr: 93.5% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.d.log.gz 2024-03-26T19:18:22.071 INFO:teuthology.orchestra.run.smithi097.stderr: 90.2% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-26T19:18:22.116 INFO:teuthology.orchestra.run.smithi144.stderr: 91.7% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-26T19:18:22.336 INFO:teuthology.orchestra.run.smithi105.stderr: 91.8% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-26T19:18:22.371 INFO:teuthology.orchestra.run.smithi105.stderr: 94.2% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.h.log.gz 2024-03-26T19:18:22.772 INFO:teuthology.orchestra.run.smithi097.stderr: 90.1% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-26T19:18:27.255 INFO:teuthology.orchestra.run.smithi105.stderr: 93.7% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-mds.b.log.gz 2024-03-26T19:18:50.704 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.9.log.gz 2024-03-26T19:18:54.820 INFO:teuthology.orchestra.run.smithi144.stderr: 91.1% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.8.log.gz 2024-03-26T19:18:58.449 INFO:teuthology.orchestra.run.smithi105.stderr: 91.7% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-26T19:19:02.671 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-26T19:19:05.441 INFO:teuthology.orchestra.run.smithi105.stderr: 91.0% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.10.log.gz 2024-03-26T19:19:05.598 INFO:teuthology.orchestra.run.smithi144.stderr: 91.2% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-26T19:19:06.958 INFO:teuthology.orchestra.run.smithi144.stderr: 91.1% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-26T19:19:08.822 INFO:teuthology.orchestra.run.smithi144.stderr: 90.9% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.11.log.gz 2024-03-26T19:19:08.824 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-03-26T19:19:08.824 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m47.176s 2024-03-26T19:19:08.824 INFO:teuthology.orchestra.run.smithi144.stderr:user 2m44.777s 2024-03-26T19:19:08.824 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m5.134s 2024-03-26T19:19:14.165 INFO:teuthology.orchestra.run.smithi105.stderr: 91.0% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-26T19:19:16.337 INFO:teuthology.orchestra.run.smithi097.stderr: 90.9% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-26T19:19:17.774 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-26T19:19:17.779 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-26T19:19:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m56.115s 2024-03-26T19:19:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:user 2m49.940s 2024-03-26T19:19:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m12.301s 2024-03-26T19:19:21.308 INFO:teuthology.orchestra.run.smithi105.stderr: 91.0% -- replaced with /var/log/ceph/2172d50a-eba1-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-26T19:19:21.310 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-26T19:19:21.310 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m59.662s 2024-03-26T19:19:21.310 INFO:teuthology.orchestra.run.smithi105.stderr:user 3m13.496s 2024-03-26T19:19:21.311 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m5.854s 2024-03-26T19:19:21.311 INFO:tasks.cephadm:Archiving logs... 2024-03-26T19:19:21.311 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817/remote/smithi097/log 2024-03-26T19:19:21.312 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T19:19:51.729 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817/remote/smithi105/log 2024-03-26T19:19:51.730 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T19:20:25.523 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817/remote/smithi144/log 2024-03-26T19:20:25.524 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T19:20:55.307 INFO:tasks.cephadm:Removing cluster... 2024-03-26T19:20:55.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 --force 2024-03-26T19:20:55.634 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T19:20:56.914 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 --force 2024-03-26T19:20:57.110 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T19:20:57.667 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2172d50a-eba1-11ee-95d0-87774f69a715 --force 2024-03-26T19:20:57.857 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 2172d50a-eba1-11ee-95d0-87774f69a715 2024-03-26T19:20:58.445 INFO:tasks.cephadm:Removing cephadm ... 2024-03-26T19:20:58.445 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T19:20:58.503 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T19:20:58.523 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T19:20:58.538 INFO:tasks.cephadm:Teardown complete 2024-03-26T19:20:58.539 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-26T19:20:58.549 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-26T19:20:58.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T19:20:58.557 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T19:20:58.565 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T19:20:58.711 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-26T19:20:58.721 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-26T19:20:58.721 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T19:20:58.729 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T19:20:58.732 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T19:20:58.745 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-26T19:20:58.745 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 +639us[ +688us] +/- 69ms 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -861us[ -811us] +/- 79ms 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +201us[ +250us] +/- 84ms 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +666us[ +666us] +/- 69ms 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv02.front.sepia.ceph.com 3 6 377 31 -849us[ -778us] +/- 79ms 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 +216us[ +216us] +/- 84ms 2024-03-26T19:20:58.751 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T19:20:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-26T19:20:58.781 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T19:20:58.781 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-26T19:20:58.781 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +656us[ +656us] +/- 69ms 2024-03-26T19:20:58.781 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -827us[ -774us] +/- 79ms 2024-03-26T19:20:58.781 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +230us[ +230us] +/- 84ms 2024-03-26T19:20:58.781 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T19:20:58.782 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-26T19:20:58.792 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-26T19:20:58.793 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-26T19:20:58.802 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T19:20:59.030 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T19:20:59.107 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T19:20:59.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T19:20:59.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T19:20:59.284 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T19:20:59.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T19:20:59.317 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T19:20:59.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T19:20:59.348 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-26T19:20:59.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-26T19:20:59.401 INFO:teuthology.task.internal:Duration was 2612.733570 seconds 2024-03-26T19:20:59.401 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-26T19:20:59.490 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-26T19:20:59.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T19:20:59.498 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T19:20:59.501 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T19:20:59.544 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T19:20:59.546 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T19:20:59.661 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T19:20:59.937 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-26T19:20:59.937 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-03-26T19:20:59.937 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T19:20:59.999 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-03-26T19:20:59.999 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T19:21:00.031 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-03-26T19:21:00.031 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T19:21:00.061 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-26T19:21:00.062 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T19:21:00.068 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T19:21:00.073 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T19:21:00.159 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-26T19:21:00.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T19:21:00.166 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T19:21:00.168 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T19:21:00.675 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-26T19:21:00.688 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-26T19:21:00.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T19:21:00.785 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T19:21:00.813 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T19:21:00.841 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-26T19:21:00.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T19:21:00.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T19:21:00.862 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T19:21:00.885 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-03-26T19:21:00.907 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-03-26T19:21:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-03-26T19:21:01.017 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T19:21:01.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T19:21:01.070 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T19:21:01.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T19:21:01.084 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T19:21:01.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T19:21:01.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-26T19:21:01.111 INFO:teuthology.task.internal:Transferring archived files... 2024-03-26T19:21:01.112 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817/remote/smithi097 2024-03-26T19:21:01.112 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T19:21:01.229 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817/remote/smithi105 2024-03-26T19:21:01.229 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T19:21:01.270 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817/remote/smithi144 2024-03-26T19:21:01.270 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T19:21:01.309 INFO:teuthology.task.internal:Removing archive directory... 2024-03-26T19:21:01.309 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T19:21:01.317 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T19:21:01.319 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T19:21:01.367 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-26T19:21:01.381 INFO:teuthology.task.internal:Not uploading archives. 2024-03-26T19:21:01.381 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-26T19:21:01.412 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-26T19:21:01.412 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T19:21:01.419 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T19:21:01.422 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T19:21:01.434 INFO:teuthology.orchestra.run.smithi105.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 19:21 /home/ubuntu/cephtest 2024-03-26T19:21:01.435 INFO:teuthology.orchestra.run.smithi144.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 19:21 /home/ubuntu/cephtest 2024-03-26T19:21:01.466 INFO:teuthology.orchestra.run.smithi097.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 19:21 /home/ubuntu/cephtest 2024-03-26T19:21:01.470 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-26T19:21:01.491 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-26T19:21:02.114 INFO:teuthology.nuke:Checking targets against current locks 2024-03-26T19:21:02.145 DEBUG:teuthology.nuke:shortname: smithi097 2024-03-26T19:21:02.145 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T19:21:02.174 DEBUG:teuthology.nuke:shortname: smithi105 2024-03-26T19:21:02.175 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T19:21:02.180 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 18:28:56.892677', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPXlOA3/hBjX3/UQCMdIc5Kxd+dwYhKV8kmzAPyFhNV1/16javNIxvKzL/6xmEIH+E05aA2BJsJRk+ixSyyKRAI='} 2024-03-26T19:21:02.196 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 18:28:56.890783', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGNqjiTX4EDAoMqp7oUa40hM1jCxmPcujTGj3Lozueao4FyU22SfY4slt1lzC5riz62Gi6SLNeifivbsHUWzD1o='} 2024-03-26T19:21:02.200 DEBUG:teuthology.nuke:shortname: smithi144 2024-03-26T19:21:02.201 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T19:21:02.223 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623817', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 18:28:56.894297', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJiep7pffesW1SUf7ISWXkdHXqUioyvL9H9E9ZF5bhCQeyN7YaNCXoEDefv+0J7Wu99Vv6+gxSoISwXRBVaMDyI='} 2024-03-26T19:21:02.232 INFO:teuthology.orchestra.console.smithi097:Power off 2024-03-26T19:21:02.232 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T19:21:02.253 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-03-26T19:21:02.254 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:02.273 INFO:teuthology.orchestra.console.smithi105:Power off 2024-03-26T19:21:02.273 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T19:21:02.285 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-26T19:21:02.295 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2024-03-26T19:21:02.295 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:02.309 INFO:teuthology.orchestra.console.smithi144:Power off 2024-03-26T19:21:02.310 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T19:21:02.319 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-26T19:21:02.331 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-03-26T19:21:02.331 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:02.356 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-26T19:21:06.286 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:06.321 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:06.357 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:06.398 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-26T19:21:06.430 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-26T19:21:06.471 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-26T19:21:10.403 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:10.431 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:10.472 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:10.513 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-26T19:21:10.541 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-26T19:21:10.582 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-26T19:21:14.518 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:14.542 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:14.583 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T19:21:14.629 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-03-26T19:21:14.653 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-03-26T19:21:14.693 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-03-26T19:21:14.730 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-03-26T19:21:14.753 INFO:teuthology.orchestra.console.smithi105:Power off completed 2024-03-26T19:21:14.794 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-03-26T19:21:14.899 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/postgres}} duration: 2612.733570098877 failure_reason: 'The following counters failed to be set on mds daemons: {''mds.exported'', ''mds.imported''}' flavor: default owner: scheduled_lflores@teuthology status: fail success: false 2024-03-26T19:21:14.899 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T19:21:14.975 INFO:teuthology.run:FAIL